CA3166091A1 - An identification method, device computer equipment and storage medium for identity document reproduction - Google Patents

An identification method, device computer equipment and storage medium for identity document reproduction

Info

Publication number
CA3166091A1
CA3166091A1 CA3166091A CA3166091A CA3166091A1 CA 3166091 A1 CA3166091 A1 CA 3166091A1 CA 3166091 A CA3166091 A CA 3166091A CA 3166091 A CA3166091 A CA 3166091A CA 3166091 A1 CA3166091 A1 CA 3166091A1
Authority
CA
Canada
Prior art keywords
image
certificate
rephotographing
prediction
identified
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CA3166091A
Other languages
French (fr)
Inventor
Zhiguan WANG
Yang GU
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
10353744 Canada Ltd
Original Assignee
10353744 Canada Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 10353744 Canada Ltd filed Critical 10353744 Canada Ltd
Publication of CA3166091A1 publication Critical patent/CA3166091A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The present application relates to a method of identifying re-photography of certificates, and corresponding device, computer equipment and storage medium. The method comprises: obtaining a certificate image to be identified; employing a well-trained first convolutional neural network to process the certificate image, and obtaining corresponding overall image rephotographing prediction value; rectifying the certificate image, obtaining a rectified certificate image, preprocessing it to obtain an overall feature map and local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values corresponding to the overall feature map and the various local feature maps, and obtaining a local image rephotographing prediction value of the certificate image to be identified; and determining a re-photography identifying result according to the overall image rephotographing prediction value and the local image rephotographing prediction value. Embodiments of the present invention enable enhancement of re-photography identifying precision rate of certificate images.

Description

AN IDENTIFICATION METHOD, DEVICE COMPUTER EQUIPMENT AND
STORAGE MEDIUM FOR IDENTITY DOCUMENT REPRODUCTION
BACKGROUND OF THE INVENTION
Technical Field [0001] The present application relates to the technical field of image classification, and more particularly to a method of identifying re-photography of a certificate, and corresponding device, computer equipment and storage medium.
Description of Related Art
[0002] The following statements merely supply background information related to the present invention, and do not necessarily make up the prior-art technology.
[0003] Identification of screen-rephotographing of ID cards aims to judge whether an ID card was rephotographed through the screen by means of such techniques as computer vision, artificial intelligence, and image processing, etc. People have been endeavoring for many years to extract such features as spectrum features, texture features, and color features, etc., by traditional methods, and to thereafter carry out classification through a support vector machine, but manual extraction of features is required in the traditional methods, and it is thus relatively subjective and somehow restrictive, whereby erroneous identification rate is rendered relatively high.
[0004] With the rapid development of artificial intelligence and the advent of deep neural networks, people start to enable computers to learn features by themselves through model training. Features acquired by computers through deep learning are more reliable. In view of this, the model for identifying screen-rephotographing of ID cards that is based on deep learning has been proposed. In comparison with traditional methods, the model for identifying screen-rephotographing of ID cards that is based on deep learning achieves lower erroneous identification rate, but the precision rate of this model is not so high with respect to some rather special certificate pictures, for instance, with respect to some rephotographed pictures whose actual non-rephotographed backgrounds occupy larger Date Regue/Date Received 2022-06-29 proportions except for portions in the screens, the model would erroneously identify them as authentic pictures, whereas with respect to certain authentic certificate pictures under background in which reflection of light exists, such as glass and glossy whiteboards, etc., due to higher illumination during photographing, the model would erroneously identify them as rephotographed pictures.
SUMMARY OF THE INVENTION
[0005] In view of the above shortcomings or deficiencies, the present invention provides a method of identifying re-photography of a certificate, and corresponding device, computer equipment and storage medium, and embodiments of the present invention enable further enhancement of re-photography identifying precision rate of certificate images.
[0006] According to the first aspect, the present invention provides a method of identifying re-photography of a certificate. In one embodiment, the method comprises:
[0007] obtaining a certificate image to be identified;
[0008] employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds;
[0009] rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds; and
[0010] determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be Date Regue/Date Received 2022-06-29 identified corresponds.
[0011] In one embodiment, the step of rectifying the certificate image to be identified and obtaining a rectified certificate image includes:
[0012] employing a picture splitting model to extract a mask image of the certificate image to be identified, wherein the mask image includes a certificate region and a background region;
[0013] employing a comer point detecting algorithm to detect a plurality of corner points in the mask image;
[0014] employing a preset clustering algorithm to determine four cluster centers from the plural comer points to serve as four comer points of the certificate region; and
[0015] performing a perspective transformation process on the certificate region according to the four comer points of the certificate region, and obtaining the rectified certificate image.
[0016] In one embodiment, the step of preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds includes:
[0017] processing the rectified certificate image into a feature map, and taking the feature map as an overall feature map;
[0018] splitting the feature map on the basis of a preset region splitting scale, and obtaining a plurality of local feature maps;
[0019] performing a pooling process on the overall feature map and the various local feature maps, and obtaining a global feature vector extracted from the overall feature map and local feature vectors extracted from the various local feature maps; and
[0020] performing a classification prediction process on the global feature vector and the various local feature vectors, obtaining a rephotographing prediction value to which the overall Date Regue/Date Received 2022-06-29 feature map corresponds and rephotographing prediction values to which the various local feature maps correspond, and taking the rephotographing prediction value corresponding to the overall feature map and the rephotographing prediction values corresponding to the plural local feature maps as local image rephotographing prediction values to which the certificate image to be identified corresponds.
[0021] In one embodiment, the second convolutional neural network is a convolutional neural network that employs a ResNet18 network as a backbone network, wherein the backbone network includes a first predicting module, and a plurality of second predicting modules;
[0022] the backbone network is employed to process the rectified certificate image into feature maps and includes the first four submodules of the ResNet18 network, wherein stride of a first convolution layer in the fourth submodule of the ResNet18 network is altered as 1;
[0023] a splitting module is employed to, on basis of the preset region splitting scale, split a feature map obtained by processing of a backbone module into a plurality of local feature maps, and to respectively input the plural local feature maps into the plural predicting modules, wherein the number of the predicting modules is identical with the number of the local feature maps;
[0024] the first predicting module includes a convolution layer, an average pooling layer, a full connection layer and a classifier, and the first predicting module is employed for performing a pooling process on the input overall feature map to obtain a global feature vector, and performing a classification prediction process on the obtained global feature vector to obtain a rephotographing prediction value to which the input overall feature map corresponds;
[0025] the second predicting module each includes a convolution layer, a global pooling layer, a full connection layer and a classifier, and the second predicting module is employed for performing a pooling process on an input local feature map to obtain a local feature vector, and performing a classification prediction process on the obtained local feature vector to obtain a rephotographing prediction value to which the input local feature map Date Regue/Date Received 2022-06-29 corresponds.
[0026] In one embodiment, the convolution layer included in the second predicting module is a convolution layer of 512*1*1*128 dimensions; and the full connection layer included in the second predicting module is a full connection layer of 128*2 dimensions.
[0027] In one embodiment, a process of training the second convolutional neural network includes:
[0028] obtaining plural rephotographed certificate images and non-rephotographed certificate images to serve as training samples;
[0029] constructing an initial second convolutional neural network whose parameters are random values; and
[0030] employing the training samples to iteratively train the initial second convolutional neural network, wherein, after each training, a prediction loss value of the current training is calculated according to a local image rephotographing prediction value generated by the initial second convolutional neural network in the current training, and then it is judged whether a loss of the initial second convolutional neural network is reduced to a minimum value according to the prediction loss value of the current training and prediction loss values of plural rounds of previous trainings, if yes, training is decided to be completed, and the initial second convolutional neural network at the end of the training is taken to serve as a well-trained second convolutional neural network; if not, a gradient of a loss function is calculated, network parameters of the initial second convolutional neural network are updated by means of a gradient descending method, and the next training is carried out.
[0031] In one embodiment, the step of determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds includes:
[0032] calculating a mean value of the overall image rephotographing prediction value and the Date Regue/Date Received 2022-06-29 local image rephotographing prediction value to which the certificate image to be identified corresponds; and
[0033] comparing the mean value with a preset threshold, determining the re-photography identifying result, to which the certificate image to be identified corresponds, as rephotographed when it is determined that the mean value is greater than the preset threshold, and determining the re-photography identifying result, to which the certificate image to be identified corresponds, as not rephotographed when it is determined that the mean value is not greater than the preset threshold.
[0034] According to the second aspect, the present invention provides a device for identifying re-photography of a certificate. In one embodiment, the device comprises:
[0035] an image obtaining module, for obtaining a certificate image to be identified;
[0036] a first predicting module, for employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds;
[0037] a second predicting module, for rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds; and
[0038] an identification result determining module, for determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds.

Date Regue/Date Received 2022-06-29
[0039] According to the third aspect, the present invention provides a computer equipment that comprises a memory, a processor, and a computer program stored on the memory and operable on the processor, and steps of the method according to anyone of the foregoing embodiments are realized when the processor executes the computer program.
[0040] According to the fourth aspect, the present invention provides a computer-readable storage medium storing a computer program thereon, and steps of the method according to anyone of the foregoing embodiments are realized when the computer program is executed by a processor.
[0041] In the embodiments of the present invention, a well-trained first convolutional neural network is firstly used to process the certificate image to be identified to obtain an corresponding overall image rephotographing prediction value, which is then subjected to a rectifying process to obtain a rectified certificate image, after that the rectified certificate image is preprocessed to obtain an overall feature map and a plurality of local feature maps; a well-trained second convolutional neural network is thereafter used to respectively calculate rephotographing prediction values, to which the overall feature map and the various local feature maps correspond, to obtain a local image rephotographing prediction value to which the certificate image to be identified corresponds, and a re-photography identifying result to which the certificate image to be identified corresponds is hence determined according to the overall image rephotographing prediction value and the local image rephotographing prediction value.
The embodiments take global information and local information of the certificate image to be identified into consideration, merge the two together, and weaken the influence of the background in the image, whereby is achieved more precise and more reliable differentiation as to whether the certificate image is a rephotographed image.
BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Fig. 1 is a flowchart schematically illustrating the method of identifying re-photography of a certificate in an embodiment;

Date Regue/Date Received 2022-06-29
[0043] Fig. 2 is a flowchart schematically illustrating rectifying the certificate image to be identified in an embodiment;
[0044] Fig. 3 (a) is a view schematically illustrating a certificate image to be identified in an embodiment;
[0045] Fig. 3 (b) is a view schematically illustrating a mask image obtained in an embodiment;
[0046] Fig. 4 (a) is a view schematically illustrating a plurality of corner points detected and obtained from the mask image in an embodiment;
[0047] Fig. 4 (b) is a view schematically illustrating four cluster centers in an embodiment;
[0048] Fig. 4 (c) is a view schematically illustrating a rectified certificate image in an embodiment;
[0049] Fig. 5 is a flowchart schematically illustrating obtaining a local rephotographing prediction value in an embodiment;
[0050] Fig. 6 is a view schematically illustrating the comparison between the fourth submodule in the original ResNet18 with the backbone module in an embodiment;
[0051] Fig. 7 is a view schematically illustrating the region splitting scale in an embodiment;
[0052] Fig. 8 is a view illustrating the architecture of the second convolutional neural network in an embodiment;
[0053] Fig. 9 is a flowchart schematically illustrating training of the second convolutional neural network in an embodiment;
[0054] Fig. 10 is a block diagram illustrating the structure of the device for identifying re-photography of a certificate in an embodiment; and
[0055] Fig. 11 is a view illustrating the internal structure of the computer equipment in an embodiment.
DETAILED DESCRIPTION OF THE INVENTION

Date Regue/Date Received 2022-06-29
[0056] To make more lucid and clear the objectives, technical solutions and advantages of the present application, the present application is described in greater detail below in conjunction with accompanying drawings and embodiments. As should be understood, the specific embodiments described in this context are merely meant to explain the present application, rather than to restrict the present application.
[0057] The present invention provides a method of identifying re-photography of a certificate.
In one embodiment, the method of identifying re-photography of a certificate comprises steps as shown in Fig. 1, and the method is described in greater detail below.
[0058] S110 - obtaining a certificate image to be identified.
[0059] The method of identifying re-photography of a certificate provided by this embodiment can be applied to business scenarios of such fields as finance, insurance, e-commerce, and 020, etc., in which are required to identify whether certificate images are rephotographed images, wherein it is specifically possible to identify whether certificate images provided by users or settled merchants are rephotographed images in scenarios where it is required to perform real name authentication of ID cards of users or settled merchants to filter out fraudulent users or merchants, and to control business risks. The executing subject of the method can be such a computer equipment as a server, the certificate image to be identified can be collected by such an equipment as a pick-up head, a mobile phone, a camera capable of collecting images in real time, and the certificate image can be an ID card image, a social security card image, and so on.
[0060] 5120 - employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds.
[0061] In this embodiment, the first convolutional neural network is a convolutional neural network (CNN) that can be embodied by any network of the ResNet series (such as ResNet18, ResNet50, etc.), and can also be embodied as a ResNeXt network, an Inception-ResNet network, a VGG network, or an AlexNet network.

Date Regue/Date Received 2022-06-29
[0062] In one mode of execution, considering that ResNet18 is smaller, quicker in speed and capable of achieving the requirement on precision as compared with other ResNet networks, ResNet18 is employed to serve as the first convolutional neural network, and binary classification prediction is performed on the rectified certificate image directly thereby and overall loss is calculated through a loss function.
[0063] The process of training ResNet18 can be to firstly construct an initial ResNet18 network model and select training samples, and thereafter employ the training samples to train the network model. The training samples include rephotographed certificate images and non-rephotographed certificate images, when rephotographed certificate images are selected to serve as training samples, certificate images (images that contain certificates) containing computer screens and mobile phone screens as backgrounds can be selected, wherein the backgrounds in the screens should be diversified backgrounds that contain desktops, notebooks, and word documents as far as possible besides black and white colors, and the training data further includes pictures with other backgrounds than the screens; as for the non-rephotographed certificate images, they can be selected from pictures with tables and paper, etc., that are relatively similar to the screen-rephotographed background color.
[0064] When it is started to reiteratively train the network model, parameters of the entire network model are random numbers, during the process of each training, probabilities (namely prediction values) for the overall image to pertain to various categories can be calculated through forward propagation, a loss function, such as softmax, is then used to calculate the prediction loss value of the current training, it is hence judged whether the loss of the entire network has been reduced to a minimum value through the prediction loss value of the current training and prediction loss values calculated from several rounds of previous trainings, if it is determined that the loss is not reduced to the minimum value, parameter values of the network model are updated through reverse propagation (when the parameter values are updated, it is possible to calculate the gradient of the loss function, and to thereafter update by means of a stochastic gradient descent (SGD) Date Regue/Date Received 2022-06-29 method), and the next training is continuously performed, so on and so forth;
the loss of the network model is incessantly reduced through incessant trainings and updating of parameters (reduction of the loss of the entire network means that parameter selection is rather correct, and it is needed to further adjust the parameters, such as the learning speed), until it is determined that the loss of the entire network has been reduced to the minimum value, at which time parameters of the entire network model can be obtained, that is to say, a well-trained network model is obtained.
[0065] When it is judged whether the loss of the entire network has been reduced to a minimum value through the prediction loss value of the current training and the prediction loss values of several rounds of previous trainings, such judging is specifically based on the prediction loss values and loss differences of recent N rounds of trainings.
The recent N
rounds of trainings include the current training and previous N-1 rounds of trainings, and the number of rounds N can be adjusted according to requirements of different scenarios, for instance, the recent four rounds, five rounds, and more rounds of trainings can be observed; exemplarily, the loss difference of each training can be derived by subtracting the prediction loss value of the previous training from the prediction loss value of the current training. If loss differences of recent N rounds of trainings are no longer changed, i.e., the various differences are all zero, or the differences are substantially unchanged, i.e., the loss differences of recent N rounds of trainings are fluctuating but the minimum prediction loss value is not being reduced, it is then determined that the loss of the entire network has been reduced to the minimum value, and the training can be completed, otherwise it is determined that the loss of the entire network has not been reduced to the minimum value, and it is required to continue training, for instance, the loss differences of several rounds of previous trainings are all zero, but the prediction loss value of the current training is still reduced much as compared with the prediction loss value of the previous training, it is then required to continue training.
[0066] Taking for example the loss function being softmax, the prediction probability for an input image to pertain to each category (namely rephotographed and non-rephotographed) Date Regue/Date Received 2022-06-29 can be calculated through the following formula:
.04) ezk /(Ej e")
[0067]
[0068] where z is an input to the softmax layer, and f(z) is the output.
[0069] The softmax loss is calculated through the following formula:
/(y, yelogi f(z,))
[0070]
[0071] where y is a true value, f(z) is a prediction probability for the input image to pertain to the corresponding category as derived after passing through the softmax layer, and 1 is a prediction loss value (for expressing the difference between the prediction value and the true value).
[0072] The certificate image to be identified is input into the well-trained network model, the network model will output a prediction value (namely an overall image rephotographing prediction value) to which the image corresponds, and the prediction value can be restricted in the range [0, 11 and is used for expressing the probability of the image being a rephotographed image.
[0073] S130 - rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds.
[0074] In this embodiment, when the certificate image to be identified is rectified, the mask of the certificate image to be identified is firstly extracted, four comer points in the mask are then identified, and rectification is achieved by performing perspective transformation Date Regue/Date Received 2022-06-29 on the mask through the corner points.
[0075] In this embodiment, the second convolutional neural network is not the currently available convolutional neural network (CNN), but a novel network proposed by this embodiment to address shortcomings in the state of the art. It can employ any network of the ResNet series (such as ResNet18, ResNet50), or a ResNeXt network, an Inception-ResNet network, a VGG network, or an AlexNet network to serve as the backbone network, and is subsequently added with the corresponding convolution layer, global pooling layer, full connection layer, and softmax, so as to process the rectified certificate image into a feature map, to split the feature map into a plurality of local feature maps, to hence respectively calculate and obtain rephotographing prediction values to which the various local feature maps correspond, and to take the feature map as an overall feature map and calculate its corresponding rephotographing prediction value, whereby the rephotographing prediction values to which the overall feature map and the various local feature maps correspond are precisely the local image rephotographing prediction value to which the certificate image to be identified corresponds.
[0076] S140 - determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds.
[0077] In this embodiment, when the re-photography identifying result to which the certificate image to be identified corresponds is determined according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds, it is required to integrate the overall image rephotographing prediction value and the local image rephotographing prediction value to obtain a comprehensive value, wherein the comprehensive value obtained by such integration can be a mean value calculated from the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds, the mean value is then Date Regue/Date Received 2022-06-29 compared with a preset threshold, and the re-photography identifying result to which the certificate image to be identified corresponds is determined as rephotographed when it is determined that the mean value is greater than the preset threshold, alternatively, the re-photography identifying result to which the certificate image to be identified corresponds is determined as not rephotographed when it is determined that the mean value is not greater than the preset threshold. Of course, it is also possible, when the comprehensive value is integrated and obtained, to multiply the overall image rephotographing prediction value and the local image rephotographing prediction value respectively with their corresponding preset weight values (such as 0.4 or 0. 6, which can be specifically adjusted as practically required), and to then summate the results to obtain the comprehensive value.
[0078] In this embodiment, by processing with a well-trained first convolutional neural network, a corresponding overall image rephotographing prediction value is obtained, the certificate image to be identified is rectified to obtain a rectified certificate image, which is then preprocessed to obtain an overall feature map and a plurality of local feature maps, a well-trained second convolutional neural network is thereafter employed to respectively calculate rephotographing prediction values, to which the overall feature map and the various local feature maps correspond, to obtain a local image rephotographing prediction value to which the certificate image to be identified corresponds, whereby are obtained the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds, and a re-photography identifying result to which the certificate image to be identified corresponds is hence determined according to the overall image rephotographing prediction value and the local image rephotographing prediction value. This embodiment takes global information and local information of the certificate image to be identified into consideration, and merges the two together to weaken the influence of the background in the image, whereby is achieved more precise and more reliable differentiation as to whether the certificate image is a rephotographed image.

Date Regue/Date Received 2022-06-29
[0079] In one embodiment, as shown in Fig. 2, the step of rectifying the certificate image to be identified and obtaining a rectified certificate image includes:
[0080] S121 - employing a picture splitting model to extract a mask image of the certificate image to be identified, wherein the mask image includes a certificate region and a background region;
[0081] S122 - employing a corner point detecting algorithm to detect a plurality of corner points in the mask image;
[0082] S123 - employing a preset clustering algorithm to determine four cluster centers from the plural corner points to serve as four corner points of the certificate region;
and
[0083] S124 - performing a perspective transformation process on the certificate region according to the four corner points of the certificate region, and obtaining the rectified certificate image.
[0084] The picture splitting model can be a certificate splitting network that is frequently used at present. In one mode of execution, the currently frequently used certificate splitting network is not ideal in the effect of certificate identification, it is therefore possible to employ a deep learning network framework UNet to perform certificate splitting; it is required to collect a training set when UNet is employed to perform certificate splitting, because it is rather difficult to collect the certificate images per se, that is to say, it is difficult to obtain a larger training set, so the method of employing UNet to perform certificate splitting is not common.
[0085] Selection of the UNet framework is due to its relative simplicity, smaller model size, and shorter prediction time. When the model is trained, considering that the model is mainly used for the identification of rephotographing, the same numbers of rephotographed certificate images and non-rephotographed certificate images are chosen at the same time, and the background, shooting angle, and light circumstances while photographing the various certificates are contained as far as possible. In addition, manually marked masks Date Regue/Date Received 2022-06-29 are taken to serve as actual labels, a binary cross entropy statistic model is used to calculate differences between prediction values and true values, and to calculate losses.
[0086] Exemplarily, the process of training a picture splitting model realized by UNet, namely a UNet model, can be as follows: when training is started, parameters of the entire model are random values, during the process of training, prediction labels (for instance, 0 indicates background, 1 indicates certificate) of different pixel points of the image can be calculated through forward propagation, losses of the prediction value and the true label value are thereafter calculated through binary cross entropy, parameter values of the model are then updated through reverse propagation to carry out the next training, the losses are incessantly reduced through iterative trainings, when the losses are reduced to a state of being substantially unchanged, parameters of the entire model can be obtained, namely a picture splitting model realized by UNet. As shown in Fig. 3(a), once the certificate image is input into the well-trained model, a mask image of the certificate image can be obtained, as shown in Fig. 3(b), in this mask image, the white portion is a certificate region, and the black portion is a background region.
[0087] The corner point detecting algorithm can be embodied as the Harris algorithm, through which corner point detection is performed on the mask image and a series of corner points can be obtained, as shown in Fig. 4(a).
[0088] The preset clustering algorithm can be the K-Means clustering algorithm. The K-Means clustering algorithm is employed to cluster and obtain four cluster centers from the series of corner points detected by the corner point detecting algorithm, as shown in Fig. 4(b), and the four cluster centers are then taken to serve as four corner points of the certificate region.
[0089] A perspective transformation process is finally performed on the certificate region on the basis of the four corner points of the certificate region, whereby the rectified certificate image can be obtained, as shown in Fig. 4(c).
[0090] In this embodiment, before the deep learning technique is employed to predict the Date Regue/Date Received 2022-06-29 certificate image to be identified, the certificate image to be identified is firstly rectified, whereby can be further enhanced the precision rate of rephotographing identification.
[0091] In one embodiment, as shown in Fig. 5, the step of preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds includes:
[0092] S131 - processing the rectified certificate image into a feature map, and taking the feature map as an overall feature map;
[0093] S132 - splitting the feature map on the basis of a preset region splitting scale, and obtaining a plurality of local feature maps;
[0094] S133 - performing a pooling process on the overall feature map and the various local feature maps, and obtaining a global feature vector extracted from the overall feature map and local feature vectors extracted from the various local feature maps; and
[0095] S134 - performing a classification prediction process on the global feature vector and the various local feature vectors, obtaining a rephotographing prediction value to which the overall feature map corresponds and rephotographing prediction values to which the various local feature maps correspond, and taking the rephotographing prediction value corresponding to the overall feature map and the rephotographing prediction values corresponding to the plural local feature maps as local image rephotographing prediction values to which the certificate image to be identified corresponds.
[0096] The above steps are described below with an example of the second convolutional neural network employing a ResNet18 network as the backbone network.
[0097] In this example, the second convolutional neural network employs a ResNet18 network as the backbone network that includes a plurality of predicting modules, which are specifically a first predicting module, and a plurality of second predicting modules.

Date Regue/Date Received 2022-06-29
[0098] The backbone network is employed to process the rectified certificate image into feature maps, it consists of the first four submodules of the ResNet18 network, and in the backbone module, stride of a first convolution layer in the fourth submodule of the ResNet18 network is altered as 1. In other words, spatial down-sampling in the fourth submodule in the original ResNet18 network is deleted, and the last global pooling layer and full connection layer in original ResNet18 network are deleted.
Exemplarily, the fourth submodule and the following global pooling layer (GAP) and full connection layer (fc) in the original ResNet18 network can be as shown in (a) of Fig. 6, while the backbone module in the second convolutional neural network can be as shown in (b) of Fig. 6, in (a), stride=2 in the first convolution layer in the fourth submodule aims to perform the spatial down-sampling operation, and the spatial down-sampling operation can be deleted when it is altered as 1.
[0099] After the backbone network has processed the certificate image to be identified into a feature map, it can be split into a plurality of local feature maps through a preset splitting module. The splitting module is employed to split the feature map processed and obtained by the backbone network into a plurality of local feature maps on the basis of a preset splitting scale (that can be set as practically required). Exemplarily, taking for example the certificate image being an ID card, it is possible to split the image in accordance with the region splitting scale shown in Fig. 7, as it is thus possible to split the entire certificate region into three portions, the first portion is a basic information region of the person, the second portion is an ID card head poi _____________________________________ ti ait region, and the third portion is an ID card number region. Suppose the ID card has a width of w and a height of h, through coarse estimation, the first portion as the basic information region has a width of w1=2/3w and a height of h1=4/5h, the second portion as the head poi ___________________ hait region has a width of w2=1/3w and a height of h2=4/5h, and the third portion as the ID card number region has a width of w3=w and a height of h3=1/5h.
[0100] After the above splitting process has been completed, the plural local feature maps obtained by such splitting and the feature map that is not split (serving as the overall Date Regue/Date Received 2022-06-29 feature map) are respectively input into the plural predicting modules, the overall feature map is processed by the first predicting module, and the various local feature maps are processed by the various second predicting modules, so the number of the predicting modules is the same as the number of the local feature maps. Each predicting module includes a convolution layer, a pooling layer, a full connection layer, and a classifier (softmax), and each predicting module is employed to perform a pooling process on the input feature map, to obtain a feature vector (a global feature vector can be obtained by processing the overall feature map, and a local feature vector can be obtained by processing a local feature map), to perform a classification prediction process on the obtained feature vector, and to obtain a rephotographing prediction value to which the input feature map corresponds.
[0101] In other words, the predicting modules in the second convolutional neural network can be classified into two types, namely a first predicting module and second predicting modules, wherein the second predicting modules are employed to process split feature maps, while the first predicting module is employed to process the feature map that is not split. The pooling layers included in the second predicting modules are global pooling layers, while the pooling layer included in the first predicting module is an average pooling layer. In this mode of execution, through division of the certificate region it is made possible to obtain more representative local features, and this is conducive to enhancing precision rate of rephotographing identification, while the addition of the overall feature map makes it possible to further enhance precision rate of rephotographing identification.
[0102] An example is taken below to describe the process of using a well-trained second convolutional neural network to process the rectified certificate image.
[0103] In this example, the certificate image to be identified is an ID card image, and the framework of the second convolutional neural network can be as shown in Fig.
8.
[0104] (1) The certificate image to be identified is input into the second convolutional neural network, and the image is processed through the backbone network of the second Date Regue/Date Received 2022-06-29 convolutional neural network, whereby a feature map can be obtained.
[0105] (2) The feature map is split into three portions, namely three split feature maps that correspond to three portions as the ID card basic information, the ID card head poll" ait, and the ID card number, by means of a splitting module in accordance with the region splitting scale shown in Fig. 7, and the three split feature maps and the feature map that is not split are respectively input into the corresponding predicting modules, that is to say, the second predicting modules are used to process the split feature maps, and the first predicting module is used to process the feature map that is not split. With respect to each split feature map, it is accessed to the convolution layer (c) and the global pooling layer (g) in a second predicting module, since the number of channels of a feature map obtained by splitting is 512, the channel data is reduced to 128 through a 512*1*1*128 convolution layer, and each split feature map is thereafter globally pooled to get a mean value, and to obtain a feature vector (referred to as a global feature) of 128 dimensions.
With respect to the feature map that is not split, it is accessed to the convolution layer (c) and the average pooling layer (g) in the first predicting module to obtain a feature vector (to facilitate differentiation, it is referred to as a global feature). With respect to both the global feature and the local features, full connection layers of 128*2 dimensions are followed up to perform classification prediction, and differences between prediction values and true values are calculated through the softmax loss.
[0106] In one embodiment, as shown in Fig. 9, a process of training the second convolutional neural network includes:
[0107] S210 - obtaining plural rephotographed certificate images and non-rephotographed certificate images to serve as training samples;
[0108] S220 - constructing an initial second convolutional neural network whose parameters are random values; and
[0109] S230 - employing the training samples to iteratively train the initial second convolutional neural network, wherein, after each training, a prediction loss value of the current training Date Regue/Date Received 2022-06-29 is calculated according to a local image rephotographing prediction value generated by the initial second convolutional neural network in the current training, and then it is judged whether a loss of the initial second convolutional neural network is reduced to a minimum value according to the prediction loss value of the current training and prediction loss values of plural rounds of previous trainings, if yes, training is decided to be completed, and the initial second convolutional neural network at the end of the training is taken to serve as a well-trained second convolutional neural network; if not, a gradient of a loss function is calculated, network parameters of the initial second convolutional neural network are updated by means of a gradient descending method, and the next training is carried out.
[0110] The process of training the second convolutional neural network is similar to the process of training the first convolutional neural network, and the two processes differ only in their calculations of the loss, so the training process can be inferred from the above relevant contents, while no repetition is made in this context. The following description is directed to the calculation of the loss: in the process of training the second convolutional neural network, it is required to calculate the local loss to which each split feature map corresponds and the global loss to which the feature map, which is not split, corresponds, these losses are thereafter superposed to serve as the overall loss of the second convolutional neural network.
[0111] Figs. 1, 2, 5 and 9 are flowcharts each schematically illustrating a method of identifying re-photography of a certificate in an embodiment. As should be understood, although the various steps in the flowcharts of Figs. 1, 2, 5 and 9 are sequentially displayed as indicated by arrows, these steps are not necessarily executed in the sequences indicated by arrows.
Unless otherwise explicitly noted in this paper, execution of these steps is not restricted by any sequence, as these steps can also be executed in other sequences (than those indicated in the drawings). Moreover, at least partial steps in the flowcharts of Figs. 1, 2, and 9 may include plural sub-steps or multi-phases, these sub-steps or phases are not necessarily completed at the same timing, but can be executed at different timings, and Date Regue/Date Received 2022-06-29 these sub-steps or phases are also not necessarily sequentially performed, but can be performed in turns or alternately with other steps or with at least some of sub-steps or phases of other steps.
[0112] Based on the same inventive conception, the present invention further provides a device for identifying re-photography of a certificate. In one embodiment, as shown in Fig. 10, the device for identifying re-photography of a certificate comprises the following modules:
[0113] an image obtaining module 110, for obtaining a certificate image to be identified;
[0114] a first predicting module 120, for employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds;
[0115] a second predicting module 130, for rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds; and
[0116] an identification result determining module 140, for determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds.
[0117] In one embodiment, the second predicting module includes:
[0118] a mask extracting submodule, for employing a picture splitting model to extract a mask image of the certificate image to be identified, wherein the mask image includes a Date Regue/Date Received 2022-06-29 certificate region and a background region;
[0119] a corner point detecting submodule, for employing a corner point detecting algorithm to detect a plurality of corner points in the mask image;
[0120] a clustering submodule, for employing a preset clustering algorithm to determine four cluster centers from the plural corner points to serve as four corner points of the certificate region; and
[0121] a perspective transforming submodule, for performing a perspective transformation process on the certificate region according to the four corner points of the certificate region, and obtaining the rectified certificate image.
[0122] In one embodiment, the second predicting module further includes:
[0123] a feature map processing submodule, for processing the rectified certificate image into a feature map, and taking the feature map as an overall feature map;
[0124] a splitting submodule, for splitting the feature map on the basis of a preset region splitting scale, and obtaining a plurality of local feature maps;
[0125] a pooling submodule, for performing a pooling process on the overall feature map and the various local feature maps, and obtaining a global feature vector extracted from the overall feature map and local feature vectors extracted from the various local feature maps;
and
[0126] a predicting submodule, for performing a classification prediction process on the global feature vector and the various local feature vectors respectively, obtaining a rephotographing prediction value to which the overall feature map corresponds and rephotographing prediction values to which the various local feature maps correspond, and taking the rephotographing prediction value corresponding to the overall feature map and the rephotographing prediction values corresponding to the plural local feature maps as local image rephotographing prediction values to which the certificate image to be identified corresponds.

Date Regue/Date Received 2022-06-29
[0127] In one embodiment, the second convolutional neural network is a convolutional neural network that employs a ResNet18 network as a backbone network, wherein the backbone network includes a first predicting module, and a plurality of second predicting modules;
[0128] the backbone network is employed to process the rectified certificate image into feature maps and includes the first four submodules of the ResNet18 network, wherein stride of a first convolution layer in the fourth submodule of the ResNet18 network is altered as 1;
[0129] the first predicting module includes a convolution layer, an average pooling layer, a full connection layer, and a classifier, and the first predicting module is employed for performing a pooling process on the input overall feature map to obtain a global feature vector, and performing a classification prediction process on the obtained global feature vector to obtain a rephotographing prediction value to which the input overall feature map corresponds;
[0130] the second predicting module each includes a convolution layer, a global pooling layer, a full connection layer, and a classifier, and the second predicting module is employed for performing a pooling process on an input local feature map to obtain a local feature vector, and performing a classification prediction process on the obtained local feature vector to obtain a rephotographing prediction value to which the input local feature map corresponds.
[0131] In one embodiment, the convolution layer included in the second predicting module is a convolution layer of 512*1*1*128 dimensions; and the full connection layer included in the second predicting module is a full connection layer of 128*2 dimensions.
[0132] In one embodiment, the device further comprises:
[0133] a sample obtaining module, for obtaining plural rephotographed certificate images and non-rephotographed certificate images to serve as training samples;
[0134] a network constructing module, for constructing an initial second convolutional neural network whose parameters are random values; and Date Regue/Date Received 2022-06-29
[0135] a network training module, for employing the training samples to iteratively train the initial second convolutional neural network, wherein, after each training, a prediction loss value of the current training is calculated according to a local image rephotographing prediction value generated by the initial second convolutional neural network in the current training, and then it is judged whether a loss of the initial second convolutional neural network is reduced to a minimum value according to the prediction loss value of the current training and prediction loss values of plural rounds of previous trainings, if yes, training is decided to be completed, and the initial second convolutional neural network at the end of the training is taken to serve as a well-trained second convolutional neural network; if not, a gradient of a loss function is calculated, network parameters of the initial second convolutional neural network are updated by means of a gradient descending method, and the next training is carried out.
[0136] In one embodiment, the identification result determining module includes:
[0137] a mean value calculating module, for calculating a mean value of the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds; and
[0138] an identification result determining submodule, for comparing the mean value with a preset threshold, determining the re-photography identifying result to which the certificate image to be identified corresponds as rephotographed when it is determined that the mean value is greater than the preset threshold, and determining the re-photography identifying result to which the certificate image to be identified corresponds as not rephotographed when it is determined that the mean value is not greater than the preset threshold.
[0139] Specific definitions relevant to the device for identifying re-photography of a certificate may be inferred from the aforementioned definitions to the method of identifying re-photography of a certificate, while no repetition is made in this context. The various modules in the aforementioned device for identifying re-photography of a certificate can be wholly or partly realized via software, hardware, and a combination of software with Date Regue/Date Received 2022-06-29 hardware. The various modules can be in the form of hardware embedded in a processor in a computer equipment or independent of any computer equipment, and can also be stored in the form of software in a memory in a computer equipment, so as to facilitate the processor to invoke the aforementioned various modules and perform corresponding operations.
[0140] In one embodiment, a computer equipment is provided, and its internal structure can be as shown in Fig. 11. The computer equipment comprises a processor, a memory, a network interface and a database connected to each other via a system bus. The processor of the computer equipment is employed to provide computing and controlling capabilities.
The memory of the computer equipment includes a nonvolatile storage medium and an internal memory. The nonvolatile storage medium stores therein an operating system, a computer program and a database. The internal memory provides environment for the running of the operating system and the computer program in the nonvolatile storage medium. The database of the computer equipment is employed to store such data as certificate images to be identified, and the data as specifically stored can be further inferred from the definitions to the aforementioned method embodiment. The network interface of the computer equipment is employed to connect to an external terminal via network for communication. The computer program realizes a method of identifying re-photography of a certificate when it is executed by a processor.
[0141] As understandable to persons skilled in the art, the structure illustrated in Fig. 11 is merely a block diagram of partial structure relevant to the solution of the present application, and does not constitute any restriction to the computer equipment on which the solution of the present application is applied, as the specific computer equipment may comprise component parts that are more than or less than those illustrated in the figure, or may combine certain component parts, or may have different layout of component parts.
[0142] In one embodiment, there is provided a computer equipment that comprises a memory, a processor and a computer program stored on the memory and operable on the processor, and the following steps are realized when the processor executes the computer program:

Date Regue/Date Received 2022-06-29
[0143] obtaining a certificate image to be identified; employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds; rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds; and determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds.
[0144] In one embodiment, when the processor executes the computer program to realize the step of rectifying the certificate image to be identified and obtaining a rectified certificate image, the following steps are further realized:
[0145] employing a picture splitting model to extract a mask image of the certificate image to be identified, wherein the mask image includes a certificate region and a background region;
employing a comer point detecting algorithm to detect a plurality of corner points in the mask image; employing a preset clustering algorithm to determine four cluster centers from the plural comer points to serve as four comer points of the certificate region; and performing a perspective transformation process on the certificate region according to the four comer points of the certificate region, and obtaining the rectified certificate image.
[0146] In one embodiment, when the processor executes the computer program to realize the step of preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image Date Regue/Date Received 2022-06-29 rephotographing prediction value to which the certificate image to be identified corresponds, the following steps are further realized:
[0147] processing the rectified certificate image into a feature map, and taking the feature map as an overall feature map; splitting the feature map on the basis of a preset region splitting scale, and obtaining a plurality of local feature maps; performing a pooling process on the overall feature map and the various local feature maps, and obtaining a global feature vector extracted from the overall feature map and local feature vectors extracted from the various local feature maps; and performing a classification prediction process on the global feature vector and the various local feature vectors, obtaining a rephotographing prediction value to which the overall feature map corresponds and rephotographing prediction values to which the various local feature maps correspond, and taking the rephotographing prediction value corresponding to the overall feature map and the rephotographing prediction values corresponding to the plural local feature maps as local image rephotographing prediction values to which the certificate image to be identified corresponds.
[0148] In one embodiment, when the processor executes the computer program, the following steps are further realized:
[0149] obtaining plural rephotographed certificate images and non-rephotographed certificate images to serve as training samples; constructing an initial second convolutional neural network whose parameters are random values; and employing the training samples to iteratively train the initial second convolutional neural network, wherein, after each training, a prediction loss value of the current training is calculated according to a local image rephotographing prediction value generated by the initial second convolutional neural network in the current training, and then it is judged whether a loss of the initial second convolutional neural network is reduced to a minimum value according to the prediction loss value of the current training and prediction loss values of plural rounds of previous trainings, if yes, training is decided to be completed, and the initial second convolutional neural network at the end of the training is taken to serve as a well-trained Date Regue/Date Received 2022-06-29 second convolutional neural network; if not, a gradient of a loss function is calculated, network parameters of the initial second convolutional neural network are updated by means of a gradient descending method, and the next training is carried out.
[0150] In one embodiment, when the processor executes the computer program to realize the step of determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds, the following steps are further realized:
[0151] calculating a mean value of the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds; and comparing the mean value with a preset threshold, determining the re-photography identifying result to which the certificate image to be identified corresponds as rephotographed when it is determined that the mean value is greater than the preset threshold, and determining the re-photography identifying result to which the certificate image to be identified corresponds as not rephotographed when it is determined that the mean value is not greater than the preset threshold.
[0152] In one embodiment, there is provided a computer-readable storage medium storing thereon a computer program, and the following steps are realized when the computer program is executed by a processor:
[0153] obtaining a certificate image to be identified; employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds; rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified Date Regue/Date Received 2022-06-29 corresponds; and determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds.
[0154] In one embodiment, when the computer program is executed by a processor to realize the step of rectifying the certificate image to be identified and obtaining a rectified certificate image, the following steps are further realized:
[0155] employing a picture splitting model to extract a mask image of the certificate image to be identified, wherein the mask image includes a certificate region and a background region;
employing a comer point detecting algorithm to detect a plurality of corner points in the mask image; employing a preset clustering algorithm to determine four cluster centers from the plural comer points to serve as four comer points of the certificate region; and performing a perspective transformation process on the certificate region according to the four comer points of the certificate region, and obtaining the rectified certificate image.
[0156] In one embodiment, when the computer program is executed by a processor to realize the step of preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds, the following steps are further realized:
[0157] processing the rectified certificate image into a feature map, and taking the feature map as an overall feature map; splitting the feature map on the basis of a preset region splitting scale, and obtaining a plurality of local feature maps; performing a pooling process on the overall feature map and the various local feature maps, and obtaining a global feature vector extracted from the overall feature map and local feature vectors extracted from the various local feature maps; and performing a classification prediction process on the global feature vector and the various local feature vectors, obtaining a rephotographing Date Regue/Date Received 2022-06-29 prediction value to which the overall feature map corresponds and rephotographing prediction values to which the various local feature maps correspond, and taking the rephotographing prediction value corresponding to the overall feature map and the rephotographing prediction values corresponding to the plural local feature maps as local image rephotographing prediction values to which the certificate image to be identified corresponds.
[0158] In one embodiment, when the computer program is executed by a processor, the following steps are further realized:
[0159] obtaining plural rephotographed certificate images and non-rephotographed certificate images to serve as training samples; constructing an initial second convolutional neural network whose parameters are random values; and employing the training samples to iteratively train the initial second convolutional neural network, wherein, after each training, a prediction loss value of the current training is calculated according to a local image rephotographing prediction value generated by the initial second convolutional neural network in the current training, and then it is judged whether a loss of the initial second convolutional neural network is reduced to a minimum value according to the prediction loss value of the current training and prediction loss values of plural rounds of previous trainings, if yes, training is decided to be completed, and the initial second convolutional neural network at the end of the training is taken to serve as a well-trained second convolutional neural network; if not, a gradient of a loss function is calculated, network parameters of the initial second convolutional neural network are updated by means of a gradient descending method, and the next training is carried out.
[0160] In one embodiment, when the computer program is executed by a processor to realize the step of determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds, the following steps are further realized:
[0161] calculating a mean value of the overall image rephotographing prediction value and the Date Regue/Date Received 2022-06-29 local image rephotographing prediction value to which the certificate image to be identified corresponds; and comparing the mean value with a preset threshold, determining the re-photography identifying result to which the certificate image to be identified corresponds as rephotographed when it is determined that the mean value is greater than the preset threshold, and determining the re-photography identifying result to which the certificate image to be identified corresponds as not rephotographed when it is determined that the mean value is not greater than the preset threshold.
[0162] As comprehensible to persons ordinarily skilled in the art, the entire or partial flows in the methods according to the aforementioned embodiments can be completed via a computer program instructing relevant hardware, wherein the computer program can be stored in a nonvolatile computer-readable storage medium, and the computer program can include the flows as embodied in the aforementioned various methods when executed.
Any reference to the memory, storage, database or other media used in the various embodiments provided by the present application can all include nonvolatile and/or volatile memory/memories. The nonvolatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM) or a flash memory. The volatile memory can include a random access memory (RAM) or an external cache memory. To serve as explanation rather than restriction, the RAM is obtainable in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM
(SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.
[0163] Technical features of the aforementioned embodiments are randomly combinable, while all possible combinations of the technical features in the aforementioned embodiments are not exhausted for the sake of brevity, but all these should be considered to fall within the scope recorded in the Description as long as such combinations of the technical features are not mutually contradictory.

Date Regue/Date Received 2022-06-29
[0164] The foregoing embodiments are merely directed to several modes of execution of the present application, and their descriptions are relatively specific and detailed, but they should not be hence misunderstood as restrictions to the inventive patent scope. As should be pointed out, persons with ordinary skill in the art may further make various modifications and improvements without departing from the conception of the present application, and all these should pertain to the protection scope of the present application.
Accordingly, the patent protection scope of the present application shall be based on the attached Claims.

Date Regue/Date Received 2022-06-29

Claims (10)

What is claimed is:
1. A method of identifying re-photography of a certificate, characterized in that the method comprises:
obtaining a certificate image to be identified;
employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds;
rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds; and determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds.
2. The method of identifying re-photography of a certificate according to Claim 1, characterized in that the step of rectifying the certificate image to be identified and obtaining a rectified certificate image includes:
employing a picture splitting model to extract a mask image of the certificate image to be identified, wherein the mask image includes a certificate region and a background region;
employing a corner point detecting algorithm to detect a plurality of comer points in the mask image;
employing a preset clustering algorithm to determine four cluster centers from the plural comer points to serve as four corner points of the certificate region; and Date Regue/Date Received 2022-06-29 performing a perspective transformation process on the certificate region according to the four corner points of the certificate region, and obtaining the rectified certificate image.
3. The method of identifying re-photography of a certificate according to Claim 1, characterized in that the step of preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps correspond, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds includes:
processing the rectified certificate image into a feature map, and taking the feature map as an overall feature map;
splitting the feature map on the basis of a preset region splitting scale, and obtaining a plurality of local feature maps;
performing a pooling process on the overall feature map and the various local feature maps, and obtaining a global feature vector extracted from the overall feature map and local feature vectors extracted from the various local feature maps; and performing a classification prediction process on the global feature vector and the various local feature vectors, obtaining a rephotographing prediction value to which the overall feature map corresponds and rephotographing prediction values to which the various local feature maps correspond, and taking the rephotographing prediction value corresponding to the overall feature map and the rephotographing prediction values corresponding to the plural local feature maps as local image rephotographing prediction values to which the certificate image to be identified corresponds.
4. The method of identifying re-photography of a certificate according to Claim 3, characterized in that the second convolutional neural network employs a ResNet18 network as a backbone network, wherein the backbone network includes a first predicting module, and a plurality of second predicting modules;
Date Regue/Date Received 2022-06-29 the backbone network is employed to process the rectified certificate image into feature maps and includes the first four submodules of the ResNet18 network, wherein stride of a first convolution layer in the fourth submodule of the ResNet18 network is altered as 1;
the first predicting module in the second convolutional neural network includes a convolution layer, an average pooling layer, a full connection layer and a classifier, and the first predicting module is employed for performing a pooling process on the input overall feature map to obtain a global feature vector, and performing a classification prediction process on the obtained global feature vector to obtain a rephotographing prediction value to which the input overall feature map corresponds; and any of the second predicting modules in the second convolutional neural network includes a convolution layer, a global pooling layer, a full connection layer and a classifier, and the second predicting module is employed for performing a pooling process on an input local feature map to obtain a local feature vector, and performing a classification prediction process on the obtained local feature vector to obtain a rephotographing prediction value to which the input local feature map corresponds.
5. The method of identifying re-photography of a certificate according to Claim 4, characterized in that the convolution layer included in the second predicting module is a convolution layer of 512*1*1*128 dimensions; and the full connection layer included in the second predicting module is a full connection layer of 128*2 dimensions.
6. The method of identifying re-photography of a certificate according to Claim 1, characterized in that a process of training the second convolutional neural network includes:
obtaining plural rephotographed certificate images and non-rephotographed certificate images to serve as training samples;
constructing an initial second convolutional neural network whose parameters are random values;

Date Regue/Date Received 2022-06-29 and employing the training samples to iteratively train the initial second convolutional neural network, wherein, after each training, a prediction loss value of the current training is calculated according to a local image rephotographing prediction value generated by the initial second convolutional neural network in the current training, and then it is judged whether a loss of the initial second convolutional neural network is reduced to a minimum value according to the prediction loss value of the current training and prediction loss values of plural rounds of previous trainings, if yes, training is decided to be completed, and the initial second convolutional neural network at the end of the training is taken to serve as a well-trained second convolutional neural network; if not, a gradient of a loss function is calculated, network parameters of the initial second convolutional neural network are updated by means of a gradient descent method, and the next training is carried out.
7. The method of identifying re-photography of a certificate according to Claim 1, characterized in that the step of determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds includes:
calculating a mean value of the overall image rephotographing prediction value and the local image rephotographing prediction value to which the certificate image to be identified corresponds; and comparing the mean value with a preset threshold, determining the re-photography identifying result, to which the certificate image to be identified corresponds, as rephotographed when it is determined that the mean value is greater than the preset threshold, and determining the re-photography identifying result, to which the certificate image to be identified corresponds, as not rephotographed when it is determined that the mean value is not greater than the preset threshold.
8. A device for identifying re-photography of a certificate, characterized in that the device comprises:

Date Regue/Date Received 2022-06-29 an image obtaining module, for obtaining a certificate image to be identified;
a first predicting module, for employing a well-trained first convolutional neural network to process the certificate image to be identified, and obtaining an overall image rephotographing prediction value to which the certificate image to be identified corresponds;
a second predicting module, for rectifying the certificate image to be identified, obtaining a rectified certificate image, preprocessing the rectified certificate image to obtain an overall feature map and a plurality of local feature maps, employing a well-trained second convolutional neural network to respectively calculate rephotographing prediction values to which the overall feature map and the various local feature maps corresponcl, and obtaining a local image rephotographing prediction value to which the certificate image to be identified corresponds; and an identification result determining module, for determining a re-photography identifying result, to which the certificate image to be identified corresponds, according to the overall image rephotographing prediction value and the local image rephotographing prediction value to which the rectified certificate image corresponds.
9. A computer equipment, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that steps of the method according to anyone of Claims 1 to 7 are realized when the processor executes the computer program.
10. A computer-readable storage medium, storing a computer program thereon, characterized in that steps of the method according to anyone of Claims 1 to 7 are realized when the computer program is executed by a processor.

Date Regue/Date Received 2022-06-29
CA3166091A 2021-06-29 2022-06-29 An identification method, device computer equipment and storage medium for identity document reproduction Pending CA3166091A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110725619.9A CN113344000A (en) 2021-06-29 2021-06-29 Certificate copying and recognizing method and device, computer equipment and storage medium
CN202110725619.9 2021-06-29

Publications (1)

Publication Number Publication Date
CA3166091A1 true CA3166091A1 (en) 2022-12-29

Family

ID=77481255

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3166091A Pending CA3166091A1 (en) 2021-06-29 2022-06-29 An identification method, device computer equipment and storage medium for identity document reproduction

Country Status (2)

Country Link
CN (1) CN113344000A (en)
CA (1) CA3166091A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114066807B (en) * 2021-10-09 2023-02-10 西安深信科创信息技术有限公司 Multi-column convolution neural network reproduced picture detection method based on wavelet transformation
CN114005019B (en) * 2021-10-29 2023-09-22 北京有竹居网络技术有限公司 Method for identifying flip image and related equipment thereof
CN114066894A (en) * 2022-01-17 2022-02-18 深圳爱莫科技有限公司 Detection method for display image reproduction, storage medium and processing equipment
CN115035030A (en) * 2022-05-07 2022-09-09 北京大学深圳医院 Image recognition method, image recognition device, computer equipment and computer-readable storage medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108460649A (en) * 2017-02-22 2018-08-28 阿里巴巴集团控股有限公司 A kind of image-recognizing method and device
FR3086884B1 (en) * 2018-10-09 2020-11-27 Idemia Identity & Security France DOCUMENTARY FRAUD DETECTION PROCESS.
CN110569808A (en) * 2019-09-11 2019-12-13 腾讯科技（深圳）有限公司 Living body detection method and device and computer equipment
CN110688939B (en) * 2019-09-25 2022-07-29 支付宝(杭州)信息技术有限公司 Method, system and equipment for verifying certificate image to be identified
CN112686812B (en) * 2020-12-10 2023-08-29 广州广电运通金融电子股份有限公司 Bank card inclination correction detection method and device, readable storage medium and terminal
CN112767394A (en) * 2021-03-04 2021-05-07 重庆赛迪奇智人工智能科技有限公司 Image detection method, device and equipment
CN112801057B (en) * 2021-04-02 2021-07-13 腾讯科技（深圳）有限公司 Image processing method, image processing device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN113344000A (en) 2021-09-03

Similar Documents

Publication Publication Date Title
CN111080628B (en) Image tampering detection method, apparatus, computer device and storage medium
CA3166091A1 (en) An identification method, device computer equipment and storage medium for identity document reproduction
CN111460968B (en) Unmanned aerial vehicle identification and tracking method and device based on video
CN111989689A (en) Method for identifying objects within an image and mobile device for performing the method
CN111160102B (en) Training method of face anti-counterfeiting recognition model, face anti-counterfeiting recognition method and device
CN111191568A (en) Method, device, equipment and medium for identifying copied image
CN113239907B (en) Face recognition detection method and device, electronic equipment and storage medium
CN110874574B (en) Pedestrian re-recognition method, device, computer equipment and readable storage medium
CN110009003A (en) Training method, the device and system of image procossing and image comparison model
CN111368666A (en) Living body detection method based on novel pooling and attention mechanism double-current network
CN111414858A (en) Face recognition method, target image determination method, device and electronic system
CN111275694B (en) Attention mechanism guided progressive human body division analysis system and method
CN112464775A (en) Video target re-identification method based on multi-branch network
CN112651333A (en) Silence living body detection method and device, terminal equipment and storage medium
CN113111880A (en) Certificate image correction method and device, electronic equipment and storage medium
CN109670423A (en) A kind of image identification system based on deep learning, method and medium
CN114387548A (en) Video and liveness detection method, system, device, storage medium and program product
Huang et al. Multi-Teacher Single-Student Visual Transformer with Multi-Level Attention for Face Spoofing Detection.
CN115115552B (en) Image correction model training method, image correction device and computer equipment
CN113723310B (en) Image recognition method and related device based on neural network
CN112287932B (en) Method, device, equipment and storage medium for determining image quality
CN116958615A (en) Picture identification method, device, equipment and medium
CN115170414A (en) Knowledge distillation-based single image rain removing method and system
CN112784781A (en) Method and device for detecting forged faces based on difference perception meta-learning
CN116012248B (en) Image processing method, device, computer equipment and computer storage medium