CN113344000A

CN113344000A - Certificate copying and recognizing method and device, computer equipment and storage medium

Info

Publication number: CN113344000A
Application number: CN202110725619.9A
Authority: CN
Inventors: 王止观; 顾扬
Original assignee: Nanjing Xingyun Digital Technology Co Ltd
Current assignee: Nanjing Xingyun Digital Technology Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2021-09-03
Also published as: CA3166091A1

Abstract

The application relates to the technical field of image classification, in particular to a certificate copying and identifying method, a certificate copying and identifying device, computer equipment and a storage medium. The method comprises the following steps: acquiring a certificate image to be identified; processing the certificate image to be recognized by using the trained first convolution neural network to obtain a corresponding overall image reproduction predicted value; carrying out correction processing on a certificate image to be identified to obtain a certificate correction image, and preprocessing the certificate correction image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams; respectively calculating the reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using the trained second convolutional neural network to obtain the reproduction predicted values of the local images corresponding to the certificate image to be recognized; and determining a reproduction identification result corresponding to the certificate image to be identified according to the reproduction predicted value of the whole image and the reproduction predicted value of the local image. The embodiment of the invention can improve the copying and identifying accuracy of the certificate image.

Description

Certificate copying and recognizing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of image classification technologies, and in particular, to a certificate copying identification method, apparatus, computer device, and storage medium.

Background

The following statements are merely provided to provide background information related to the present disclosure and may not necessarily constitute prior art.

The identification of identity card screen reproduction aims at utilizing computer vision, artificial intelligence, image processing and other technologies to judge whether the identity card is shot again through the screen. For many years, people strive to extract features such as frequency spectrum features, texture features, color features and the like by using a traditional method, and then classify the features by using a support vector machine, however, the traditional method needs to artificially extract the features, so that the traditional method is subjective and has certain limitations, and the misrecognition rate is high.

With the rapid development of artificial intelligence and the appearance of a deep neural network, people begin to learn characteristics by a computer through model training. The features obtained by the computer through deep learning are more reliable. In view of this, an identification card screen-copying recognition model based on deep learning is proposed. Compared with the traditional method, the error identification rate of the identification card screen reproduction identification model based on deep learning is lower, however, for some more special certificate pictures, the accuracy rate of the model is not high, for example, for some reproduction pictures except the screen with a larger proportion of real non-reproduction background, the model can identify the reproduction pictures as the error identification rate, and for the real certificate pictures with background reflection, such as glass, smooth white board and the like, which have stronger light intensity during shooting, the model can identify the reproduction pictures as the error identification rate.

Disclosure of Invention

Aiming at the defects or shortcomings, the invention provides a certificate copying and identifying method, a certificate copying and identifying device, computer equipment and a storage medium.

The invention provides a document reproduction identification method according to a first aspect, which in one embodiment comprises:

acquiring a certificate image to be identified;

processing the certificate image to be identified by using the trained first convolution neural network to obtain an integral image reproduction predicted value corresponding to the certificate image to be identified;

correcting the certificate image to be recognized to obtain a certificate corrected image, preprocessing the certificate corrected image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, and respectively calculating reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network to obtain reproduction predicted values corresponding to the certificate image to be recognized;

and determining a reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified.

In one embodiment, the step of performing correction processing on the certificate image to be recognized to obtain the certificate correction image comprises the following steps:

extracting a mask image of the certificate image to be identified by using the image segmentation model, wherein the mask image comprises a certificate area and a background area;

detecting a plurality of corner points in the mask image by using a corner point detection algorithm;

determining 4 clustering centers from the plurality of angular points by using a preset clustering algorithm as four angular points of the certificate area;

and carrying out perspective transformation processing on the certificate area according to the four corner points of the certificate area to obtain a certificate correction image.

In one embodiment, the method for obtaining the reproduction prediction value of the local image corresponding to the certificate image to be recognized includes the steps of preprocessing the certificate correction image to obtain an overall feature map and a plurality of local feature maps, respectively calculating the reproduction prediction values corresponding to the overall feature map and the local feature maps by using a trained second convolutional neural network, and obtaining the reproduction prediction value of the local image corresponding to the certificate image to be recognized, including:

processing the certificate correction image into a feature map, and taking the feature map as an integral feature map;

segmenting the feature map based on a preset region segmentation proportion to obtain a plurality of local feature maps;

pooling the overall characteristic diagram and each local characteristic diagram to obtain a global characteristic vector extracted from the overall characteristic diagram and a local characteristic vector extracted from each local characteristic diagram;

and respectively carrying out classification prediction processing on the global feature vectors and the local feature vectors to obtain reproduction predicted values corresponding to the global feature maps and reproduction predicted values corresponding to the local feature maps, and taking the reproduction predicted values corresponding to the global feature maps and the reproduction predicted values corresponding to the local feature maps as local image reproduction predicted values corresponding to the certificate images to be identified.

In one embodiment, the second convolutional neural network is a convolutional neural network using a ResNet18 network as a backbone network, and comprises a first prediction module and a plurality of second prediction modules;

the backbone network is used for processing the certificate rectification image into a feature map, and the feature map is the first four sub-modules of the ResNet18 network, wherein the stride of the first convolution layer in the fourth sub-module of the ResNet18 network is set to be 1;

the segmentation module is used for segmenting a feature map obtained by processing the backbone module into a plurality of local feature maps based on a preset region segmentation proportion, and inputting the plurality of local feature maps into the plurality of prediction modules respectively, wherein the number of the prediction modules is the same as that of the local feature maps;

the first prediction module comprises a convolution layer, a tie pooling layer, a full-link layer and a classifier, and is used for pooling the input overall feature map to obtain an overall feature vector and performing classification prediction processing on the obtained overall feature vector to obtain a reproduction prediction value corresponding to the input overall feature map;

the second prediction module comprises a convolution layer, a global pooling layer, a full-link layer and a classifier, and is used for pooling the input local feature map to obtain a local feature vector and performing classification prediction processing on the obtained local feature vector to obtain a reproduction prediction value corresponding to the input local feature map.

In one embodiment, the second prediction module includes convolution layers with dimensions 512 x 1 x 128 convolution layers; the fully connected layer included in the second prediction module is a fully connected layer with dimension 128 x 2.

In one embodiment, the training process of the second convolutional neural network includes:

acquiring a plurality of copied certificate images and non-copied certificate images as training samples;

constructing an initial second convolutional neural network, wherein the parameter of the initial second convolutional neural network is a random value;

the initial second convolutional neural network is iteratively trained using training samples, wherein, after each training, calculating the prediction loss value of the training according to the local image reproduction prediction value generated by the initial second convolutional neural network in the training, judging whether the loss of the initial second convolutional neural network is reduced to the minimum value according to the predicted loss value of the current training and the predicted loss values of the previous training, determining that training is complete upon determining that the loss of the initial second convolutional neural network decreases to a minimum, and the initial second convolutional neural network after the training is finished is used as the trained second convolutional neural network, and when the loss of the initial second convolutional neural network is not reduced to the minimum value, calculating the gradient of the loss function, updating the network parameters of the initial second convolutional neural network by using a gradient descent method, and carrying out next training.

In one embodiment, the step of determining the reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified comprises the following steps:

calculating the average value of the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified;

and comparing the mean value with a preset threshold value, determining the copying identification result corresponding to the certificate image to be identified as copying when the mean value is determined to be larger than the preset threshold value, and determining the copying identification result corresponding to the certificate image to be identified as non-copying when the mean value is determined to be not larger than the preset threshold value.

The invention provides according to a second aspect a document reproduction identification apparatus, which in one embodiment comprises:

the image acquisition module is used for acquiring the certificate image to be identified;

the first prediction module is used for processing the certificate image to be recognized by using the trained first convolution neural network to obtain an integral image reproduction prediction value corresponding to the certificate image to be recognized;

the second prediction module is used for correcting the certificate image to be recognized to obtain the certificate corrected image, preprocessing the certificate corrected image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, and respectively calculating the reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network to obtain the reproduction predicted values corresponding to the certificate image to be recognized;

and the identification result determining module is used for determining the reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified.

The present invention provides according to a third aspect a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of an embodiment of any of the methods described above when executing the computer program.

The present invention provides according to a fourth aspect a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the embodiments of the method of any one of the above.

In the embodiment of the invention, for the certificate image to be identified, a trained first convolution neural network is used for processing the certificate image to be identified to obtain a corresponding integral image reproduction predicted value; and then correcting the certificate image to obtain a certificate correction image, preprocessing the certificate correction image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, then respectively calculating the reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network to obtain local image reproduction predicted values corresponding to the certificate image to be recognized, and further determining a reproduction recognition result corresponding to the certificate image to be recognized according to the overall image reproduction predicted values and the local image reproduction predicted values.

Drawings

FIG. 1 is a schematic flow chart of a method for document reproduction identification in one embodiment;

FIG. 2 is a schematic flow chart illustrating correction of an image of a document to be recognized according to an embodiment;

FIG. 3(a) is a schematic view of an image of a document to be identified in one embodiment;

FIG. 3(b) is a schematic view of a mask image obtained in one embodiment;

FIG. 4(a) is a diagram illustrating a plurality of corner points detected from a mask image in one embodiment;

FIG. 4(b) is a schematic diagram of 4 cluster centers in one embodiment;

FIG. 4(c) is a schematic illustration of a rectified image of a document according to one embodiment;

FIG. 5 is a schematic diagram of a process for obtaining a local snap prediction value according to an embodiment;

FIG. 6 is a schematic diagram illustrating a comparison of a fourth submodule of an original ResNet18 with a backbone module in one embodiment;

FIG. 7 is a diagram illustrating a region segmentation scale in one embodiment;

FIG. 8 is an architecture diagram of a second convolutional neural network in one embodiment;

FIG. 9 is a schematic flow chart of training a second convolutional neural network in one embodiment;

FIG. 10 is a block diagram of a credential reproduction identification device in one embodiment;

FIG. 11 is a diagram illustrating an internal structure of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The invention provides a certificate copying and identifying method. In one embodiment, the document reproduction identification method is as shown in FIG. 1, and is described in detail below.

S110: and acquiring an image of the certificate to be identified.

The certificate copying identification method provided by the embodiment can be applied to business scenes in the fields of finance, insurance, e-commerce, O2O and the like, in which whether a certificate image is a copied image needs to be identified, specifically, a scene of identity authentication for a user, a merchant and the like can be specifically used, and the business risk can be controlled by identifying whether the certificate image provided by the user or the merchant is the copied image to filter a fraudulent user or the merchant. The execution main body of the method can be a computing device such as a server, the certificate image to be identified can be acquired by a camera, a mobile phone, a camera and other devices capable of acquiring the image in real time, and the certificate image can be an identity card image, a social security card image and the like.

S120: and processing the certificate image to be recognized by using the trained first convolution neural network to obtain an integral image reproduction predicted value corresponding to the certificate image to be recognized.

In this embodiment, the first Convolutional Neural network is a Convolutional Neural Network (CNN), which can be implemented by any series of Networks of the ResNet (such as ResNet18, ResNet50, etc.), and can also be implemented by a ResNeXt network, an inclusion-ResNet network, a VGG network, or an AlexNet network.

In one embodiment, considering that ResNet18 is relatively small, fast, and meets accuracy requirements relative to other ResNet networks, ResNet18 is used as the first convolutional neural network, through which binary predictions of the image for evidence correction are made directly and the overall loss is calculated by the loss function.

The training process of the ResNet18 may be to first construct an initial ResNet18 network model and select training samples, and then train the network model with the training samples. The training sample comprises a copied certificate image and a non-copied certificate image, when the copied certificate image is selected as the training sample, the certificate image (including the certificate image) including a computer screen and a mobile phone screen as backgrounds can be selected, the backgrounds in the screens except black and white comprise diversified backgrounds such as a desktop, a notebook, a word document and the like as far as possible, and meanwhile, the training data also comprises other background pictures except the screens; for the non-reproduction certificate image, pictures with similar colors to the background of the screen reproduction such as tables and paper can be selected.

When the iterative training of the network model is started, the parameters of the whole network model are random values, in each training process, the probability (namely a predicted value) that the whole image belongs to each category can be calculated through forward propagation, then the predicted loss value of the training is calculated through a loss function such as softmax, then whether the loss of the whole network is reduced to the minimum value or not is judged through the predicted loss value of the training and the predicted loss values calculated through multiple times of previous training, if the loss is not reduced to the minimum value, the parameter values of the network model are updated through backward propagation (the gradient of the loss function can be calculated when the parameter values are updated, then the updating is carried out by using a random gradient descent method (SGD)), the next training is continued, and by analogy, the loss of the network model is continuously reduced through continuous training and parameter updating (the loss of the whole network is reduced, the parameter selection is correct, and further parameter adjustment, such as learning rate, is needed) until the loss of the whole network is determined to be reduced to the minimum value, and at the moment, the parameters of the whole network model can be obtained, namely the trained network model can be obtained.

When judging whether the loss of the whole network is reduced to the minimum value according to the predicted loss value of the current training and the predicted loss values of the previous training, specifically, judging based on the predicted loss value and the loss difference value of the last N times of training. The last N times of training include the current training and the previous N-1 times of training, and the number N of times can be adjusted according to different scene requirements, for example, the last 4 times, 5 times, more times of training and the like can be observed; illustratively, the loss difference for each training may be obtained by subtracting the predicted loss value of the last training from the predicted loss value of the training. If the loss difference values of the last N times of training are not changed any more, namely the difference values are all 0, or the loss difference values of the last N times of training are basically unchanged, namely the loss difference values of the last N times of training oscillate, but the minimum predicted loss value is not reduced, the loss of the whole network is judged to be reduced to the minimum value, and the training can be completed; otherwise, it is determined that the loss of the entire network is not reduced to the minimum value, and training needs to be continued, for example, the loss difference of the previous training is 0, but the predicted loss value of the current training is much reduced compared with the predicted loss value of the previous training, and then training needs to be continued.

Taking the loss function as softmax as an example, the prediction probability that the input image belongs to each category (i.e., pan and no-pan) can be calculated by the following equation:

where z is the input to the softmax layer and f (z) is the output.

The softmax loss was calculated using the following equation:

where y is the true value, f (z) is the prediction probability of the input image belonging to the category calculated after passing through the softmax layer, and l is the prediction loss value (used to represent the difference between the predicted value and the true value).

And inputting the certificate image to be identified into the trained network model, wherein the network model can output a predicted value corresponding to the image (namely the whole image reproduction predicted value), the predicted value can be limited to 0, 1, and the predicted value is used for representing the probability that the image is the reproduction image.

S130: correcting the certificate image to be recognized to obtain a certificate correction image, preprocessing the certificate correction image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, respectively calculating the reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network, and obtaining the reproduction predicted values corresponding to the local image to be recognized

In this embodiment, when the certificate image to be recognized is corrected, the mask of the certificate image to be recognized is extracted first, then four corner points in the mask are recognized, and finally the mask is subjected to perspective transformation through the corner points to realize correction.

In the present embodiment, the second Convolutional Neural network is not an existing Convolutional Neural Network (CNN), but a new network is proposed in the present embodiment to solve the deficiencies of the prior art. Any series of networks of ResNet (such as ResNet18, ResNet50 and the like), or a ResNeXt network, an inclusion-ResNet network, a VGG network, an AlexNet network and the like can be adopted as a backbone network, and a corresponding convolutional layer, a global pooling layer, a full connection layer and softmax are added in sequence, so that the certificate rectification image is processed into a feature map, the feature map is divided into a plurality of local feature maps, the duplication prediction values corresponding to the local feature maps are respectively calculated, the feature map is used as the overall feature map, the corresponding duplication prediction values are calculated, and the calculated duplication prediction values corresponding to the overall feature map and the local feature maps are the local image duplication prediction values corresponding to the certificate image to be identified.

S140: and determining a reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified.

In this embodiment, when determining the reproduction identification result corresponding to the document image to be identified according to the overall image reproduction predicted value and the local image reproduction predicted value corresponding to the document image to be identified, the overall image reproduction predicted value and the local image reproduction predicted value need to be integrated to obtain a comprehensive value, when the comprehensive value is obtained by integration, the average value of the overall image reproduction predicted value and the local image reproduction predicted value corresponding to the document image to be identified can be calculated, and then the average value is compared with a preset threshold value, and when determining that the average value is greater than the preset threshold value, the reproduction identification result corresponding to the document image to be identified is determined as reproduction, or when determining that the average value is not greater than the preset threshold value, the reproduction identification result corresponding to the document image to be identified is determined as non-reproduction. Of course, when the comprehensive value is obtained by integration, the overall image reproduction predicted value and the local image reproduction predicted value may be respectively multiplied by respective corresponding preset weight values (for example, 0.4 and 0.6, which may be specifically adjusted according to actual needs), and then summed to obtain the comprehensive value.

In the embodiment, a trained first convolution neural network is used for processing the certificate image to obtain a corresponding overall image reproduction predicted value, the certificate image to be identified is corrected to obtain a certificate corrected image, the certificate corrected image is preprocessed to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, then a trained second convolution neural network is used for calculating the overall characteristic diagram and the reproduction predicted value corresponding to each local characteristic diagram respectively to obtain a local image reproduction predicted value corresponding to the certificate image to be identified so as to obtain the overall image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified, and then a reproduction identification result corresponding to the certificate image to be identified is determined according to the overall image reproduction predicted value and the local image reproduction predicted value, the embodiment takes the overall information and the local information of the certificate image to be identified into account, and the two images are fused to weaken the influence of the background in the image, so that whether the certificate image is a reproduced image or not can be distinguished more accurately and reliably.

In one embodiment, as shown in fig. 2, the step of performing correction processing on the image of the document to be recognized to obtain a corrected image of the document includes:

s121: extracting a mask image of the certificate image to be identified by using the image segmentation model, wherein the mask image comprises a certificate area and a background area;

s122: detecting a plurality of corner points in the mask image by using a corner point detection algorithm;

s123: determining 4 clustering centers from the plurality of angular points by using a preset clustering algorithm as four angular points of the certificate area;

s124: and carrying out perspective transformation processing on the certificate area according to the four corner points of the certificate area to obtain a certificate correction image.

The image segmentation model can be a certificate segmentation network which is commonly used at present. In one embodiment, the realization effect of the current commonly used certificate segmentation network is not ideal when certificate identification is performed, so that a deep learning network architecture UNet can be used for certificate segmentation, training sets need to be collected when certificate segmentation is performed by using UNet, and the method for performing certificate segmentation by using UNet is not common because certificate images are difficult to collect, namely a large training set is difficult to obtain.

UNet architectures are chosen because of their relative simplicity, small model, and short prediction time. When the model is trained, the model is mainly used for copying and recognition, so that the copied certificate images and the non-copied certificate images with the same quantity are simultaneously selected, and the conditions of backgrounds, shooting angles and light rays when various certificates are shot are contained as much as possible; in addition, an artificial labeling mask is used as a real label, and loss is calculated through the difference between a predicted value and a real value calculated by a two-classification cross entropy statistical model.

Illustratively, the training process of the UNet model, which is a picture segmentation model implemented by UNet, may be that, at the beginning of training, parameters of the entire model are random values, in the training process, prediction labels (e.g., 0 is a background and 1 is a certificate) of different pixel points of an image may be calculated through forward propagation, then losses of a predicted value and a true label value are calculated through two-class cross entropy, then parameter values of the model are updated through backward propagation for next training, the losses are continuously reduced through iterative training, and when the losses are reduced to be substantially unchanged, parameters of the entire model, which is the picture segmentation model implemented by UNet, may be obtained. Inputting the certificate image, as shown in fig. 3(a), into the trained model to obtain a mask image of the certificate image, as shown in fig. 3(b), in the mask image, the white part is the certificate area, and the black part is the background area.

The corner detection algorithm may adopt a Harris algorithm, and a series of corners may be obtained by performing corner detection on the mask image through the Harris algorithm, for example, as shown in fig. 4 (a).

The pre-set clustering algorithm may be a K-Means clustering algorithm. Clustering by using a K-Means clustering algorithm from a series of corner points detected by a corner point detection algorithm to obtain 4 clustering centers, such as shown in FIG. 4(b), and then using the 4 clustering centers as four corner points of the certificate area.

And finally, performing perspective transformation processing on the certificate area based on four corner points of the certificate area to obtain a certificate correction image, for example, as shown in fig. 4 (c).

In the embodiment, before the deep learning technology is used for performing prediction processing on the certificate image to be recognized, the certificate image to be recognized is corrected, so that the accuracy of final copying and recognition can be further improved.

In an embodiment, as shown in fig. 5, the step of preprocessing the certificate correction image to obtain an overall feature map and a plurality of local feature maps, and using a trained second convolutional neural network to calculate the reproduction prediction values corresponding to the overall feature map and the local feature maps, so as to obtain the reproduction prediction values corresponding to the certificate image to be recognized includes:

s131: processing the certificate correction image into a feature map, and taking the feature map as an integral feature map;

s132: segmenting the feature map based on a preset region segmentation proportion to obtain a plurality of local feature maps;

s133: pooling the overall characteristic diagram and each local characteristic diagram to obtain a global characteristic vector extracted from the overall characteristic diagram and a local characteristic vector extracted from each local characteristic diagram;

s134: and respectively carrying out classification prediction processing on the global feature vectors and the local feature vectors to obtain reproduction predicted values corresponding to the global feature maps and reproduction predicted values corresponding to the local feature maps, and taking the reproduction predicted values corresponding to the global feature maps and the reproduction predicted values corresponding to the local feature maps as local image reproduction predicted values corresponding to the certificate images to be identified.

The above steps will be described below by taking the second convolutional neural network as an example of adopting a ResNet18 network as a backbone network.

In this example, the second convolutional neural network employs a ResNet18 network as a backbone network, which includes a plurality of prediction modules, specifically, a first prediction module and a plurality of second prediction modules.

The backbone network is used for processing the certificate correction image into a feature map and is composed of the first four sub-modules of the ResNet18 network, and in the backbone module, stride of the first convolution layer in the fourth sub-module of the ResNet18 network is set to be 1. I.e. the spatial down-sampling in the fourth sub-module in the original ResNet18 network is removed, and the last global pooling layer and full connectivity layer in the original ResNet18 network is removed. For example, the fourth sub-module in the original ResNet18 network and the following global pooling layer (GAP) and full connection layer (fc) may be as shown in (a) of fig. 6, and the backbone module in the second convolutional neural network may be as shown in (b) of fig. 6, in (a), stride 2 in the first convolutional layer in the fourth sub-module is changed to 1 for the spatial downsampling operation, i.e. the spatial downsampling operation may be deleted.

After the certificate image to be identified is processed into the feature map, the backbone network can divide the certificate image into a plurality of local feature maps through a preset dividing module. The segmentation module is configured to segment a feature map obtained by processing a backbone network into a plurality of local feature maps based on a preset region segmentation ratio (which may be set according to actual requirements), and exemplarily, taking a certificate image as an identity card, the image may be segmented according to the region segmentation ratio shown in fig. 7, so that the entire certificate region may be segmented into three parts, where a first part is a basic information region of the identity card, a second part is an identity card head image region, and a third part is an identity card number region. Assuming that the width of the identity card is w and the height of the identity card is h, after rough estimation, the width of the basic information area of the first part is w 1-2/3 w, the height of the basic information area is h 1-4/5 h, the width of the avatar area of the second part is w 2-1/3 w, the height of the avatar area of the second part is h 2-4/5 h, the width of the identity card area of the third part is w 3-w, and the height of the avatar area of the third part is h 3-1/5 h.

After the above-described division process is completed, the plurality of local feature maps obtained by division and the undivided feature map (used as an overall feature map) are input to the plurality of prediction modules, respectively, the overall feature map is processed by the first prediction module, and the respective local feature maps are processed by the respective second prediction modules, so that the number of prediction modules is the same as the number of local feature maps. Each prediction module comprises a convolution layer, a pooling layer, a full-link layer and a classifier (softmax), and is used for pooling the input feature map to obtain a feature vector (processing the whole feature map to obtain a global feature vector and processing the local feature map to obtain a local feature vector), and classifying and predicting the obtained feature vector to obtain a reproduction prediction value corresponding to the input feature map.

That is, the prediction modules in the second convolutional neural network can be divided into two types, namely a first prediction module and a second prediction module, wherein the second prediction module is used for processing the segmented feature map, and the first prediction module is used for processing the non-segmented feature map. And the pooling layer included in the second prediction module is a global pooling layer and the pooling layer included in the first prediction module is a tie pooling layer. In the embodiment, more representative local features can be obtained by dividing the certificate area, so that the reproduction identification accuracy rate can be improved, and the reproduction identification accuracy rate can be further improved by adding the overall feature map.

The process of processing a document correction image using a trained second convolutional neural network is described below by way of an example.

In the present example, where the document image to be identified is an identification card image, the architecture of this second convolutional neural network may be as shown in fig. 8.

(1) Inputting the certificate image to be identified into the second convolutional neural network, and processing the image through a backbone network of the second convolutional neural network to obtain a characteristic diagram;

(2) the feature map is divided into three parts by the dividing module according to the area dividing proportion shown in fig. 7, namely three divided feature maps respectively corresponding to the three parts of the identity card basic information, the identity card head portrait and the identity card number, the three divided feature maps and the non-divided feature map are respectively input into the corresponding predicting modules, namely the divided feature maps are processed by the second predicting module, and the non-divided feature maps are processed by the first predicting module. For each segmented feature map, the segmented feature map is accessed to the convolutional layer (c) and the global pooling layer (g) in the second prediction module, and since the number of channels for obtaining the segmented feature map is 512, the channel data is reduced to 128 by the convolutional layer of 512 x 1 x 128, and then the local value of each segmented feature map is averaged by the global pooling, so that a 128-dimensional feature vector (called local feature) is obtained. The undivided feature map is accessed into the convolutional layer (c) and the local pooling layer (g) in the first prediction module to obtain a feature vector (for convenience of distinction, this is referred to as a global feature). For global features and local features, a fully-connected layer with the dimension of 128 x 2 is connected behind the global features and the local features to conduct classification prediction, and the difference between a predicted value and a true value is calculated through softmax loss.

In one embodiment, the training process of the second convolutional neural network as in fig. 9 includes:

s210: acquiring a plurality of copied certificate images and non-copied certificate images as training samples;

s220: constructing an initial second convolutional neural network, wherein the parameter of the initial second convolutional neural network is a random value;

s230: the initial second convolutional neural network is iteratively trained using training samples, wherein, after each training, calculating the prediction loss value of the training according to the local image reproduction prediction value generated by the initial second convolutional neural network in the training, judging whether the loss of the initial second convolutional neural network is reduced to the minimum value according to the predicted loss value of the current training and the predicted loss values of the previous training, determining that training is complete upon determining that the loss of the initial second convolutional neural network decreases to a minimum, and the initial second convolutional neural network after the training is finished is used as the trained second convolutional neural network, and when the loss of the initial second convolutional neural network is not reduced to the minimum value, calculating the gradient of the loss function, updating the network parameters of the initial second convolutional neural network by using a gradient descent method, and carrying out next training.

The training process of the second convolutional neural network is similar to the training process of the first convolutional neural network, except that the loss calculation is different, so the training process can refer to the above related contents, and is not repeated herein, and the loss calculation is described below.

Fig. 1, fig. 2, fig. 5 and fig. 9 are schematic flow charts of a certificate duplication recognition method in one embodiment. It should be understood that, although the steps in the flowcharts of fig. 1, 2, 5 and 9 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1, 2, 5, and 9 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternatingly with other steps or at least some of the sub-steps or stages of other steps.

Based on the same inventive concept, the invention also provides a certificate copying and identifying device. In one embodiment, as shown in FIG. 10, the document reproduction identification device includes the following modules:

the image acquisition module 110 is used for acquiring a certificate image to be identified;

the first prediction module 120 is configured to process the document image to be recognized by using the trained first convolutional neural network, so as to obtain an overall image reproduction prediction value corresponding to the document image to be recognized;

the second prediction module 130 is configured to perform correction processing on a certificate image to be recognized to obtain a certificate correction image, perform preprocessing on the certificate correction image to obtain an overall feature map and a plurality of local feature maps, and calculate, by using a trained second convolutional neural network, a reproduction prediction value corresponding to the overall feature map and each local feature map to obtain a reproduction prediction value corresponding to the certificate image to be recognized;

and the identification result determining module 140 is configured to determine a reproduction identification result corresponding to the certificate image to be identified according to the overall image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified.

In one embodiment, the second prediction module comprises:

the mask extraction submodule is used for extracting a mask image of the certificate image to be identified by using the image segmentation model, and the mask image comprises a certificate area and a background area;

the corner detection submodule is used for detecting a plurality of corners in the mask image by using a corner detection algorithm;

the clustering submodule is used for determining 4 clustering centers from the plurality of angular points as four angular points of the certificate area by using a preset clustering algorithm;

and the perspective transformation submodule is used for carrying out perspective transformation processing on the certificate area according to the four corner points of the certificate area to obtain a certificate correction image.

In one embodiment, the second prediction module further comprises:

the characteristic map processing submodule is used for processing the certificate correction image into a characteristic map and taking the characteristic map as an integral characteristic map;

the segmentation submodule is used for segmenting the feature map based on a preset region segmentation proportion to obtain a plurality of local feature maps;

the pooling sub-module is used for pooling the whole feature map and each local feature map in each local feature map to obtain a global feature vector extracted from the whole feature map and a local feature vector extracted from each local feature map;

and the prediction submodule is used for respectively carrying out classification prediction processing on the global characteristic vectors and the local characteristic vectors to obtain a reproduction predicted value corresponding to the global characteristic diagram and a reproduction predicted value corresponding to each local characteristic diagram, and taking the reproduction predicted value corresponding to the global characteristic diagram and the reproduction predicted values corresponding to the local characteristic diagrams as local image reproduction predicted values corresponding to the certificate image to be identified.

In one embodiment, the apparatus further comprises:

the sample acquisition module is used for acquiring a plurality of copied certificate images and non-copied certificate images as training samples;

the network construction module is used for constructing an initial second convolutional neural network, and the parameter of the initial second convolutional neural network is a random value;

a network training module for performing iterative training on an initial second convolutional neural network by using a training sample, wherein after each training, a predicted loss value of the current training is calculated according to a local image reproduction predicted value generated by the initial second convolutional neural network in the training, whether the loss of the initial second convolutional neural network is reduced to a minimum value is judged according to the predicted loss value of the current training and the predicted loss values of multiple previous training, when the loss of the initial second convolutional neural network is determined to be reduced to the minimum value, the training is determined to be completed, the initial second convolutional neural network after the training is completed is used as the trained second convolutional neural network, when the loss of the initial second convolutional neural network is determined not to be reduced to the minimum value, the gradient of a loss function is calculated, and the network parameters of the initial second convolutional neural network are updated by using a gradient descent method, and performing the next training.

In one embodiment, the recognition result determining module includes:

the mean value calculation submodule is used for calculating the mean value of the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified;

and the recognition result determining submodule is used for comparing the average value with a preset threshold value, determining the copying recognition result corresponding to the certificate image to be recognized as copying when the average value is determined to be larger than the preset threshold value, and determining the copying recognition result corresponding to the certificate image to be recognized as non-copying when the average value is determined to be not larger than the preset threshold value.

For the specific definition of the document reproduction identification device, reference may be made to the above definition of the document reproduction identification method, which is not described in detail here. All or part of each module in the certificate copying and recognizing device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 11. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data such as the image of the certificate to be identified, and the specific stored data can also be referred to the definition in the above method embodiment. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of document reproduction identification.

Those skilled in the art will appreciate that the architecture shown in fig. 11 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:

acquiring a certificate image to be identified; processing the certificate image to be identified by using the trained first convolution neural network to obtain an integral image reproduction predicted value corresponding to the certificate image to be identified; correcting the certificate image to be recognized to obtain a certificate corrected image, preprocessing the certificate corrected image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, and respectively calculating reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network to obtain reproduction predicted values corresponding to the certificate image to be recognized; and determining a reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified.

In one embodiment, the processor executes a computer program to realize correction processing on a certificate image to be identified, and when the certificate corrected image is obtained, the following steps are further realized:

extracting a mask image of the certificate image to be identified by using the image segmentation model, wherein the mask image comprises a certificate area and a background area; detecting a plurality of corner points in the mask image by using a corner point detection algorithm; determining 4 clustering centers from the plurality of angular points by using a preset clustering algorithm as four angular points of the certificate area; and carrying out perspective transformation processing on the certificate area according to the four corner points of the certificate area to obtain a certificate correction image.

In one embodiment, the processor executes a computer program to implement preprocessing the certificate correction image to obtain an overall feature map and a plurality of local feature maps, and respectively calculates the reproduction predicted values corresponding to the overall feature map and the local feature maps by using a trained second convolutional neural network to obtain the reproduction predicted values corresponding to the local image of the certificate image to be recognized, and further implements the following steps:

processing the certificate correction image into a feature map, and taking the feature map as an integral feature map; segmenting the feature map based on a preset region segmentation proportion to obtain a plurality of local feature maps; pooling the overall characteristic diagram and each local characteristic diagram to obtain a global characteristic vector extracted from the overall characteristic diagram and a local characteristic vector extracted from each local characteristic diagram; and respectively carrying out classification prediction processing on the global feature vectors and the local feature vectors to obtain reproduction predicted values corresponding to the global feature maps and reproduction predicted values corresponding to the local feature maps, and taking the reproduction predicted values corresponding to the global feature maps and the reproduction predicted values corresponding to the local feature maps as local image reproduction predicted values corresponding to the certificate images to be identified.

In one embodiment, the processor executes the computer program, further implementing the steps of:

acquiring a plurality of copied certificate images and non-copied certificate images as training samples; constructing an initial second convolutional neural network, wherein the parameter of the initial second convolutional neural network is a random value; the initial second convolutional neural network is iteratively trained using training samples, wherein, after each training, calculating the prediction loss value of the training according to the local image reproduction prediction value generated by the initial second convolutional neural network in the training, judging whether the loss of the initial second convolutional neural network is reduced to the minimum value according to the predicted loss value of the current training and the predicted loss values of the previous training, determining that training is complete upon determining that the loss of the initial second convolutional neural network decreases to a minimum, and the initial second convolutional neural network after the training is finished is used as the trained second convolutional neural network, and when the loss of the initial second convolutional neural network is not reduced to the minimum value, calculating the gradient of the loss function, updating the network parameters of the initial second convolutional neural network by using a gradient descent method, and carrying out next training.

In one embodiment, when the processor executes the computer program to determine the copying identification result corresponding to the certificate image to be identified according to the integral image copying predicted value and the local image copying predicted value corresponding to the certificate image to be identified, the following steps are also realized:

calculating the average value of the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified; and comparing the mean value with a preset threshold value, determining the copying identification result corresponding to the certificate image to be identified as copying when the mean value is determined to be larger than the preset threshold value, and determining the copying identification result corresponding to the certificate image to be identified as non-copying when the mean value is determined to be not larger than the preset threshold value.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program is executed by a processor, and when the certificate image to be identified is subjected to correction processing to obtain the certificate corrected image, the following steps are further implemented:

In one embodiment, the computer program is executed by a processor,

preprocessing the certificate correction image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, respectively calculating the reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network, and when obtaining the reproduction predicted values corresponding to the local image of the certificate image to be recognized, further realizing the following steps:

In one embodiment, the computer program is executed by a processor, further implementing the steps of:

In one embodiment, the computer program is executed by a processor,

when determining the reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified, further realizing the following steps:

It will be understood by those skilled in the art that all or part of the processes of the embodiments of the methods described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for identifying a reproduction of a document, the method comprising:

acquiring a certificate image to be identified;

processing the certificate image to be identified by using a trained first convolution neural network to obtain an integral image reproduction predicted value corresponding to the certificate image to be identified;

correcting the certificate image to be recognized to obtain a certificate correction image, preprocessing the certificate correction image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, and respectively calculating reproduction predicted values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network to obtain reproduction predicted values corresponding to the local image of the certificate image to be recognized;

and determining a reproduction identification result corresponding to the certificate image to be identified according to the whole image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified.

2. The method for document reproduction identification as claimed in claim 1,

the step of correcting the certificate image to be identified to obtain the certificate corrected image comprises the following steps:

extracting a mask image of the certificate image to be identified by using an image segmentation model, wherein the mask image comprises a certificate area and a background area;

determining 4 clustering centers from the plurality of corner points by using a preset clustering algorithm as four corner points of the certificate area;

3. The method for document reproduction identification as claimed in claim 1,

the method comprises the following steps of preprocessing the certificate correction image to obtain an overall feature map and a plurality of local feature maps, respectively calculating the reproduction predicted values corresponding to the overall feature map and the local feature maps by using a trained second convolutional neural network, and obtaining the reproduction predicted values corresponding to the local image of the certificate image to be recognized, wherein the steps comprise:

and carrying out classified prediction processing on the global feature vectors and the local feature vectors to obtain reproduction predicted values corresponding to the global feature maps and reproduction predicted values corresponding to the local feature maps, and taking the reproduction predicted values corresponding to the global feature maps and the reproduction predicted values corresponding to the local feature maps as local image reproduction predicted values corresponding to the certificate images to be identified.

4. The method of document reproduction identification as claimed in claim 3,

the second convolutional neural network adopts a ResNet18 network as a backbone network and comprises a first prediction module and a plurality of second prediction modules;

the backbone network is used for processing the certificate rectification image into a feature map, and the feature map is the first four sub-modules of a ResNet18 network, wherein stride of a first convolution layer in a fourth sub-module of the ResNet18 network is set to be 1;

the first prediction module in the second convolutional neural network comprises a convolutional layer, a tie pooling layer, a full-link layer and a classifier, and the prediction module is used for pooling the input overall feature map to obtain an overall feature vector and performing classification prediction processing on the obtained overall feature vector to obtain a reproduction prediction value corresponding to the input overall feature map;

any one second prediction module in the second convolutional neural network comprises a convolutional layer, a global pooling layer, a full-link layer and a classifier, and the prediction module is used for pooling the input local feature map to obtain a local feature vector and performing classification prediction processing on the obtained local feature vector to obtain a reprinting prediction value corresponding to the input local feature map.

5. The method of document reproduction identification as claimed in claim 4,

the convolution layer included in the second prediction module is a convolution layer with a dimension of 512 x 1 x 128;

the fully connected layer included in the second prediction module is a fully connected layer having a dimension of 128 x 2.

6. The method for document reproduction identification as claimed in claim 1,

the training process of the second convolutional neural network comprises the following steps:

performing iterative training on the initial second convolutional neural network by using the training sample, wherein after each training, a predicted loss value of the current training is calculated according to a local image reproduction predicted value generated by the initial second convolutional neural network in the current training, whether the loss of the initial second convolutional neural network is reduced to a minimum value is judged according to the predicted loss value of the current training and the predicted loss values of multiple previous training, when the loss of the initial second convolutional neural network is determined to be reduced to the minimum value, the training is determined to be completed, the initial second convolutional neural network after the training is completed is used as the trained second convolutional neural network, when the loss of the initial second convolutional neural network is determined not to be reduced to the minimum value, the gradient of a loss function is calculated, and the network parameters of the initial second convolutional neural network are updated by using a gradient descent method, and performing the next training.

7. The method for document reproduction identification as claimed in claim 1,

the step of determining the reproduction identification result corresponding to the certificate image to be identified according to the overall image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate image to be identified comprises the following steps:

8. A document reproduction identification device, the device comprising:

the first prediction module is used for processing the certificate image to be recognized by using a trained first convolution neural network to obtain an integral image reproduction prediction value corresponding to the certificate image to be recognized;

the second prediction module is used for correcting the certificate image to be recognized to obtain a certificate corrected image, preprocessing the certificate corrected image to obtain an overall characteristic diagram and a plurality of local characteristic diagrams, and respectively calculating the reproduction prediction values corresponding to the overall characteristic diagram and the local characteristic diagrams by using a trained second convolutional neural network to obtain the reproduction prediction values corresponding to the certificate image to be recognized;

and the identification result determining module is used for determining the reproduction identification result corresponding to the certificate image to be identified according to the integral image reproduction predicted value and the local image reproduction predicted value corresponding to the certificate correction image.

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented when the computer program is executed by the processor.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.