CN114926829A

CN114926829A - Certificate detection method and device, electronic equipment and storage medium

Info

Publication number: CN114926829A
Application number: CN202210597261.0A
Authority: CN
Inventors: 王小东; 朱羽; 廖浩; 吕文勇; 周智杰
Original assignee: Chengdu New Hope Finance Information Co Ltd
Current assignee: Chengdu New Hope Finance Information Co Ltd
Priority date: 2022-05-30
Filing date: 2022-05-30
Publication date: 2022-08-19

Abstract

The application provides a certificate detection method, a certificate detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: marking a plurality of texts of the certificate image according to the text information in the certificate image to obtain marking information; according to the text information and the labeling information, segmenting the text in the text to obtain segmentation data, wherein the segmentation data comprises a plurality of text boxes; calculating the proportional characteristics of the text boxes according to the segmentation data to obtain proportional characteristic data; and detecting the positive alignment of the certificate image according to the proportion characteristic. By calculating the proportional characteristic data through calculating the vertexes of the external polygons of the plurality of text boxes, whether the text in the certificate image is over against the screen or not is detected according to the proportional characteristic data, and the certificate detection efficiency is improved. And the certificate is comprehensively identified and detected, so that the certificate detection accuracy is improved.

Description

Certificate detection method and device, electronic equipment and storage medium

Technical Field

The application relates to the field of image recognition, in particular to a certificate detection method and device, an electronic device and a storage medium.

Background

At present, many business transactions need to identify certificates, when users take pictures and upload certificates for OCR (Optical Character Recognition), the problems of incomplete information extraction, wrong text extraction, text extraction failure and the like exist in Character extraction often because certificate images taken by the users do not meet requirements, and further the OCR link failure rate is high. Therefore, in the identification of the certificate, it is necessary to ensure that the certificate photograph meets the identification requirement by detection.

Disclosure of Invention

The embodiment of the invention aims to provide a certificate detection method, a certificate detection device, electronic equipment and a storage medium, wherein a text is marked according to text information of a certificate through a marking tool, and only required texts can be marked with data; segmenting a text required by people by utilizing a segmentation algorithm to obtain the vertex coordinates and the text box probability of each segmented text box; calculating the slope of each text box according to the vertex coordinates of each divided text box; and segmenting according to the slope and a certain threshold value, and detecting the certificate image. And each pixel is segmented through a segmentation algorithm, and the generated probability graph is converted into an enclosure of a text in the certificate, so that whether the text in the certificate image is over against a screen is detected according to the text box proportion characteristic data, and the certificate detection efficiency is improved.

Through carrying out light detection to the certificate, whether have the certificate to detect, whether the certificate detects in the frame, the certificate direction detects, the certificate text is just to screen detection, the certificate integrity detects, certificate fuzzy detection, the true and false detection of certificate and certificate manufacture false detection, realize carrying out comprehensive discernment to the certificate and detect, improve the accuracy and the efficiency that the certificate detected simultaneously for user experience feels better.

In a first aspect, an embodiment of the present application provides a certificate detection method, including: marking a plurality of texts of the certificate image according to the text information in the certificate image to obtain marking information; according to the text information and the labeling information, segmenting the text in the text to obtain segmentation data, wherein the segmentation data comprises a plurality of text boxes; calculating the proportional characteristics of the text boxes according to the segmentation data to obtain proportional characteristic data; and detecting the positive alignment of the certificate image according to the proportion characteristic.

In the implementation process, the text in the certificate image is labeled, the text in the certificate image is divided into a plurality of text boxes, and the proportional characteristic data is calculated by calculating the vertexes of the external polygons of the text boxes, so that whether the text in the certificate image is over against the screen or not is detected according to the proportional characteristic data, and the certificate detection efficiency is improved.

Optionally, in this embodiment of the present application, the segmentation data further includes text box vertex coordinate information; calculating the proportion characteristics of a plurality of text boxes according to the segmentation data to obtain proportion characteristic data, wherein the proportion characteristic data comprises the following steps: sorting the text boxes according to the vertex coordinate information of the text boxes to obtain sorting data; and calculating the proportional characteristics of the plurality of text boxes according to the sequencing data of the text boxes and the coordinate information of the vertexes of the plurality of text boxes to obtain proportional characteristic data, wherein the proportional characteristic data comprises slope data. In the implementation process, after the text is divided into a plurality of text boxes, the unordered text boxes are sorted, and the proportion feature data of the text boxes are calculated according to the vertex coordinates of the text boxes in the division data and the sorting data of the text boxes, wherein the proportion feature data comprises slope data. And detecting the text inclination of the identity card by a method of calculating the slope of the text box.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: obtaining coordinate information of the certificate image, and calculating the aspect ratio of the certificate image according to the coordinate information; determining the detection frame parameters of the target object detection model according to the aspect ratio of the certificate image; identifying a certificate image through a target object detection model; wherein the target object detection model comprises a Yolov5 network.

In the implementation process, the detection frame parameters of the target object detection model are determined according to the aspect ratio of the certificate image, so that the certificate detection precision and performance are improved; and constructing a target of the identity card detection model by taking the Yolov5 network as a basic model framework, and detecting the certificate by using the target object detection model.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: processing the collected image by using computer vision software to extract a certificate image; adjusting the certificate image to a preset size to obtain a certificate image to be identified; inputting a certificate image to be identified into a preset identity card integrity detection model to obtain a first feature classification of the certificate image to be identified; wherein, ID card integrity detection model includes: the identification card comprises a convolutional neural network, a plurality of convolutional layers, a pooling layer and a SoftMax function, wherein the first characteristic is used for classifying whether the identification card of the identification card image to be identified is complete or not.

In the implementation process, the to-be-identified document image is input into a preset identity card integrity detection model, a first feature classification of the to-be-identified document image is obtained, and the first feature classification represents whether the identity card of the to-be-identified document image is complete or not. The integrity of the certificate is detected through the neural network with the 7-layer structure, and the integrity detection model of the identity card is a two-classification model, so that the detection speed of the integrity of the certificate is improved, and the user experience is improved.

Optionally, in an embodiment of the present application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: processing the collected image by using computer vision software to extract a certificate image; adjusting the certificate image to a preset size to obtain a certificate image to be identified; inputting the certificate image to be identified into a preset identity card shooting direction detection model to obtain a second characteristic classification of the certificate image to be identified; wherein, the direction detection model is shot to the ID card includes: the identification card comprises a convolutional neural network, a plurality of convolutional layers, a pooling layer, a full connection layer and a SoftMax function, and the second characteristic is used for classifying and representing the direction of the identification card of the to-be-identified certificate image.

In the implementation process, the acquired image is processed to obtain a to-be-identified document image, the to-be-identified document image is input into a preset identification card shooting direction detection model, a second characteristic classification of the to-be-identified document image is obtained, and the second characteristic classification represents the identification card direction of the to-be-identified document image. The identity card shooting direction detection model is a lightweight neural network, and the direction of the identity card can be rapidly judged.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: acquiring a certificate image through a detection frame of the terminal equipment to obtain shooting data; calculating the distance between the detection frame and the top and the left of the shooting frame according to the shooting data; the shooting data comprises the height of a shooting frame, the width of the shooting frame, the height of a detection frame and the width of the detection frame; acquiring certificate image data according to the coordinate information of the certificate image; the certificate image data comprise certificate image height, certificate image width, collected image height and collected image width; calculating the distance between the certificate image and the top and the left of the collected image according to the certificate image data; and detecting the certificate image according to the shooting data and the certificate image data. In the implementation process, whether the certificate is in the detection frame or not is judged according to the shooting data and the certificate image data, and corresponding prompt is given, so that the certificate detection accuracy is improved. The problem that the position of the shot identity card is not fixed, so that the detection of the identity card fails is avoided.

Optionally, in this embodiment of the application, after detecting the positive feature of the certificate image according to the scale feature data, the method further includes: calculating the aspect ratio of the certificate image according to the coordinate information; classifying the proportion characteristic data to obtain proportion characteristic classification data; calculating the aspect ratio data corresponding to each proportion feature classification data to obtain proportion feature classification aspect ratio data; and classifying the aspect ratio data according to the aspect ratio and the proportional characteristic of the certificate image to detect the certificate image.

In the implementation process, the certificate image is detected according to the aspect ratio and the proportional characteristic classification aspect ratio data of the certificate image, whether the identity card is complete or not is detected through the aspect ratio and the proportional characteristic classification aspect ratio data of the certificate image subjected to statistical analysis, and the certificate detection precision and timeliness are improved.

Optionally, in an embodiment of the present application, after detecting the positive alignment of the certificate image according to the scale feature data, the method further includes: acquiring a large number of characteristic certificate images, and performing enhancement processing on the characteristic certificate images to obtain a certificate image training set; wherein the characteristic document image comprises a blurred, clear, counterfeit and authentic document image; training a certificate image training set through a convolutional neural network to obtain a certificate fuzzy feature recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model; respectively and sequentially inputting the acquired certificate image into a certificate feature fuzzy recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model to obtain a third feature classification, a fourth feature classification and a fifth feature classification of the certificate image; the third feature classification represents a fuzzy recognition result of the acquired certificate image; the fourth characteristic classification represents the true and false identification results of the acquired certificate image; and the fifth characteristic classification represents the counterfeiting identification result of the acquired certificate image.

In the implementation process, a certificate fuzzy feature recognition model, a certificate true and false feature recognition model and a certificate fake feature recognition model are obtained by training the collected sample image; and the fuzzy, true and false identification is carried out on the certificate through the certificate fuzzy feature identification model, the certificate true and false feature identification model and the certificate false feature identification model. The failure of certificate identification caused by network reasons is avoided, and the authenticity of the certificate is detected; detecting the authenticity of the identity card and prompting a user in real time; the certificate can be identified again on the basis of identification card authenticity identification, comprehensive detection of the certificate is achieved, the pass rate of certificate detection is improved, and user experience is improved on the whole.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: acquiring a collected image comprising the certificate image and RGB data of the collected image; converting the RGB data into HSV data; and judging whether the shooting light of the acquired image is within a preset range or not according to the HSV data and a preset HSV data threshold. Through utilizing color space information to carry out light identification, detect the light of certificate detection environment to judge whether the light that the certificate detected is in reasonable within range.

In a second aspect, an embodiment of the present application further provides a credential detection device, including: the marking module is used for marking a plurality of texts of the certificate image according to the text information in the certificate image to obtain marking information; the segmentation module is used for segmenting the text in the text according to the text information and the labeling information to obtain segmentation data, and the segmentation data comprises a plurality of text boxes; the calculation module is used for calculating the proportional characteristics of the text boxes according to the segmentation data to obtain proportional characteristic data; and the detection module is used for detecting the positive alignment of the certificate image according to the proportion characteristics.

Optionally, in an embodiment of the present application, the certificate detection apparatus, wherein the segmentation data further includes text box vertex coordinate information; the calculation module is also used for sorting the text boxes according to the vertex coordinate information of the text boxes to obtain sorting data; and calculating the proportional characteristics of the plurality of text boxes according to the sequencing data of the text boxes and the vertex coordinate information of the plurality of text boxes to obtain proportional characteristic data, wherein the proportional characteristic data comprises slope data.

Optionally, in an embodiment of the present application, the credential detection device further includes: the certificate detection module is used for obtaining the coordinate information of the certificate image and calculating the aspect ratio of the certificate image according to the coordinate information; determining a detection frame parameter of a target object detection model according to the aspect ratio of the certificate image; identifying the certificate image through the target object detection model; wherein the target object detection model comprises a Yolov5 network.

Optionally, in an embodiment of the present application, the document detection apparatus further includes: the first integrity detection module is used for processing the acquired image by using computer vision software to extract the certificate image; adjusting the certificate image to a preset size to obtain a certificate image to be identified; inputting the certificate image to be identified into a preset identity card integrity detection model to obtain a first feature classification of the certificate image to be identified; wherein, the identity card integrity detection model comprises: the identification card comprises a convolutional neural network, a plurality of convolutional layers, a pooling layer and a SoftMax function, wherein the first characteristic is used for classifying and representing whether the identification card of the image of the certificate to be identified is complete or not.

Optionally, in an embodiment of the present application, the document detection apparatus further includes: the direction recognition module is used for processing the collected image by using computer vision software so as to extract the certificate image; adjusting the certificate image to a preset size to obtain a certificate image to be identified; inputting the to-be-identified certificate image into a preset identity card shooting direction detection model to obtain a second feature classification of the to-be-identified certificate image; wherein, the ID card shooting direction detection model comprises: convolutional neural network, a plurality of convolution layers, pooling layer, full connection layer and SoftMax function, the categorised sign of second characteristic is the ID card direction of waiting to discern the certificate image.

Optionally, in an embodiment of the present application, the document detection apparatus further includes: the in-frame detection module is used for acquiring the certificate image through a detection frame of the terminal equipment to obtain shooting data; calculating the distance between the detection frame and the top and the left of the shooting frame according to the shooting data; the shooting data comprises the height of a shooting frame, the width of the shooting frame, the height of a detection frame and the width of the detection frame; acquiring certificate image data according to the coordinate information of the certificate image; wherein the credential image data comprises a credential image height and a credential image width; calculating the distance between the certificate image and the top and the left of the shooting frame according to the certificate image data and the shooting data; and detecting the certificate image according to the shooting data and the certificate image data.

Optionally, in an embodiment of the present application, the document detection apparatus further includes: the second integrity detection module is used for calculating the aspect ratio of the certificate image according to the coordinate information; classifying the proportional characteristic data to obtain proportional characteristic classification data; calculating the aspect ratio data corresponding to each proportion feature classification data to obtain proportion feature classification aspect ratio data; and classifying the aspect ratio data according to the aspect ratio and the proportion characteristic of the certificate image to detect the certificate image.

Optionally, in an embodiment of the present application, the document detection apparatus further includes: the certificate real identification module is used for acquiring a large number of characteristic certificate images and enhancing the characteristic certificate images to acquire a certificate image training set; wherein the characteristic document image comprises a blurred, sharp, counterfeit and authentic document image; training a certificate image training set through a convolutional neural network to obtain a certificate fuzzy feature recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model; respectively and sequentially inputting the acquired certificate image into a certificate feature fuzzy recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model to obtain a third feature classification, a fourth feature classification and a fifth feature classification of the certificate image; the third feature classification represents a fuzzy recognition result of the acquired certificate image; the fourth characteristic classification represents the true and false identification results of the acquired certificate image; and the fifth characteristic classification represents the counterfeiting identification result of the acquired certificate image.

Optionally, in an embodiment of the present application, the document detection apparatus further includes: the light ray identification module is used for acquiring a collected image comprising the certificate image and RGB data of the collected image; converting the RGB data into HSV data; and judging whether the shooting light of the acquired image is within a preset range or not according to the HSV data and a preset HSV data threshold.

In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory, the memory storing processor-executable machine-readable instructions which, when executed by the processor, perform a method as described above.

In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the above-described method.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

FIG. 1 is a schematic flow chart of a method for detecting a document according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of coordinates of a textbox according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of corner coordinates of a certificate image according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an identity card integrity detection model according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an identity card shooting direction detection model provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of a certificate image, a detection frame and a terminal device provided by an embodiment of the application;

FIG. 7 is a schematic view of a document testing device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are merely used to more clearly illustrate the technical solutions of the present application, and therefore are only examples, and the protection scope of the present application is not limited thereby.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

In the description of the embodiments of the present application, the technical terms "first", "second", and the like are used only for distinguishing different objects, and are not to be construed as indicating or implying relative importance or implicitly indicating the number, specific order, or primary-secondary relationship of the technical features indicated. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

For convenience of description, coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) are used in some embodiments of the present application to describe the coordinates, but it should be understood that these coordinates represent only one code number and are not limited to the same coordinates.

Before describing the scan filtering method, apparatus, electronic device and storage medium provided in the present application, the concept of the DB algorithm in the embodiments of the present application is introduced:

db (differential localization) algorithm: the method is called as differential binarization processing, and scene text detection based on segmentation is to convert a probability map (thermodynamic diagram) generated by a segmentation method into a boundary box and a character area, wherein a binarization post-processing process is included. The binarization process is very critical, and the conventional binarization operation is implemented by setting a fixed threshold, which is difficult to adapt to complex and variable detection scenarios.

Please refer to fig. 1, which illustrates a flowchart of a certificate detection method provided in an embodiment of the present application.

Step S110: and marking a plurality of texts of the certificate image according to the text information in the certificate image to obtain marking information.

The embodiment of the step S110 includes: after the certificate image is acquired, labeling a plurality of texts of the certificate image according to text information in the certificate image, wherein the text information can be identifications of different texts, such as text color labeling and the like, and the text required by people can be labeled according to the text color. Specifically, for example, a certificate image to be labeled is selected, a text to be labeled is continuously selected, and different labeling schemes are selected by using an image labeling tool, wherein the labeling schemes can be polygons, rectangles and the like. After the labeling is finished, a labeling area is formed, and then the labeling information is labeled and input, so that the labeling can be finished. And if the text crossing the text lines appears in the text, marking according to the lines, and marking each line.

Step S120: and segmenting the text in the text according to the text information and the labeling information to obtain segmented data, wherein the segmented data comprises a plurality of text boxes.

The embodiment of the step S120 includes: according to the method, texts in the texts are segmented according to text information and labeling information, specifically, model training can be carried out through a DB network, a text box segmentation algorithm is trained, binarization processing is carried out in the text box segmentation algorithm in the network, a binarization threshold value can be automatically set by the segmentation network, a probability graph output by the network is segmented through a fixed threshold value, post-processing is simplified, and performance is enhanced. The method comprises the steps of using a text box segmentation algorithm to segment a marked text in a pixel level to obtain segmentation data, wherein the segmentation data comprises a plurality of text boxes, the text boxes can be external matrixes of the segmented text, and in the identity document, if people pay more attention to the text with black characters, the external matrixes with the black characters can be segmented.

Step S130: and calculating the proportional characteristics of the plurality of text boxes according to the segmentation data to obtain proportional characteristic data.

The embodiment of the step S130 includes: the segmentation data further comprises vertex coordinates of the segmented text box, the proportion feature of the text box is calculated according to the vertex coordinates of the text box, the text box can be provided with four vertexes, 2 proportion feature data are calculated according to the four vertex coordinates, and the maximum value of the proportion feature data is taken, wherein the proportion feature data comprise slopes, specifically, the slopes of an external matrix of black characters in the identity document.

Step S140: and detecting the positive alignment of the certificate image according to the proportion characteristic data. And detecting the certificate image based on certain threshold segmentation according to the proportion characteristic, and determining whether the text is over against the screen. The positive property of the certificate image is detected, for example, whether the certificate image is directly opposite to the acquisition terminal is detected, the acquisition terminal is a mobile terminal for identifying the certificate image, namely, whether a text box of the certificate image is directly opposite to a screen of the acquisition terminal is detected through a slope, and therefore whether the certificate image is directly opposite to the screen of the acquisition terminal is determined.

In the implementation process, the text in the certificate image is labeled and is segmented at the pixel level, the text in the certificate image is segmented into a plurality of text boxes, and the proportional characteristic data is calculated by calculating the vertexes of the external polygons of the text boxes, so that whether the text in the certificate image is over against the screen or not is detected according to the proportional characteristic data, and the certificate detection efficiency is improved.

Please refer to fig. 2, which illustrates a schematic diagram of coordinates of a text box according to an embodiment of the present application.

Optionally, in an embodiment of the present application, the segmentation data further includes coordinate information of a vertex of the text box; calculating the proportional characteristics of a plurality of text boxes according to the segmentation data to obtain proportional characteristic data, wherein the proportional characteristic data comprises the following steps: sorting the text boxes according to the vertex coordinate information of the text boxes to obtain sorting data; and calculating the proportional characteristics of the plurality of text boxes according to the sequencing data of the text boxes and the vertex coordinate information of the plurality of text boxes to obtain proportional characteristic data, wherein the proportional characteristic data comprises slope data.

The implementation mode of the steps is as follows: the segmentation data also includes text box vertex coordinate information, and text box probabilities. Specifically, an angled polygon of each text box is segmented, and the coordinates of the polygon text box are expressed as: { [ (x1, y1), (x2, y2), (x3, y3), (x4, y4) ], proba }, wherein the first item is coordinate information of four vertices of the quadrangle and the second item is a probability of the divided text box. After obtaining the coordinate information of the text boxes, sorting the text boxes according to the coordinate information, calculating proportion characteristic data for the sorted text boxes, wherein the proportion characteristic data comprises slope data, the calculation formula of the slope data can be y2-y1/x2-x1, y4-y3/x4-x3, each text box obtains two slope data through calculation, and the maximum value of the slope data is taken. And storing the slope data of each text box into a slope data table according to the arrangement sequence of the text boxes.

In the implementation process, after the text is divided into a plurality of text boxes, the unordered text boxes are sorted, and the proportion feature data of the text boxes are calculated according to the vertex coordinates of the text boxes in the division data and the sorting data of the text boxes, wherein the proportion feature data comprises slope data. And detecting the text inclination of the identity card by a method of calculating the slope of the text box.

Optionally, in this embodiment of the present application, the slope data includes a slope of a first text box and a slope of a last text box corresponding to the sorting data; detecting the certificate image according to the proportion characteristics, comprising: and detecting whether the certificate image is over against the screen or not according to the slope of the first text box, the slope of the last text box and a preset slope threshold value.

The implementation mode of the steps is as follows: the slope data comprises the slope of the first text box and the slope of the last text box corresponding to the sorting data, the slope of the first text box and the slope of the last text box can be taken out from the slope data table, and whether the text in the certificate image is over against the screen or not is detected based on certain threshold segmentation.

In the implementation process, after the text is divided into a plurality of text boxes, the unordered text boxes are sequenced, whether the text is inclined or not can be determined by taking the slope of the first text box and the slope of the last text box in the slope data, and the certificate detection efficiency is improved.

Optionally, in this embodiment of the application, where the text information includes a text color, labeling the multiple texts of the certificate image according to the text information in the certificate image includes: and marking the plurality of texts of the certificate image according to the text colors of the plurality of texts in the certificate image by using an image marking tool.

The implementation manner of the above steps is as follows: the text information comprises text colors, and the text in the certificate image is labeled by using an image labeling tool according to different font colors. Specifically, the certificate image generally includes a black font and a blue font, the black font is a font that we need to extract or process, the font that we need to extract can be easily and conveniently distinguished through font color, so as to perform annotation, wherein the image annotation tool may be LabelMe. When the image marking tool is used for marking the certificate image, only the text which needs to be extracted or processed can be marked according to the information such as the text color and the like.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: acquiring coordinate information of the certificate image, and calculating the aspect ratio of the certificate image according to the coordinate information; determining a detection frame parameter of a target object detection model according to the aspect ratio of the certificate image; identifying a certificate image through a target object detection model; wherein the target object detection model comprises a Yolov5 network.

The implementation manner of the above steps is as follows: and obtaining the coordinate information of the certificate image, and calculating the aspect ratio of the certificate image according to the coordinate information, wherein the obtained coordinate information of the certificate image can be obtained through a marking tool or can be marked manually. The coordinate information of the certificate can be coordinate information of four vertexes of the identity card. The coordinate information of the document image may be StartX, StartY, EndX, EndY. The aspect ratio of the document image is calculated according to the coordinate information, and specifically, the aspect ratio calculation formula of the document image may be (EndX-StartX)/(EndY-StartY). Through a large amount of data analysis and calculation, the aspect ratio of the certificate image is usually in the range of 1.2-1.8, and therefore through the aspect ratio in the range of 1.2-1.8, the proportion parameters for determining the object detection frame can be 1:1.2, 1:1.5 and 1: 1.8.

Determining a detection frame parameter of a target object detection model according to the aspect ratio of the certificate image, specifically determining a proportion parameter of an object detection frame according to the vertex coordinate information of the identity card, and identifying the certificate image through the target object detection model after the proportion parameter is designed; the target object detection model comprises a Yolov5 network. The target object detection model uses a backbone network defaulted by Yolov5, and an FP16 is opened to accelerate the model operation speed. The method comprises the steps of building a model by using a Pythrch, training the model by using a GPU, storing the model, and detecting an input image by using a model prediction interface.

In the implementation process, the detection frame parameters of the target object detection model are determined according to the aspect ratio of the certificate image, so that the certificate detection precision is improved; and constructing a target of the identity card detection model by taking the Yolov5 network as a basic model architecture, and detecting whether the identity card detection model is over against the screen or not by using the target object detection model. The collected image is subjected to real-time dead-screen pre-detection, a user is prompted in real time until the collected image is dead-screen, then an OCR recognition algorithm is called to extract a text, the detection efficiency is high, millisecond-level response is achieved, and user experience is improved.

Please refer to fig. 3, which illustrates a schematic diagram of corner coordinates of a certificate image according to an embodiment of the present application.

Optionally, in an embodiment of the present application, the certificate detection method further includes: acquiring a certificate image; obtaining a plurality of corner coordinates of the certificate image through a regression model; obtaining a first slope and a second slope through a plurality of corner point coordinates; and judging whether the certificate image is over against the screen or not according to the first slope, the second slope and a preset slope threshold.

The implementation manner of the above steps is as follows: and (3) utilizing computer vision software to perform cutout processing on the detection frame for collecting the certificate image, and taking out the image of the certificate part, namely the certificate image, from the collected image. Wherein the computer vision software comprises Opencv. And obtaining a plurality of corner point coordinates of the certificate image through a regression model, wherein the regression model comprises a regression network. And inputting the acquired certificate image into a regression model, and performing full-range regression on the certificate image through a regression network of the regression model to obtain coordinates of four corner points of a text region of the certificate image. The corner point coordinates of the certificate image are obtained only through the regression model, feature classification is not needed through a classification layer, and the calculation complexity is reduced.

As shown in fig. 3, the obtained four corner point coordinates may be [ (x1, y1), (x2, y2), (x3, y3), (x4, y4) ], where (x1, y1) is the upper left corner point coordinate, (x2, y2) is the upper right corner point coordinate, (x3, y3) is the lower right corner point coordinate, and (x4, y4) is the lower left corner point coordinate. Obtaining a first slope and a second slope according to the four intersection point coordinates, wherein a calculation formula of the first slope can be y2-y1/x2-x1, and the calculation formula of the first slope is used as the slope of the uppermost text of the certificate image; the calculation formula of the second slope is y4-y3/x4-x3, and the calculation formula of the second slope is used as the slope of the lowermost text of the certificate image. And judging whether the certificate image is over against the screen or not according to the first slope, the second slope and a preset slope threshold, wherein the preset slope threshold can be set according to actual requirements, and if the first slope and the second slope are obtained to be respectively in the range of the respective slope threshold, namely the slope of the uppermost row of characters of the certificate image and the slope of the lowermost row of characters of the certificate image are both in the range of the respective slope threshold, determining that the text of the certificate image is over against the screen at the moment.

It should be noted that the detection of whether the certificate is over the screen is accomplished through two algorithms, namely, the aspect ratio of the certificate image is calculated through the coordinate information of four vertexes of the identity card, whether the identity card is over the screen is detected according to the aspect ratio, the proportion characteristic data is calculated through calculating the vertexes of the circumscribed polygons of the text boxes, and whether the text in the certificate image is over the screen is detected according to the proportion characteristic data. If any algorithm of the two algorithms detects that the certificate is not over against the screen, the certificate is determined to be not over against the screen.

The certificate image is detected through two algorithms, whether the certificate is over against the screen or not is judged, and the situation that the acquired certificate image is not clear enough due to network problems, shaking of shooting equipment and other reasons is avoided, so that the certificate detection result is inaccurate, and the precision of the certificate over against the screen is improved.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: processing the collected image by using computer vision software to extract a certificate image; adjusting the certificate image to a preset size to obtain a certificate image to be identified; inputting a certificate image to be recognized into a preset identity card integrity detection model to obtain a first feature classification of the certificate image to be recognized; wherein, ID card integrity detection model includes: the identification card recognition system comprises a convolutional neural network, a plurality of convolutional layers, a pooling layer and a SoftMax function, wherein a first characteristic is used for classifying and representing whether the identification card of a to-be-recognized identification card image is complete or not.

The implementation manner of the above steps is as follows: and processing the collected image by using computer vision software to extract the certificate image. Specifically, the computer vision software is used for carrying out cutout processing on the collected image and taking out the image of the evidence part from the collected image. Wherein the computer vision software comprises Opencv.

And adjusting the certificate image to a preset size to obtain the certificate image to be identified, wherein the preset size can be 300 × 300. And inputting the certificate image to be recognized into a preset identity card integrity detection model to obtain a first feature classification of the certificate image to be recognized.

Please refer to fig. 4, which illustrates a schematic structural diagram of an identity card integrity detection model provided in an embodiment of the present application.

Specifically, the identity card integrity detection model can have 7 layers of neural networks, the image of the certificate to be identified is input into the preset identity card integrity detection model, after 3 × 3 convolution, the convolution kernel size is 256, the step length is 1, no completion or pooling is needed, and a 100 × 256 feature map is obtained after convolution;

after 1 × 1 convolution, the convolution kernel size is 256, the step size is 1, no completion is needed, the maximum pooling is adopted, the pooling frame size is 2, and a characteristic diagram of 50 × 256 is obtained after convolution;

after 1 × 1 convolution, the convolution kernel size is 128, the step size is 1, no padding is needed, the maximum pooling is adopted, the pooling frame size is 2, and the image size after convolution is 25 × 128;

after convolution with 1 × 1, the convolution kernel size is 256, the step size is 1, no padding is needed, and the image size after convolution is 25 × 256;

after convolution with 2 × 2, the convolution kernel size is 1024, the step size is 1, 1 pixel is used for completion, and a feature map with 13 × 1024 is obtained after convolution;

after convolution with 2 × 2, the convolution kernel size is 512, the step size is 1, padding is 0, and a feature map with 6 × 512 is obtained after convolution;

and finally, obtaining a first characteristic classification of the certificate image to be identified through Softmax secondary classification.

In the implementation process, the to-be-identified document image is input into a preset identity card integrity detection model, a first feature classification of the to-be-identified document image is obtained, and the first feature classification represents whether the identity card of the to-be-identified document image is complete or not. The integrity of the certificate is detected through the neural network with the 7-layer structure, and the identity card integrity detection model is a two-classification model, so that the detection speed of the integrity of the certificate is improved.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: processing the collected image by using computer vision software to extract a certificate image; adjusting the certificate image to a preset size to obtain a certificate image to be identified; inputting the certificate image to be recognized into a preset identity card shooting direction detection model to obtain a second characteristic classification of the certificate image to be recognized; wherein, the direction detection model is shot to the ID card includes: the convolution neural network, the plurality of convolution layers, the pooling layer, the full connection layer and the SoftMax function, and the second characteristic is used for classifying and representing the direction of the identity card of the certificate image to be identified. Wherein the second feature classification includes identity card integrity and identity card incompleteness.

Please refer to fig. 5, which illustrates a schematic structural diagram of an identity card shooting direction detection model provided in an embodiment of the present application.

The implementation manner of the above steps is as follows: the collected image is processed by using computer vision software to extract the certificate image, and specifically, the collected image can be subjected to matting processing by using the computer vision software Opencv to extract the certificate image. Adjusting a certificate image to a preset size according to equipment for identifying the certificate, for example, if the certificate is scanned through a mobile terminal, the certificate image can be adjusted to 400 × 300 and then input to a preset identity card shooting direction detection model, the preset identity card shooting direction detection model has a 6-layer network structure, specifically, the image input of 400 × 300 × 3 is subjected to 2 × 2 convolution, the convolution kernel size is 512, the step length is 1, no completion or pooling is needed, and a characteristic map of 200 × 150 × 512 is obtained after the convolution; after 1 × 1 convolution, the convolution kernel size is 256, the step size is 1, no completion is needed, maximum pooling is adopted, the pooling frame size is 2, and a characteristic diagram of 100 × 75 × 256 is obtained after convolution; performing convolution on the convolved image by 3 × 3, wherein the size of a convolution kernel is 128, the step size is 1, no compensation is needed, no pooling is needed, and a 33 × 25 × 128 feature map is obtained after convolution; performing convolution on the convolved image by 2 × 2, wherein the size of a convolution kernel is 512, the step size is 1, and the feature map of 16 × 12 × 512 is obtained after convolution without padding; performing convolution on the convolved image by 2 × 2, wherein the size of a convolution kernel is 256, the step size is 1, alignment is not required, and a 8 × 6 × 1024 feature map is obtained after convolution; and finally, obtaining a classification result representing the direction of the identity card of the certificate image to be recognized through Softmax secondary classification. The classification result of the identification card direction comprises 90 degrees, 180 degrees and 270 degrees.

Please refer to fig. 6, which illustrates a schematic diagram of a certificate image, a detection frame and a terminal device provided in the embodiment of the present application.

Optionally, in this embodiment of the application, before labeling the plurality of texts of the certificate image according to the text information in the certificate image, the method further includes: acquiring a certificate image through a detection frame of the terminal equipment to obtain shooting data; calculating the distance between the detection frame and the top and the left of the shooting frame according to the shooting data; the shooting data comprises the height of a shooting frame, the width of the shooting frame, the height of a detection frame and the width of the detection frame; acquiring certificate image data according to the coordinate information of the certificate image; the certificate image data comprise certificate image height, certificate image width, collected image height and collected image width; calculating the distance between the certificate image and the top and the left of the collected image according to the certificate image data; and detecting the certificate image according to the shooting data and the certificate image data.

According to the coordinate information of the certificate image and the coordinate information of the certificate image, reading the height and the width of the certificate image by utilizing Opencv, according to the coordinate information of the certificate image and the information of the collected image, the distance between the certificate image and the top and the left of the collected image can be obtained, according to the occupation ratio of the detection frame on the shooting frame and the distance between the detection frame and the top and the left of the shooting frame, the occupation ratio of the certificate image on the shooting frame and the distance between the certificate image and the top and the left of the shooting frame are compared, and whether the certificate image is in the detection frame is detected.

The implementation manner of the above steps is as follows: when gathering the certificate image to the detection frame through terminal equipment, can be through showing a detection frame on terminal equipment's screen, can gather complete certificate image when the certificate image is complete within detecting the frame. It is therefore necessary to detect whether the document image is within the detection frame.

Specifically, for example, the shooting data of the height of the shooting frame, the width of the shooting frame, the height of the detection frame, and the width of the detection frame of the terminal device is acquired. The terminal device can be a mobile phone, and the height of the shooting frame and the width of the shooting frame are respectively the height of a mobile phone screen and the width of the mobile phone screen. The height of the mobile phone screen is marked as cm _ h, and the width of the mobile phone screen is marked as cm _ w. Obtaining coordinate information of the document image, the coordinates of the document image may be [ (x1, y1), (x2, y2), (x3, y3), (x4, y4) ]; and obtaining the height of the certificate image and the width of the detection frame according to the coordinate information of the certificate image. The certificate image height mark is dt _ h, and dt _ h is y3-y 1; the document image width is labeled dt _ w, which is x3-x 1.

And reading the height and the width of the acquired image by utilizing Opencv, wherein the acquired image is an acquired original image, the height of the acquired image is marked as im _ h, and the width of the acquired image is marked as im _ w.

And calculating the distance between the certificate image and the top and the left of the collected image according to the coordinate information of the certificate image, the height of the collected image of the terminal equipment and the width of the collected image. The distance between the certificate image and the top of the collected image is marked as dt _ r, and the dt _ r is y 1; the distance between the certificate image and the left side of the collected image is marked as dt _ l, and the dt _ l is x 1.

And acquiring the width and the height of a detection frame for prompting the certificate shooting position, wherein the height of the detection frame is marked as re _ h, and the width of the detection frame is marked as re _ w.

Calculating the distance between the detection frame and the top and the left of the mobile phone screen, wherein the distance between the detection frame and the top of the mobile phone screen is marked as re _ r; the distance of the detection box from the left side of the mobile phone screen is marked as re _ l.

Firstly, judging whether the size of the certificate image and the size of the detection frame are in a reasonable range, wherein the judging method comprises the following steps:

and calculating first proportion data of which the width of the certificate image occupies the width of the acquired image, wherein the first proportion data is marked as imbw _ ratio, and the calculation formula of the imbw _ ratio is that the width of the detection frame is wider than that of the acquired image, namely, the imbw _ ratio is dt _ w/im _ w.

And calculating second proportion data of the height of the certificate image occupying the height of the acquired image, wherein the second proportion data is marked as imbh _ ratio, and the calculation formula of the imbh _ ratio is that the height of the detection frame is higher than that of the acquired image, namely, imhw _ ratio is dt _ h/im _ h.

And calculating third proportion data of which the width of the detection frame occupies the width of the mobile phone screen, wherein the third proportion data is marked as rebw _ ratio, and the calculation formula of the rebw _ ratio is the width of the mobile phone screen on the width of the detection frame, namely, the rebw _ ratio is equal to re _ w/cm _ w.

And calculating fourth proportion data of the height of the detection frame occupying the height of the mobile phone screen, wherein the fourth proportion data is marked as rebh _ ratio, and the calculation formula of the rebh _ ratio is that the height of the detection frame is higher than that of the mobile phone screen, namely rehw _ ratio is re _ h/cm _ h.

If the first proportion data is smaller than the third proportion data, the second proportion data is smaller than the fourth proportion data, and the two conditions are met simultaneously, the size of the certificate image is smaller than the size of the detection frame, specifically, if the imbw _ ratio < rebw _ ratio and the imhw _ ratio < rehw _ ratio are met simultaneously, the size of the certificate image is smaller than the size of the detection frame. If any one of the two conditions is not met, the certificate image is judged not to be in the detection frame.

Then judging whether the position height ratio of the certificate image in the collected image and the position height ratio of the detection frame in the mobile phone screen are in a reasonable range or not, wherein the method comprises the following steps:

and calculating proportion data of the top of the certificate image occupying the acquired image to obtain first proportion data, wherein the first proportion data is marked as an imbbth _ ratio, and the calculation formula of the imbbth _ ratio is that the distance between the certificate image and the top of the acquired image is greater than that of the acquired image, namely the imbbth _ ratio is dt _ r/im _ h.

And calculating proportion data of the bottom of the certificate image occupying the acquired image to obtain second proportion data, wherein the second proportion data is marked as imbbbh _ ratio, and the calculation formula of the imbbbh _ ratio is that the sum of the height of the certificate image and the distance from the certificate image to the top of the acquired image is higher than the acquired image, namely the imbbbh _ ratio is (dt _ h + dt _ r)/im _ h.

And calculating proportion data of the top of the detection frame occupying the height of the screen to obtain third proportion data, wherein the third proportion data is marked as a rebbth _ ratio, and a calculation formula of the rebbth _ ratio is that the height of the detection frame is higher than that of the mobile phone screen, namely, the rebbth _ ratio is re _ h/cm _ h.

And calculating proportion data of the bottom of the detection frame occupying the height of the screen to obtain fourth proportion data, wherein the fourth proportion data is marked as a rebbbh _ ratio, and the calculation formula of the rebbbh _ ratio is that the ratio of the height of the detection frame to the sum of the distance from the detection frame to the top of the mobile phone screen to the height of the screen is (re _ h + re _ r)/cm _ h.

If the first proportion data is larger than the third proportion data, the second proportion data is smaller than the fourth proportion data, and the two conditions are simultaneously met, the certificate image meets the range of the detection frame in the height ratio, specifically, if the conditions of imbbth _ ratio > rebbbth _ ratio and the conditions of imbbbh _ ratio < rebbbbh _ ratio are simultaneously met, the condition that the height ratio of the certificate image meets the range of the detection frame is indicated, and if any one of the two conditions is not met, the certificate image is judged not to be in the detection frame.

And then judging whether the position width ratio of the certificate image in the acquired image and the position width ratio of the detection frame in the mobile phone screen are in a reasonable range or not, wherein the method comprises the following steps of:

and calculating proportion data occupying the acquired image on the left side of the certificate image to obtain fifth proportion data, wherein the fifth proportion data is marked as imbblh _ ratio, and the calculation formula of the imbblh _ ratio is that the distance between the certificate image and the left side of the acquired image is wider than that of the acquired image, namely the imbblh _ ratio is dt _ l/im _ w.

Calculating proportion data occupying the acquired image on the right side of the certificate image to obtain sixth proportion data, wherein the sixth proportion data is marked as imbbrh _ ratio, and a calculation formula of the imbbrh _ ratio is as follows: the sum of the distance between the certificate image and the left side of the acquired image and the width of the certificate image is larger than the acquired image width, namely the acquired image width is (dt _ w + dt _ l)/im _ w.

Calculating the proportion data of the left side of the detection frame occupying the width of the screen to obtain seventh proportion data, wherein the third proportion data is marked as rebblh _ ratio, and the calculation formula of the rebblh _ ratio is as follows: the distance between the detection frame and the left side of the mobile phone screen is wider than that of the mobile phone screen, namely, the rebblh _ ratio is re _ l/cm _ w.

And calculating the proportion data of the right side of the detection frame occupying the width of the screen to obtain eighth proportion data, wherein the eighth proportion data is marked as rebbrh _ ratio, and the calculation formula of the rebbrh _ ratio is as follows: the sum of the distance between the detection frame and the left side of the mobile phone screen is larger than the width of the mobile phone screen, namely the rebbrh _ ratio is (re _ l + re _ w)/cm _ w.

If the fifth proportion data is larger than the seventh proportion data, the sixth proportion data is smaller than the eighth proportion data, and the two conditions are simultaneously met, the width ratio of the certificate image is shown to meet the range of the detection frame, specifically, if the imbblh _ ratio > rebblh _ ratio and the imbbrh _ ratio < rebbrh _ ratio, the width ratio of the certificate image is shown to meet the range of the detection frame, and if any one of the two conditions is not met, the certificate image is judged not to be in the detection frame.

To sum up, if the first proportion data is smaller than the third proportion data, the second proportion data is smaller than the fourth proportion data, the first proportion data is larger than the third proportion data, the second proportion data is smaller than the fourth proportion data, the fifth proportion data is larger than the seventh proportion data, and the sixth proportion data is smaller than the eighth proportion data, it is determined that the identity card is detected in the retest box. Namely, if all six judgment conditions are met, the certificate image is determined to be in the detection frame, and if any judgment condition is not met, the certificate image is determined not to be in the detection frame.

In the implementation process, the data is shot through the computing terminal equipment, the certificate image data is obtained, whether the certificate image is in the detection frame or not is detected according to the proportion of the detection frame in the shooting frame and the proportion of the identity card in the screen in the shot data, and the user is prompted when the detection passes through the screen of the computing terminal equipment.

The implementation manner of the above steps is as follows: collecting a certificate image, obtaining coordinate information of the certificate image, such as [ (x10, y10), (x20, y20), (x30, y30), (x40, y40) ], and calculating the aspect ratio of the certificate image according to the coordinate information; and classifying the proportional characteristic data obtained by calculation to obtain proportional characteristic classification aspect ratio data. Specifically, the proportional characteristic data may be a slope, that is, the slope is classified, and the slope may be divided into 3 intervals, which are: 0 to 0.05, 0.05 to 0.1, 0.1 to 0.2; the method comprises the steps of respectively counting and calculating the aspect ratio ranges of the identity cards corresponding to the three slope intervals, namely counting and calculating the aspect ratio range of the slope in the interval of 0-0.05, counting and calculating the aspect ratio range of the slope in the interval of 0.05-0.1, counting and calculating the aspect ratio range of the slope in the interval of 0.1-0.2, and taking the three aspect ratio ranges as the reference ratios of the identity card integrity recognition deviation caused by the shooting angle. Detecting the document image according to the proportional features includes classifying the aspect ratio data according to the aspect ratio and the proportional features of the document image, and detecting the integrity of the identity card of the document image. Specifically, for example, the slope and the width-to-height ratio of the text box of the certificate are obtained, and if the width-to-height ratio of the certificate is smaller than the minimum value of the width-to-height ratio range corresponding to the slope of the text box, the certificate image is considered to be incomplete, and a user can be prompted to shoot a complete certificate image.

In the implementation process, the text box slopes are classified into different intervals, the width-to-height ratio data of the certificate image corresponding to the slopes of the intervals are calculated, the calculated width-to-height ratio data serve as the standard width-to-height ratio for identity card integrity recognition, the width-to-height ratio data are classified according to the width-to-height ratio and the ratio characteristics of the certificate image, and the integrity of the certificate image is detected.

Optionally, in this embodiment of the application, after detecting the positive feature of the certificate image according to the scale feature data, the method further includes: acquiring a large number of characteristic certificate images, and performing enhancement processing on the characteristic certificate images to obtain a certificate image training set; wherein the characteristic document image comprises a blurred, sharp, counterfeit and authentic document image; training a certificate image training set through a convolutional neural network to obtain a certificate fuzzy feature recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model; respectively and sequentially inputting the acquired certificate image into a certificate feature fuzzy recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model to obtain a third feature classification, a fourth feature classification and a fifth feature classification of the certificate image; the third feature classification represents a fuzzy recognition result of the acquired certificate image; the fourth characteristic classification represents the true and false identification result of the acquired certificate image; and the fifth characteristic classification represents the counterfeiting identification result of the acquired certificate image.

The implementation manner of the above steps is as follows: acquiring a large number of fuzzy characteristic certificate images and clear characteristic photos, performing enhancement processing on the characteristic certificate images to acquire a certificate image training set, specifically, collecting and simulating a large number of identity card images, performing scrambling data processing on the identity card images, taking the processed identity card images as a training set of a certificate fuzzy characteristic recognition model, inputting the fuzzy characteristic certificate images and the clear characteristic photos into a preset certificate fuzzy characteristic recognition model for training to improve the precision of the certificate fuzzy characteristic recognition model, then inputting the identity cards needing to be detected into the trained certificate fuzzy characteristic recognition model to acquire a third characteristic classification representation, and performing the third characteristic classification representation on the obtained fuzzy recognition results of the certificate images. The identity card to be detected can be each frame of the document image in the video stream containing the document image.

When the identity card is detected, the authenticity of the identity card needs to be detected, for example, some identity cards are not real identity cards but printed photos, and the like, so the authenticity of the identity card needs to be identified through a certificate authenticity feature identification model. Specifically, a large number of clear and real identity card images and unreal identity card images are collected, the obtained identity card images are subjected to enhancement processing, the processed identity card images are input into a preset certificate true and false feature recognition model as a sample set to be trained, the trained certificate true and false feature recognition model is obtained, the identity card to be detected is input into the trained certificate true and false feature recognition model, a fourth feature classification is obtained, and the fourth feature classification represents the true and false recognition results of the obtained identity card images.

Further, when the identity card is detected, a lawbreaker may tamper with important information such as the identity card number or name through the picture processing technology, and therefore the counterfeit of the identity card also needs to be detected during the identity card detection. Specifically, a large number of clear and real identity card images and fake identity card images are collected, the collected images are subjected to enhancement processing, the processed images are trained through a certificate fake feature recognition model, a trained certificate fake feature recognition model is obtained, and then fake identification of the certificate images is carried out through the trained certificate fake feature recognition model.

In the implementation process, a certificate fuzzy feature recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model are obtained by training the collected sample image; and the fuzzy, true and false identification is carried out on the certificate through the certificate fuzzy feature identification model, the certificate true and false feature identification model and the certificate false feature identification model. The method has the advantages that failure of certificate identification caused by network reasons is avoided, true and false of the identity card are detected, and a user is prompted in real time; the certificate can be identified again on the basis of identification card authenticity identification, comprehensive detection of the certificate is achieved, the pass rate of certificate detection is improved, and user experience is improved on the whole.

Optionally, in an embodiment of the present application, before labeling the plurality of texts of the document image according to the text information in the document image, the method further includes: acquiring a collected image comprising the certificate image and RGB data of the collected image; converting the RGB data into HSV data; and judging whether the shooting light of the acquired image is within a preset range or not according to the HSV data and a preset HSV data threshold.

The implementation manner of the above steps is as follows: the method comprises the steps of acquiring a collected image comprising a certificate image, acquiring RGB data of the certificate image, wherein the RGB data comprise RGB color gamut, but in the certificate identification process, the light of the certificate shooting environment is difficult to detect through the RGB color gamut, so that the certificate image is converted into HSV color gamut, the brightness information of a V space is used for identification, and whether the shooting light of the collected image is in a preset range or not is judged according to the identified brightness information and a preset HSV data threshold. The HSV data threshold may be a threshold of brightness information.

By the complete and comprehensive certificate detection method, the user experience in certificate detection is improved, the accuracy and the one-time pass rate of OCR (optical character recognition) are improved, and the problem that the user needs to repeatedly upload the images and experience poor due to low certificate detection performance when the user identifies the identity card is solved. Accurate prompt can be carried out on the user in real time when unqualified shooting is carried out, which link has a problem, namely, corresponding prompt can be carried out when any detection link fails to pass detection, and therefore the certificate detection efficiency and the user experience are greatly improved. It should be noted that the above certificate detection method can be used for identity card detection, and also can be used for flow-type identification or off-line identification of certificates such as bank cards and drivers licenses.

Please refer to fig. 7, which illustrates a schematic structural diagram of an apparatus provided in an embodiment of the present application; the embodiment of the present application provides a certificate detection apparatus 200, including:

the labeling module 210 is configured to label a plurality of texts of the certificate image according to the text information in the certificate image, so as to obtain labeling information;

the segmentation module 220 is configured to segment a text in the text according to the text information and the labeling information to obtain segmentation data, where the segmentation data includes a plurality of text boxes;

a calculating module 230, configured to calculate the ratio characteristics of the multiple text boxes according to the segmentation data, and obtain ratio characteristic data;

and the detection module 240 is used for detecting the positive alignment of the certificate image according to the proportion characteristic.

It should be understood that the apparatus corresponds to the above-mentioned embodiment of the certificate detection method, and can perform the steps related to the above-mentioned embodiment of the method, and the specific functions of the apparatus can be referred to the above description, and the detailed description is appropriately omitted here to avoid redundancy. The device includes at least one software functional module that can be stored in memory in the form of software or firmware (firmware) or solidified in the Operating System (OS) of the device.

Please refer to fig. 8, which illustrates a schematic structural diagram of an electronic device according to an embodiment of the present application. An electronic device 300 provided in an embodiment of the present application includes: a processor 310 and a memory 320, the memory 320 storing machine readable instructions executable by the processor 310, the machine readable instructions when executed by the processor 310 performing the method as above.

Embodiments of the present application further provide a storage medium, where a computer program is stored, and when the computer program is executed by a processor, the method is performed.

The storage medium may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

The above description is only an alternative implementation of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any person skilled in the art can easily change or replace the embodiments of the present application within the technical scope disclosed in the embodiments of the present application.

Claims

1. A method of credential detection, comprising:

marking a plurality of texts of the certificate image according to text information in the certificate image to obtain marking information;

according to the text information and the labeling information, segmenting a text in the text to obtain segmentation data, wherein the segmentation data comprises a plurality of text boxes;

calculating the proportional characteristics of the text boxes according to the segmentation data to obtain proportional characteristic data; and

and detecting the positive alignment of the certificate image according to the proportion characteristic data.

2. The method of claim 1, wherein the segmentation data further comprises the text box vertex coordinate information; the calculating the proportion characteristics of the plurality of text boxes according to the segmentation data to obtain proportion characteristic data comprises the following steps:

sorting the text boxes according to the vertex coordinate information of the text boxes to obtain sorting data; and

and calculating the proportional characteristics of the plurality of text boxes according to the sequencing data of the text boxes and the vertex coordinate information of the plurality of text boxes to obtain the proportional characteristic data, wherein the proportional characteristic data comprises slope data.

3. The method of claim 1, wherein prior to the labeling of the plurality of text of the document image from the textual information in the document image, the method further comprises:

obtaining coordinate information of the certificate image, and calculating the aspect ratio of the certificate image according to the coordinate information;

determining a detection frame parameter of a target object detection model according to the aspect ratio of the certificate image; and

identifying the certificate image through the target object detection model;

wherein the target object detection model comprises a Yolov5 network.

4. The method of claim 1, wherein prior to the labeling of the plurality of texts of the document image based on the text information in the document image, the method further comprises:

processing the collected image by using computer vision software to extract the certificate image;

adjusting the certificate image to a preset size to obtain a certificate image to be identified; and

inputting the certificate image to be identified into a preset identity card integrity detection model to obtain a first feature classification of the certificate image to be identified;

wherein, the identity card integrity detection model comprises: the identification card comprises a convolutional neural network, a plurality of convolutional layers, a pooling layer and a SoftMax function, wherein the first characteristic is used for classifying and representing whether the identification card of the image of the certificate to be identified is complete or not.

5. The method of claim 1, wherein prior to the labeling of the plurality of texts of the document image based on the text information in the document image, the method further comprises:

inputting the to-be-identified certificate image into a preset identity card shooting direction detection model to obtain a second feature classification of the to-be-identified certificate image;

wherein, the identity card shooting direction detection model comprises: the identification card comprises a convolutional neural network, a plurality of convolutional layers, a pooling layer, a full connection layer and a SoftMax function, wherein the second characteristic is used for classifying and representing the direction of the identification card image.

6. The method of claim 1, wherein prior to the labeling of the plurality of text of the document image from the textual information in the document image, the method further comprises:

acquiring the certificate image through a detection frame of the terminal equipment to obtain shooting data;

calculating the distance between the detection frame and the top and the left of the shooting frame according to the shooting data; the shooting data comprises the height of a shooting frame, the width of the shooting frame, the height of a detection frame and the width of the detection frame;

acquiring certificate image data according to the coordinate information of the certificate image; the certificate image data comprise certificate image height, certificate image width, collected image height and collected image width;

calculating the distance between the certificate image and the top and the left of the collected image according to the certificate image data; and

and detecting the certificate image according to the shooting data and the certificate image data.

7. The method of claim 6, wherein after the detecting the positive orientation of the document image based on the scale feature data, the method further comprises:

calculating the aspect ratio of the certificate image according to the coordinate information; classifying the proportional characteristic data to obtain proportional characteristic classification data; calculating the aspect ratio data corresponding to each proportion feature classification data to obtain proportion feature classification aspect ratio data; and

and classifying the aspect ratio data according to the aspect ratio and the proportion characteristic of the certificate image, and detecting the certificate image.

8. The method of claim 1, wherein after the detecting the positive orientation of the document image based on the scale feature data, the method further comprises:

acquiring a large number of characteristic certificate images, and performing enhancement processing on the characteristic certificate images to obtain a certificate image training set; wherein the characteristic document image comprises a blurred, sharp, counterfeit and authentic document image;

training the certificate image training set through a convolutional neural network to obtain a certificate fuzzy feature recognition model, a certificate true and false feature recognition model and a certificate counterfeiting feature recognition model; and

respectively and sequentially inputting the acquired certificate image into the certificate feature fuzzy recognition model, the certificate true and false feature recognition model and the certificate counterfeiting feature recognition model to obtain a third feature classification, a fourth feature classification and a fifth feature classification of the certificate image;

the third feature classification represents a fuzzy recognition result of the acquired certificate image; the fourth feature classification represents the true and false identification results of the acquired certificate image; and the fifth feature classification represents the counterfeiting identification result of the acquired certificate image.

9. A credential detection device, comprising:

the marking module is used for marking a plurality of texts of the certificate image according to the text information in the certificate image to obtain marking information;

the segmentation module is used for segmenting the text in the text according to the text information and the labeling information to obtain segmentation data, and the segmentation data comprises a plurality of text boxes;

the calculation module is used for calculating the proportional characteristics of the text boxes according to the segmentation data to obtain proportional characteristic data; and

and the detection module is used for detecting the positive alignment of the certificate image according to the proportion characteristic.

10. An electronic device, comprising: a processor and a memory, the memory storing machine-readable instructions executable by the processor, the machine-readable instructions, when executed by the processor, performing the method of any of claims 1 to 8.

11. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the method of any one of claims 1 to 8.