CN112348008A - Certificate information identification method and device, terminal equipment and storage medium - Google Patents
Certificate information identification method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN112348008A CN112348008A CN202011228533.7A CN202011228533A CN112348008A CN 112348008 A CN112348008 A CN 112348008A CN 202011228533 A CN202011228533 A CN 202011228533A CN 112348008 A CN112348008 A CN 112348008A
- Authority
- CN
- China
- Prior art keywords
- certificate
- neural network
- information
- convolutional neural
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Character Discrimination (AREA)
Abstract
The application is applicable to the technical field of artificial intelligence, and provides a certificate information identification method, a certificate information identification device, terminal equipment and a storage medium, wherein the method comprises the following steps: extracting the characteristics of a target text area in the certificate image to obtain the character characteristics of the certificate image, wherein the character characteristics comprise a first characteristic and a second characteristic; carrying out information identification on the certificate image according to the first characteristics by utilizing a first lightweight convolutional neural network to obtain first information, and carrying out information identification on the certificate image according to the second characteristics by utilizing a second lightweight convolutional neural network to obtain second information, wherein the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network; and taking the first information and the second information as certificate information of the certificate image. And according to the difficulty degree of each character recognition task, selecting network structures with different complexity degrees to recognize character features so as to improve the recognition efficiency.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a certificate information identification method and apparatus, a terminal device, and a computer-readable storage medium.
Background
In order to ensure the normal and orderly social activities, users entering a specific place or area are subjected to user information collection and real-name registration, such as entering hotels, financial buildings, residential districts and the like. The current information registration mainly includes manual registration and information extraction and information entry by using Optical Character Recognition (OCR) technology.
In the related art, because the user information on the identity document of each person has great difference, and the identity document records the user information by adopting Chinese characters, and the structure of the Chinese characters is complex and the number of characters is large, the current OCR technology not only needs to consume a long time in the process of recognizing partial fields, but also has low recognition accuracy. It can be seen that the current OCR technology has a problem of low recognition efficiency.
Disclosure of Invention
In view of this, embodiments of the present application provide a method and an apparatus for identifying credential information, a terminal device, and a computer-readable storage medium, so as to solve the problem in the prior art that an OCR technology is low in identification efficiency.
A first aspect of an embodiment of the present application provides a method for identifying credential information, including:
extracting the characteristics of a target text area in the certificate image to obtain the character characteristics of the certificate image, wherein the character characteristics comprise a first characteristic and a second characteristic;
carrying out information identification on the certificate image according to the first characteristics by utilizing a first lightweight convolutional neural network which is constructed in advance to obtain first information, and carrying out information identification on the certificate image according to the second characteristics by utilizing a second lightweight convolutional neural network which is constructed in advance to obtain second information, wherein the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network;
and taking the first information and the second information as certificate information of the certificate image.
According to the identification method of the certificate information, the character characteristics of the certificate image are obtained by extracting the characteristics of the target text area in the certificate image, the character characteristics comprise a first characteristic and a second characteristic, for example, numbers and letters are small character sets, and Chinese characters are large character sets, so that the character characteristics can be extracted aiming at the character sets with different sizes, and the certificate information with different character types can be identified; because the characters of the small character set are fewer, the characters of the large character set are more, and the same network cannot give consideration to the recognition speed and the recognition accuracy of the second characteristic and the first characteristic, the recognition performance of the network cannot be optimized, the method utilizes the first lightweight convolutional neural network to carry out information recognition on the certificate image according to the first characteristic to obtain the first information, and utilizes the second lightweight convolutional neural network to carry out information recognition on the certificate image according to the second characteristic to obtain the second information, wherein the network layer number of the second lightweight convolutional neural network is larger than that of the first lightweight convolutional neural network, and the first information and the second information are used as the certificate image information, so that the difficulty degree of each character recognition task is fully considered, network structures with different complexity degrees are selected to recognize the character characteristics, the recognition accuracy can be ensured, and the recognition time consumption can be reduced, and further improve the recognition efficiency.
A second aspect of an embodiment of the present application provides an apparatus for identifying credential information, including:
the extraction module is used for extracting the characteristics of a target text area in the certificate image to obtain the character characteristics of the certificate image, wherein the character characteristics comprise a first characteristic and a second characteristic;
the identification module is used for carrying out information identification on the certificate image according to the first characteristics by utilizing a first lightweight convolutional neural network which is constructed in advance to obtain first information, and carrying out information identification on the certificate image according to the second characteristics by utilizing a second lightweight convolutional neural network which is constructed in advance to obtain second information, wherein the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network;
and the module is used for taking the first information and the second information as certificate information of the certificate image.
A third aspect of the embodiments of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the terminal device, where the processor implements the steps of the method for identifying credential information provided by the first aspect when executing the computer program.
A fourth aspect of embodiments of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements steps of a method for identifying credential information provided by the first aspect.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a flowchart of an implementation of a method for identifying credential information according to an embodiment of the present application;
fig. 2 is a flowchart illustrating a specific implementation of step S101 in a method for identifying credential information according to an embodiment of the present application;
fig. 3 is a flowchart of an implementation of a method for identifying credential information according to another embodiment of the present application;
fig. 4 is a block diagram of a structure of an identification apparatus for credential information according to an embodiment of the present application;
fig. 5 is a block diagram of a terminal device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
As described in the related art, because the user information on the identity document of each person has a large difference, and the identity document records the user information by using Chinese characters, which have a complex structure and many characters in similar shapes, the current OCR technology not only needs to consume a long time in the process of recognizing a part of fields, but also has low recognition accuracy. It can be seen that the current OCR technology has a problem of low recognition efficiency.
In view of this, in the identification method for certificate information provided in the embodiments of the present application, the character features of the certificate image are obtained by performing feature extraction on the target text region in the certificate image, where the character features include a first feature and a second feature, for example, numbers and letters are small character sets, and chinese characters are large character sets, so that the character features can be extracted for character sets of different sizes, and further, the certificate information of different character types can be identified; because the characters of the small character set are fewer, the characters of the large character set are more, and the same network cannot give consideration to the recognition speed and the recognition accuracy of the second characteristic and the first characteristic, the recognition performance of the network cannot be optimized, the method utilizes the first lightweight convolutional neural network to carry out information recognition on the certificate image according to the first characteristic to obtain the first information, and utilizes the second lightweight convolutional neural network to carry out information recognition on the certificate image according to the second characteristic to obtain the second information, wherein the network layer number of the second lightweight convolutional neural network is larger than that of the first lightweight convolutional neural network, and the first information and the second information are used as the certificate image information, so that the difficulty degree of each character recognition task is fully considered, network structures with different complexity degrees are selected to recognize the character characteristics, the recognition accuracy can be ensured, and the recognition time consumption can be reduced, and further improve the recognition efficiency.
Referring to fig. 1, fig. 1 is a flowchart illustrating an implementation of a method for identifying credential information according to an embodiment of the present application. The execution main body of the identification method of the certificate information provided by the embodiment of the application is terminal equipment, and the terminal equipment comprises but is not limited to terminals such as a smart phone, a tablet computer, a desktop computer, a super computer and a personal digital assistant. The identification method of the certificate information as shown in fig. 1 includes steps S101 to S103, which are described in detail below.
S101, extracting characteristics of a target text area in the certificate image to obtain character characteristics of the certificate image, wherein the character characteristics comprise a first characteristic and a second characteristic.
In this embodiment, the terminal device acquires the certificate image in advance. The certificate image is a two-dimensional image of the certificate, the certificate image can comprise a front image of the certificate and can also comprise a back image of the certificate, and the certificate comprises but is not limited to identity cards, military and official certificates, passports, drivers licenses, Hongkong and Macau passes, Taiwan hometown returns and the like which carry identity marks. It can be understood that the certificate image can be acquired by a camera shooting component or a scanning component on the terminal device, and can also be acquired by a camera shooting component or a scanning component on other terminal devices and then transplanted to the terminal device. The target text area is the area where the certificate information displayed on the certificate is located, the certificate information is the text displayed in the target text area, and the text can be Chinese characters, English letters, numbers and the like. Generally, certificate information displayed on a certificate includes a name, a gender, an address, a certificate number, a certificate validity period, and the like, and each certificate information is in a different row or a different column. Therefore, the certificate image can be divided into a plurality of target text areas in different rows or different columns according to different certificate information.
The character features are graphic features of each character in the target text area, such as outline features of the character, coordinate positions of feature points and the like. And according to the character type of the text in the target text region, dividing the character features in each target text region into second features or first features. The character types comprise Chinese characters, letters, numbers and the like, wherein the number of the numbers is only 10, and the number of the English letters is only 52, so that the numbers and the English letters are used as a small character set; the first-level Chinese characters commonly used in Chinese characters are 3755 Chinese characters, the Chinese characters are complex in structure and have more characters with similar shapes, and therefore the Chinese characters are used as a large character set. For example, the certificate is taken as an identity card, the number of the identity card and the validity period of the certificate on the identity card are both numbers or letters, and the position of each identity information on the identity card is fixed, so that the character features extracted from the target text area corresponding to the validity period of the identity card and the certificate are taken as the first features, and the character features extracted from the target text area containing Chinese characters, such as names, addresses and the like, are taken as the second features.
The terminal equipment extracts the characteristics of the target text area in the certificate image, and is a process of vectorizing the text image of the target text area. Exemplarily, the terminal device identifies a character outline in the target text region, determines an outline feature point of each character according to the character outline, and determines a coordinate position of each outline feature point based on a preset coordinate system; and vectorizing the contour feature points of each character contour according to the coordinate positions of the contour feature points to obtain character features. It will be appreciated that the above examples can be used to extract first and second features for numbers, letters and Chinese characters, where the outline feature points of Chinese characters are usually more than numbers and letters because the Chinese characters have complex structures and many similar characters, and the identification difficulty of second features is larger because the second features are more complex than the first features. Therefore, the present embodiment extracts character features for character sets of different sizes, and can further identify certificate information of different character types, so as to accurately identify different certificate information with pertinence.
S102, carrying out information identification on the certificate image according to the first characteristics by utilizing a first lightweight convolutional neural network which is constructed in advance to obtain first information, and carrying out information identification on the certificate image according to the second characteristics by utilizing a second lightweight convolutional neural network which is constructed in advance to obtain second information, wherein the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network.
In this embodiment, the terminal device stores a first lightweight convolutional neural network and a second lightweight convolutional neural network in advance. The first lightweight convolutional neural network and the second lightweight convolutional neural network are obtained by training the certificate image sample by using a machine learning algorithm. It can be understood that both the first lightweight convolutional neural network and the second lightweight convolutional neural network can be trained in advance by the terminal, or a file corresponding to the first lightweight convolutional neural network or the second lightweight convolutional neural network can be transplanted to the terminal after being trained in advance by other devices. That is, the execution subject for training the first or second lightweight convolutional neural network may be the same as or different from the execution subject for using the first or second lightweight convolutional neural network.
Because the characters of the small character set are fewer, the characters of the large character set are more, and the same network cannot give consideration to the recognition speed and the recognition accuracy of the second characteristic and the first characteristic, the recognition performance of the network cannot be optimized. Therefore, in order to reduce the time consumption for identifying the first features and improve the identification accuracy, the first features are identified by adopting a first lightweight convolutional neural network with light weight, and in order to improve the calculation performance and ensure that the network can fully extract the features of the image, the second lightweight convolutional neural network with more network layers than the first lightweight convolutional neural network is adopted to identify the large characters, namely the features.
The number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network, so that the second lightweight convolutional neural network has a wider field of view and has stronger extraction capability on complex features, and the second feature is identified by adopting the second lightweight convolutional neural network, so that character features can be extracted more fully, and the identification accuracy of the second feature is improved. And the network layer of the first lightweight convolutional neural network is less than that of the second lightweight convolutional neural network, so that the first characteristic is identified by adopting the first lightweight convolutional neural network, the calculation amount and the identification time consumption in the identification process can be reduced, and the identification efficiency of the first characteristic is improved.
In the embodiment, the first lightweight convolutional neural network suitable for the first feature recognition process and the second lightweight convolutional neural network suitable for the second feature are introduced into the certificate information recognition system, so that the difficulty degree of each character recognition task is fully considered, network structures with different complexity degrees are selected to recognize character features, the recognition precision can be ensured, the time consumed by recognition is reduced, and the recognition efficiency is improved. It can be understood that since both are lightweight networks, the mobile terminal can be further optimized subsequently on the basis of the lightweight networks, and the lightweight networks serve as the bottom-layer identification algorithm of the mobile terminal.
In one embodiment, the first lightweight convolutional neural network construction process includes: constructing a first convolution neural network with a first preset network layer number based on a MobileNet network and a CTC algorithm, wherein the first convolution neural network adopts a reverse residual error structure to perform channel expansion on convolution layers of the first convolution neural network; and training the first convolution neural network by using a preset first characteristic sample until the first convolution neural network reaches a first preset convergence condition, so as to obtain a first lightweight convolution neural network.
In this embodiment, the MobileNet network is a lightweight convolutional neural network that is dedicated to mobile terminals or embedded devices, which enables deep separable convolutions. The deep separable convolution separates two steps of the traditional convolution, namely depthwise and pointwise, and achieves the purpose of reducing the calculated amount in the convolution process by changing the channel number of a depthwise convolution layer or a pointwise convolution layer. Specifically, in this embodiment, a reversed residual structure is adopted to expand the number of channels of the depthwise convolutional layer, then the depthwise convolutional layer subjected to channel expansion is used to perform deep convolution operation on the first feature, and finally the pointwise convolutional layer is used to compress the channels, so that the number of channels before and after convolution is consistent. It will be appreciated that the inverted residual structure is a "channel expansion-convolution-channel compression" process. Illustratively, a reverse residual structure bneck in a MobileNet V2 network is adopted to perform channel expansion on a depthwise convolutional layer with the size of 1 × 1, then a depthwise convolutional layer with the size of 3 × 3 is used to perform deep convolution on a first feature to obtain a deep convolution result corresponding to the first feature, and finally a pointwise convolution with the size of 1 × 1 is used to compress the number of channels of the deep convolution result into an original size, so that the time consumption for identifying the first feature is reduced, and the identification accuracy of the first feature can be improved by channel expansion.
The ctc (connectionist temporal classification) algorithm is a loss function in the sequence tagging problem, which can be used to solve the classification problem of time series class data. That is to say, when the content in the original certificate picture sample is in a character string form, that is, when the original certificate picture sample contains a plurality of characters, the CTC algorithm can classify the character features of the character strings in the original certificate picture sample, divide the character features of different characters from the character features, and determine the corresponding character content when the maximum probability of each feature is determined according to each character feature.
The first feature sample is a sample image composed of numbers and letters, the preset convergence condition is a condition indicating that the network training is completed, for example, if a loss value obtained by a loss function is smaller than a preset loss threshold, convergence is indicated. Exemplarily, a sample image is input into a first convolutional neural network for processing, and a number and/or a letter corresponding to the sample image are obtained; calculating a loss value between the input sample image and the number and/or the letter, adjusting network parameters in the first convolution neural network when the loss value is larger than or equal to a preset loss threshold value, and returning to execute the step of inputting the sample image into the first convolution neural network for processing to obtain the number and/or the letter corresponding to the sample image; and when the loss value is smaller than a preset loss threshold value, the first convolutional neural network training is finished, and a trained first lightweight convolutional neural network is obtained. It can be understood in a colloquial way that a smaller loss value indicates that the neural network extracts more accurate feature vectors, so that the extracted feature vectors can be restored to the numbers and/or letters closest to the sample picture.
Optionally, the first convolutional neural network includes a shallow convolutional network and a deep convolutional network, the shallow convolutional network is a convolutional layer whose network layer position is before a preset network layer position in the first convolutional neural network, the shallow convolutional network uses a relu6 function as an activation function, the deep convolutional network is a convolutional layer whose network layer position is at or after the preset network layer position, and the deep convolutional network uses a Hard-swish function as the activation function.
In the embodiment, in order to reduce the computation delay, the shallow network adopts the ReLU6 as the activation function, and in order to reduce the consumption of the deep network on the computation resource and improve the overall accuracy of the network, the Hard-swish activation function is adopted in the deep network to replace the ReLU6 activation function as a new activation function. The Hard-swish activation function has the calculation formula of
It is understood that in the network training phase, the above MolibeNet network and CTC algorithm are used to train the id number recognition model, which may have an input size of 480 × 32, and the validity period recognition model, which may have an input size of 416 × 32.
In one embodiment, the second lightweight convolutional neural network construction process includes: constructing a second convolutional neural network with a second preset network layer number based on the MobileNet network and the CTC algorithm, wherein the second convolutional neural network adopts a residual error structure to perform channel compression on convolutional layers of the second convolutional neural network; and training the second convolutional neural network by using a preset second characteristic sample until the second convolutional neural network reaches a second preset convergence condition, so as to obtain a second lightweight convolutional neural network.
In this embodiment, the building process of the second lightweight convolutional neural network may refer to the building process of the first lightweight convolutional neural network, and is not described herein again. It should be noted that the second convolutional neural network performs channel compression on the convolutional layer of the second convolutional neural network by using a Residual structure, which is a structure opposite to the inverse Residual structure and may be Residual. The residual structure is to compress the number of channels of the depthwise convolutional layer, perform depth convolution operation on the second feature by using the depthwise convolutional layer after channel compression to obtain a depth convolution result corresponding to the second feature, and finally expand the channel of the depth convolution result by using the pointwise convolutional layer, so that the number of channels before and after convolution is consistent, that is, the reversed residual structure is a process of 'channel compression-convolution-channel expansion'. Since the second feature is more complex than the first feature, in order to reduce the amount of calculation in the recognition process, the second feature is channel-compressed to improve the recognition efficiency. It is understood that in the network training phase, the above-mentioned MolibeNet network and CTC algorithm is used to train a name (address) recognition model, a ethnic recognition model, wherein the input size of the name (address) may be 256 × 32, and the input size of the ethnic recognition model may be 160 × 32.
S103, the first information and the second information are used as certificate information of the certificate image.
In this embodiment, the first information and the second information obtained by recognition based on the character features of different sizes on the certificate image are fed back to the user as the certificate information of the certificate image.
Referring to fig. 2, fig. 2 is a flowchart illustrating an implementation of step S101 in a method for identifying credential information according to an embodiment of the present application. With respect to the embodiment corresponding to fig. 1, step S101 in the identification method of credential information provided in this embodiment specifically includes steps S201 to S202. The details are as follows:
s201, matching the certificate image with a preset certificate template image to determine a target text area in the certificate image, wherein the target text area comprises a first text area and a second text area.
In this embodiment, since most certificates are rigid bodies and the positions of keywords (such as blue name, gender, and other Chinese characters) in each field are fixed, the positions of the characters to be identified (such as specific identification number regions) can be found according to the position information of the keywords. Optionally, matching the certificate image with a preset certificate template image by using a template matching algorithm (such as matchTemplate function based on opencv) to determine a target text area of the name, the gender, the national address and the identification number in the certificate image.
S202, extracting the features of the first text area and the second text area to obtain a first feature and a second feature.
In this embodiment, the character image detected from the certificate image is converted into a standard code for characters by the computer, which is a key for the computer to "recognize the character". In the embodiment, for the text image in the image, the outline of the text and the coordinate position of each feature point are identified, and the continuity features are generated through vectorization, so that the features of strokes, feature points, projection information, point region distribution and the like of the text are extracted.
In one embodiment, feature extraction is performed on the first text region and the second text region to obtain a first feature and a second feature, and the feature extraction includes: respectively carrying out character outline recognition on the first text area and the second text area to respectively obtain character outlines of the first text area and the second text area; extracting contour feature points of each character contour to obtain the coordinate position of each contour feature point; and vectorizing the contour feature points of each character contour according to the coordinate positions of the contour feature points to obtain a first feature and a second feature. Therefore, the present embodiment extracts character features for character sets of different sizes, and can further identify certificate information of different character types, so as to accurately identify different certificate information with pertinence.
Referring to fig. 3, fig. 3 is a flowchart illustrating an implementation of a method for identifying credential information according to another embodiment of the present application. With respect to the embodiment shown in fig. 1, the identification method of credential information provided in this embodiment further includes step S301 to step S303 before step S101. The details are as follows:
s301, acquiring an original certificate image, and detecting the position of the target certificate in the original certificate image based on a local feature point detection algorithm.
In this embodiment, since it is impossible to ensure that the image capturing direction of the image capturing device is completely perpendicular to the document when capturing the document image, the captured image inevitably has an inclination, and therefore, a character recognition software is required to correct the inclination. Illustratively, feature point matching is carried out by utilizing an algorithm based on SIFT features, and an SIFT feature point detection algorithm is an algorithm for detecting local features, and is characterized in that the positions of certificates in an input shot picture are found by obtaining feature points in a picture and descriptors of the feature points and related dimensions and directions to obtain features and carrying out image feature point matching.
S302, intercepting the target certificate in the original certificate image according to the position of the target certificate in the original certificate image to obtain the target certificate image.
In the embodiment, the certificate image in the shot picture is intercepted, so that other unnecessary image factors are removed, and the certificate information can be rapidly identified.
And S303, carrying out affine transformation on the target certificate image according to the preset certificate size to obtain the certificate image conforming to the preset certificate size.
In this embodiment, the affine transformation principle is used to correct the target document image, so as to obtain a document image with a uniform shape and size. It is understood that affine transformation processes include, but are not limited to, scaling, rotation, mirroring, zooming, and the like. Further, the preprocessing for the certificate image can also include binarization, noise removal and the like. Specifically, most of pictures shot by the camera are color images, the information content of the color images is huge, the contents of the pictures can be simply divided into foreground and background, in order to enable a computer to recognize characters more quickly and better, the color images need to be processed first, so that only foreground information and background information of the pictures are processed, the foreground information can be simply defined to be black, and the background information can be simply defined to be white, so that a binary image is obtained. Since there may be other background information around the original image, the noise (interference) apparent in the original image is removed.
Referring to fig. 4, fig. 4 is a block diagram of a device for identifying credential information according to an embodiment of the present application. The device in this embodiment includes modules for performing the steps in the embodiments corresponding to fig. 1 to 3. Please refer to fig. 1 to 3 and fig. 1 to 3 for the corresponding embodiments. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 4, the identification apparatus of certificate information includes:
the extraction module 401 is configured to perform feature extraction on a target text region in a certificate image to obtain character features of the certificate image, where the character features include a first feature and a second feature;
the identification module 402 is configured to perform information identification on the certificate image according to the first feature by using a first lightweight convolutional neural network to obtain first information, and perform information identification on the certificate image according to the second feature by using a second lightweight convolutional neural network to obtain second information, where the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network;
and a module 403 for using the first information and the second information as certificate information of the certificate image.
According to the identification device for the certificate information, the extraction module 401 is used for extracting the features of the target text area in the certificate image to obtain the character features of the certificate image, wherein the character features comprise a first feature and a second feature, for example, numbers and letters are small character sets, and Chinese characters are large character sets, so that the character features can be extracted for the character sets with different sizes, and the certificate information with different character types can be identified; because the characters of the small character set are fewer, the characters of the large character set are more, and the same network cannot give consideration to the recognition speed and the recognition accuracy of the second characteristic and the first characteristic, the recognition performance of the network cannot be optimized, the identification module 402 performs information recognition on the certificate image according to the first characteristic by using the first lightweight convolutional neural network to obtain the first information, performs information recognition on the certificate image according to the second characteristic by using the second lightweight convolutional neural network to obtain the second information, wherein the network layer number of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network, and the first information and the second information are used as the certificate information of the certificate image by using the module 403, so that the difficulty degree of each character recognition task is fully considered, network structures with different complexity degrees are selected to recognize the character characteristics, so as to ensure the recognition precision and reduce the time consumption of recognition, and further improve the recognition efficiency.
As an embodiment of the present application, the extraction module 401 is further configured to:
matching the certificate image with a preset certificate template image to determine a target text area in the certificate image, wherein the target text area comprises a first text area and a second text area;
and performing feature extraction on the first text region and the second text region to obtain a first feature and a second feature.
As an embodiment of the present application, the extraction module 401 is further configured to:
respectively carrying out character outline recognition on the first text area and the second text area to respectively obtain character outlines of the first text area and the second text area;
extracting contour feature points of each character contour to obtain the coordinate position of each contour feature point;
and vectorizing the contour feature points of each character contour according to the coordinate positions of the contour feature points to obtain a first feature and a second feature.
As an embodiment of the present application, the identification apparatus for certificate information further includes:
the acquisition module is used for acquiring an original certificate image and detecting the position of a target certificate in the original certificate image based on a local feature point detection algorithm;
the intercepting module is used for intercepting the target certificate in the original certificate image according to the position of the target certificate in the original certificate image to obtain a target certificate image;
and the transformation module is used for carrying out affine transformation on the target certificate image according to the preset certificate size to obtain the certificate image conforming to the preset certificate size.
As an embodiment of the present application, the identification apparatus for certificate information further includes a first building module, configured to:
constructing a first convolution neural network with a first preset network layer number based on a MobileNet network and a CTC algorithm, wherein the first convolution neural network adopts a reverse residual error structure to perform channel expansion on convolution layers of the first convolution neural network;
and training the first convolution neural network by using a preset first characteristic sample until the first convolution neural network reaches a first preset convergence condition, so as to obtain a first lightweight convolution neural network.
As an embodiment of the present application, the first convolutional neural network includes a shallow convolutional network and a deep convolutional network, the shallow convolutional network is a convolutional layer whose network layer position is before a preset network layer position in the first convolutional neural network, the shallow convolutional network uses a relu6 function as an activation function, the deep convolutional network is a convolutional layer whose network layer position is at or after the preset network layer position, and the deep convolutional network uses a Hard-swish function as the activation function.
As an embodiment of the present application, the identification apparatus for certificate information further includes a second construction module, configured to:
constructing a second convolutional neural network with a second preset network layer number based on the MobileNet network and the CTC algorithm, wherein the second convolutional neural network adopts a residual error structure to perform channel compression on convolutional layers of the second convolutional neural network;
and training the second convolutional neural network by using a preset second characteristic sample until the second convolutional neural network reaches a second preset convergence condition, so as to obtain a second lightweight convolutional neural network.
It should be understood that, in the structural block diagram of the identification apparatus for credential information shown in fig. 4, each module is used to execute each step in the embodiment corresponding to fig. 1 to 3, and each step in the embodiment corresponding to fig. 1 to 3 has been explained in detail in the above embodiment, and specific reference is made to the relevant description in the embodiment corresponding to fig. 1 to 3 and fig. 1 to 3, which is not repeated herein.
Fig. 5 is a block diagram of a terminal device according to an embodiment of the present application. As shown in fig. 5, the terminal device 50 of this embodiment includes: a processor 51, a memory 52 and a computer program 53 stored in the memory 52 and executable on the processor 51, such as a program for a method of identification of credential information. The processor 51 implements the steps in each embodiment of the identification method of the respective credential information described above, such as S101 to S103 shown in fig. 1, or S201 to S202 and S301 to S303 shown in fig. 2 and 3, when executing the computer program 53. Alternatively, when the processor 51 executes the computer program 53, the functions of the modules in the embodiment corresponding to fig. 4, for example, the functions of the modules 401 to 403 shown in fig. 4, are implemented, for which reference is specifically made to the relevant description in the embodiment corresponding to fig. 4, which is not repeated herein.
Illustratively, the computer program 53 may be divided into one or more individual modules, one or more of which are stored in the memory 52 and executed by the processor 51 to perform the steps of identifying credential information of the present application. One or more of the elements may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 53 in the terminal device 50. For example, the computer program 53 may be divided into an extraction module, an identification module, and as modules, each module having the above-described specific functions.
Those skilled in the art will appreciate that fig. 5 is merely an example of a terminal device 50 and does not constitute a limitation of terminal device 50 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., terminal device 50 may also include input-output devices, network access devices, buses, etc.
The Processor 51 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 52 may be an internal storage unit of the terminal device 50, such as a hard disk or a memory of the terminal device 50. The memory 52 may also be an external storage device of the terminal device 50, such as a plug-in hard disk provided on the terminal device 50, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. Further, the memory 52 may also include both an internal storage unit of the terminal device 50 and an external storage device. The memory 52 is used for storing computer programs and other programs and data required by the terminal device. The memory 52 may also be used to temporarily store data that has been output or is to be output.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method for identifying certificate information is characterized by comprising the following steps:
extracting features of a target text area in the certificate image to obtain character features of the certificate image, wherein the character features comprise a first feature and a second feature;
carrying out information identification on the certificate image according to the first characteristics by utilizing a first lightweight convolutional neural network which is constructed in advance to obtain first information, and carrying out information identification on the certificate image according to the second characteristics by utilizing a second lightweight convolutional neural network which is constructed in advance to obtain second information, wherein the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network;
and taking the first information and the second information as certificate information of the certificate image.
2. The method for identifying the certificate information as claimed in claim 1, wherein the extracting the features of the target text region in the certificate image to obtain the character features of the certificate image comprises:
matching the certificate image with a preset certificate template image to determine a target text area in the certificate image, wherein the target text area comprises a first text area and a second text area;
and performing feature extraction on the first text region and the second text region to obtain a first feature of the first text region and a second feature of the second text region.
3. The method for identifying certificate information as claimed in claim 2, wherein the extracting features of the first text region and the second text region to obtain the first feature and the second feature comprises:
respectively carrying out character outline recognition on the first text area and the second text area to respectively obtain character outlines of the first text area and the second text area;
extracting contour feature points of each character contour to obtain the coordinate position of each contour feature point;
and vectorizing the contour feature points of each character contour according to the coordinate positions of the contour feature points to obtain the first feature and the second feature.
4. The method for identifying certificate information as claimed in claim 1, wherein before extracting the features of the text region in the certificate image to obtain the character features of the certificate image, the method further comprises:
acquiring an original certificate image, and detecting the position of a target certificate in the original certificate image based on a local feature point detection algorithm;
intercepting the target certificate in the original certificate image according to the position of the target certificate in the original certificate image to obtain a target certificate image;
and carrying out affine transformation on the target certificate image according to the preset certificate size to obtain the certificate image conforming to the preset certificate size.
5. The method for identifying the certificate information as claimed in claim 1, wherein the pre-construction process of the first lightweight convolutional neural network comprises:
constructing a first convolutional neural network with a first preset network layer number based on a MobileNet network and a CTC algorithm, wherein the first convolutional neural network adopts a reverse residual error structure to perform channel expansion on convolutional layers of the first convolutional neural network;
and training the first convolution neural network by using a preset first characteristic sample until the first convolution neural network reaches a first preset convergence condition, so as to obtain the constructed first lightweight convolution neural network.
6. The method of claim 5, wherein the first convolutional neural network comprises a shallow convolutional network and a deep convolutional network, the shallow convolutional network is a convolutional layer with a network layer position before a preset network layer position in the first convolutional neural network, the shallow convolutional network adopts a relu6 function as an activation function, the deep convolutional network is a convolutional layer with a network layer position after or at the preset network layer position, and the deep convolutional network adopts a Hard-swish function as an activation function.
7. The method for identifying the certificate information as claimed in claim 1, wherein the pre-construction process of the second lightweight convolutional neural network comprises:
constructing a second convolutional neural network with a second preset network layer number based on a MobileNet network and a CTC algorithm, wherein the second convolutional neural network adopts a residual error structure to perform channel compression on convolutional layers of the second convolutional neural network;
and training the second convolutional neural network by using a preset second characteristic sample until the second convolutional neural network reaches a second preset convergence condition, so as to obtain the constructed second lightweight convolutional neural network.
8. An apparatus for recognizing certificate information, comprising:
the extraction module is used for extracting the characteristics of a target text area in the certificate image to obtain the character characteristics of the certificate image, wherein the character characteristics comprise a first characteristic and a second characteristic;
the identification module is used for carrying out information identification on the certificate image according to the first characteristics by utilizing a first lightweight convolutional neural network which is constructed in advance to obtain first information, and carrying out information identification on the certificate image according to the second characteristics by utilizing a second lightweight convolutional neural network which is constructed in advance to obtain second information, wherein the number of network layers of the second lightweight convolutional neural network is greater than that of the first lightweight convolutional neural network;
and the module is used for taking the first information and the second information as certificate information of the certificate image.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011228533.7A CN112348008A (en) | 2020-11-06 | 2020-11-06 | Certificate information identification method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011228533.7A CN112348008A (en) | 2020-11-06 | 2020-11-06 | Certificate information identification method and device, terminal equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112348008A true CN112348008A (en) | 2021-02-09 |
Family
ID=74428860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011228533.7A Pending CN112348008A (en) | 2020-11-06 | 2020-11-06 | Certificate information identification method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348008A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990133A (en) * | 2021-04-28 | 2021-06-18 | 杭州金线连科技有限公司 | Multitask-based deep convolutional neural network identity card information identification method |
| CN113221908A (en) * | 2021-06-04 | 2021-08-06 | 深圳龙岗智能视听研究院 | Digital identification method and equipment based on deep convolutional neural network |
-
2020
- 2020-11-06 CN CN202011228533.7A patent/CN112348008A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990133A (en) * | 2021-04-28 | 2021-06-18 | 杭州金线连科技有限公司 | Multitask-based deep convolutional neural network identity card information identification method |
| CN112990133B (en) * | 2021-04-28 | 2021-08-27 | 杭州金线连科技有限公司 | Multitask-based deep convolutional neural network identity card information identification method |
| CN113221908A (en) * | 2021-06-04 | 2021-08-06 | 深圳龙岗智能视听研究院 | Digital identification method and equipment based on deep convolutional neural network |
| CN113221908B (en) * | 2021-06-04 | 2024-04-16 | 深圳龙岗智能视听研究院 | Digital identification method and device based on deep convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11830230B2 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| CN109993160B (en) | Image correction and text and position identification method and system | |
| RU2668717C1 (en) | Generation of marking of document images for training sample | |
| US8792722B2 (en) | Hand gesture detection | |
| US8750573B2 (en) | Hand gesture detection | |
| JP2019536120A (en) | System and method for verifying authenticity of ID photo | |
| CN111242124B (en) | Certificate classification method, device and equipment | |
| CN112966725B (en) | Method and device for matching template images and terminal equipment | |
| CN113704531A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN110852311A (en) | Three-dimensional human hand key point positioning method and device | |
| WO2021218706A1 (en) | Text identification method and apparatus, device, and storage medium | |
| CN113111880B (en) | Certificate image correction method, device, electronic equipment and storage medium | |
| CN112348008A (en) | Certificate information identification method and device, terminal equipment and storage medium | |
| CN110866469A (en) | Human face facial features recognition method, device, equipment and medium | |
| Sharma et al. | Lean histogram of oriented gradients features for effective eye detection | |
| CN116189162A (en) | Ship plate detection and identification method and device, electronic equipment and storage medium | |
| CN113228105A (en) | Image processing method and device and electronic equipment | |
| Wicht et al. | Mixed handwritten and printed digit recognition in Sudoku with Convolutional Deep Belief Network | |
| Fawwad Hussain et al. | Gray level face recognition using spatial features | |
| CN112597940A (en) | Certificate image recognition method and device and storage medium | |
| WO2020244076A1 (en) | Face recognition method and apparatus, and electronic device and storage medium | |
| JP6365117B2 (en) | Information processing apparatus, image determination method, and program | |
| CN108304838B (en) | Picture information identification method and terminal | |
| CN113065480B (en) | Handwriting style identification method and device, electronic device and storage medium | |
| CN114445916A (en) | Living body detection method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |