WO2021139169A1

WO2021139169A1 - Method and apparatus for card recognition, device, and storage medium

Info

Publication number: WO2021139169A1
Application number: PCT/CN2020/111203
Authority: WO
Inventors: 张国辉; 雷晨雨; 宋晨
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-07-27
Filing date: 2020-08-26
Publication date: 2021-07-15
Also published as: CN111860486A; CN111860486B

Abstract

A method and an apparatus for card recognition, a device, and a storage medium, relating to the field of financial technology. The card recognition method comprises: when it is detected that a card detection service is active, acquiring a plurality of image frames corresponding to a target card, each image frame in the plurality of image frames being an RGB format image frame (S101), then, on the basis of the plurality of image frames, determining card information of the target card (S102), the card information being used to map edges and key points of the target card, and finally, on the basis of the card information of the target card, determining whether the target card is a genuine card (S103). The invention helps to improve the efficiency and accuracy of card recognition.

Description

Card identification method, device, equipment and storage medium

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on July 27, 2020, the application number is 202010735901.0, and the invention title is "Card Recognition Method, Apparatus and Equipment", the entire content of which is incorporated into this application by reference .

Technical field

This application relates to the field of financial technology, in particular to a card identification method, device, equipment and storage medium.

Background technique

With the large-scale use of various cards such as ID cards, social security cards, and bank cards, related card identification services have also followed. For various card identification scenarios, card identification technology has become an indispensable part as card identification A very important part of the technology, the border detection algorithm of the card is getting more and more attention.

With traditional card recognition technology, users need to face complex, lengthy, and cumbersome operations. For example, users need to operate according to the operation prompt information layer by layer to recognize the card, which easily causes a poor experience for the user. It can be seen that, Traditional card recognition algorithms are not smart enough and require a lot of manual intervention, which seriously affects the user experience, causing traditional card recognition services to face more and more obvious challenges.

The inventor realized that the existing card border detection algorithms mainly use neural networks or traditional edge detection algorithms to find all the edge information in the picture, and then set various conditions to filter out some edge information to obtain the card border. Or when the edges are blurred, misjudgments are likely to occur, leading to frame detection errors and affecting the subsequent operation of other services such as card information extraction.

Summary of the invention

The embodiments of the present application provide a card identification method, device, equipment, and storage medium, which can improve the efficiency of card identification and restoration.

In the first aspect, an embodiment of the present application provides a card identification method, which includes:

When it is detected that the card detection service is started, acquiring a multi-frame image frame corresponding to the target card, each of the multi-frame image frames is an image frame in a red, green, and blue RGB format;

Determining the card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

Determine whether the target card is a real card according to the card information of the target card.

In a second aspect, an embodiment of the present application provides a card identification device, which includes:

The obtaining module is configured to obtain a multi-frame image frame corresponding to the target card when the card detection service is detected to be started, each of the multi-frame image frames is an image frame in red, green, and blue RGB format;

A determining module, configured to determine card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

The judging module is used for judging whether the target card is a real card according to the card information of the target card.

In the third aspect, an embodiment of the present application provides an electronic device, including:

Processor, suitable for implementing one or more instructions; and,

A computer-readable storage medium storing one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executing the following steps:

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program for electronic data exchange, where the computer program is used to implement the following steps when executed by a computer:

In the embodiment of the present application, when the card detection service is started, the multi-frame image frame corresponding to the target card is obtained. Each image frame in the multi-frame image frame is an image frame in red, green, and blue RGB format. Then, according to the multi-frame image The frame determines the card information of the target card. The card information is used to reflect the edge conditions and key points of the target card. Finally, according to the card information of the target card, it is judged whether the target card is a real card. That is, by adopting the card detection mechanism of detection and tracking, the accuracy of card recognition is improved. The process of card recognition does not require manual participation, which can improve the efficiency and accuracy of card recognition.

Description of the drawings

FIG. 1 is a schematic flowchart of another card identification method provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a decoder provided by an embodiment of the present application;

Figure 3 is a schematic structural diagram of another decoder provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a card identification device provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an electronic device provided by another embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Artificial intelligence technology is a comprehensive discipline, covering a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Among them, Computer Vision (CV) is a science that studies how to make machines "see". Furthermore, it refers to the use of cameras and computers instead of human eyes to identify, track, and measure targets. And further graphics processing, so that computer processing becomes more suitable for human eyes to observe or send to the instrument to detect the image. As a scientific discipline, computer vision studies related theories and technologies, trying to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and mapping Construction and other technologies also include common face recognition, fingerprint recognition and other biometric recognition technologies.

This application relates to image recognition technology in artificial intelligence. The image recognition technology is used to automatically convert images into card recognition without manual participation, which can improve the efficiency and accuracy of card recognition; this application can be applied to smart government affairs, smart education and other fields , Which is conducive to promoting the construction of smart cities.

Please refer to FIG. 1, which is a schematic flowchart of a card recognition method provided by an embodiment of the present application, which is executed by the electronic device of the embodiment of the present application. The card recognition method includes the following steps S101 to S103.

S101, when the electronic device detects that the card detection service is started, obtains a multi-frame image frame corresponding to the target card, where each image frame in the multi-frame image frame is an image frame in a red, green, and blue RGB format;

Wherein, the target card may be an identification card, such as an ID card, a pass, a driver's license, etc., and the target card may also be a welfare card, such as a social security card, a medical insurance card, a membership card, etc., the target card It may also be a financial card, such as a memory card, a credit card, etc., and the target card may also be another type of card, which is not specifically limited.

Wherein, the trigger condition for the activation of the card detection service may be that the card sensing area senses a card-like object, for example, the ID card sensing area senses that a card-like object is placed, and for example, a bank card insertion slot senses that a card-like object is inserted. , Etc., which are not specifically limited; the trigger condition for the activation of the card detection service can also be a user's triggering operation on a preset detection service activation button, where the service activation button can be a physical button or a virtual space. For example, the current card is detected as a handheld device, and the handheld device is provided with a physical button to start the card detection service. When the staff needs to determine the authenticity of the card to be detected, click the physical button of the handheld device to change the physical button to be detected. The card is placed in the sensing area of the handheld device. In a specific implementation, the sensing area of the handheld device may be the image acquisition range of the image acquisition device of the handheld device. When the image acquisition device is a camera, the image acquisition The range is within the lens range of the camera. When the image acquisition device is an infrared image sensor, the image acquisition range is within the infrared sensing range of the infrared image sensor; the trigger condition for starting the card detection service may also be Other trigger operations are not specifically limited.

Optionally, when the electronic device detects that the card detection service is started, the specific implementation manner for acquiring the multiple image frames corresponding to the target card may be: when the electronic device detects that the card detection service is started, recording the target The video corresponding to the card; the electronic device obtains multiple original image frames corresponding to the target card from the recorded video at a first preset time interval; the electronic device determines whether the multiple original image frames are RGB format image frames; if not, the electronic device converts the multiple original image frames into RGB format image frames. Wherein, the first preset time interval can be set according to user requirements and current device performance, and the value of the first time interval is not specifically limited.

Optionally, when the electronic device detects that the card detection service is started, the specific implementation manner of acquiring the multi-frame image frame corresponding to the target card may be: when the electronic device detects that the card detection service is started, the second preset Set a time interval to take pictures of the target card to obtain multiple original image frames; the electronic device determines whether the multiple original image frames are image frames in RGB format; if not, the electronic device The original image frame is converted into an image frame in RGB format. Wherein, the second preset time interval can be set according to user requirements and current device performance, and the value of the second time interval is not specifically limited.

S102: The electronic device determines card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

Optionally, the implementation manner for the electronic device to determine the card information of the target card according to the multiple image frames includes the following steps A11 to A15:

A11. The electronic device judges whether the currently processed image frame is the first image frame among the multiple image frames;

A12. If the currently processed image frame is the first image frame in the multi-frame image frame, the electronic device obtains the information of the target card in the currently processed image frame according to the first preset algorithm Card edge information, where the card edge information is used to reflect the edge condition of the target card;

Further, the electronic device acquiring the card edge information of the target card in the currently processed image frame according to the first preset algorithm includes the following steps B11 to B15:

B11. The electronic device obtains a first reference image frame according to the currently processed image frame, and the size of the first reference image frame is a first preset size;

Wherein, the first preset size may be a size whose ratio of length and width is equal to or similar to the true ratio of length and width of the target card. Different cards may have different sizes. For the first preset The size is not specifically limited.

Optionally, the implementation manner in which the electronic device obtains the first reference image frame whose size is the first preset size according to the currently processed image frame may be: the electronic device obtains the real size of the target card; The electronic device determines the first preset size corresponding to the actual size according to the actual size and the corresponding relationship between the size and the first preset size, and the corresponding relationship between the size and the first preset size is stored in advance In electronic equipment.

Optionally, the first preset size may also be a size preset in the electronic device by the user according to an application scenario, and the value of the first preset size is not specifically limited.

In a possible application scenario, as described, the current algorithm runs on a smart phone, and the smart phone is used to detect whether the bank card is a real card. Normally, the phone is rectangular, and the captured image frame is rectangular. The size of the length and width is close to two to one, that is, the above-mentioned multi-frame image frame can be obtained according to the ratio of length to width of two to one for unified processing. It should be noted that the larger the size of the image frame, the longer the calculation time. , The higher the accuracy, and in order to ensure that the time and accuracy meet the requirements at the same time, the first preset size can be set to 128*256. In this case, the image frame can be scaled to a pixel size of 128*256. Yes, the first preset size may also be other values, and the first preset size is not specifically limited.

B12. The electronic device obtains a second reference image frame according to the first reference image frame, and the pixel value of the first reference image frame is 255 times the pixel value of the second reference image frame;

Wherein, the obtaining the second reference image frame according to the first reference image frame refers to converting the first reference image frame into a binary image.

B13. The electronic device imports the second reference image frame into the target neural network model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame;

Wherein, the target neural network model includes a semantic segmentation model and a weighted least squares model, and the second reference image frame is imported into the target neural network model to obtain the initial image of the target card in the currently processed image frame The edge line parameters and four initial reference points include: importing the second reference image frame into the semantic segmentation model to obtain the feature map of the target card in the currently processed image frame; importing the feature map To the weighted least squares model, the initial edge line parameters and four initial reference points of the target card in the currently processed image frame are obtained.

Wherein, the target neural network model is the shufflenet_basic_128 model, which is an improved model based on the deeplab v3 model.

Wherein, the shufflenet_basic_128 model includes an encoder Encoder, a decoder Decoder, and a least squares module Weighted_least_squares consisting of three parts. Among them, the Encoder adopts the shufflenet_0.5 network, and the Decoder adopts the simplified structure of the deepnet v3 model. Specifically, the structure that the decoder can adopt can be as shown in Figure 2. The decoder includes the pooling layer Average Pool and the first 1x1 The convolution layer, the first activation function BN+RELU, the bilinear difference Resize Bilinear layer, and the fully connected Concat layer are connected in sequence, and the second 1x1Conv convolution layer, the second BN+RELU, and the Concat layer are connected in sequence. The structure that the decoder can adopt can also be as shown in FIG. 3, the first 1x1 convolutional layer, the second 1x1 convolutional layer, the fitting layer Dropout, the bilinear difference layer, and the parameter layer ArgMax.

Wherein, the electronic device obtains the initial edge line parameters of the target card and the four initial reference points in the currently processed image frame according to the second reference image frame. The network model is implemented in the shufflenet_basic_128 model.

In the following, taking the size of the first reference image frame of 128*256 as an example, the second reference image frame is imported into the target neural network model for the electronic device to obtain the target card in the currently processed image frame The initial edge line parameters and the realization process of the four initial reference points are introduced.

Import the first reference image frame with the size of 128*256 into the semantic segmentation model to obtain the semantic segmentation result of the first reference image frame, that is, the feature map, the size of which is 4*128*256; The 128*256 feature map is imported into the weighted least squares model. If, for each 128*256 feature map, set as x, the calculation is as follows: X_map is the feature map of the x-axis coordinate; y_map is the y-axis coordinate Feature map of, calculate the equation set W*[y_map,1]=A*W*x_map, get A=inv(T(WY)*WY)*(T(WY)*WX), where T(x) is The transpose of x, inv(x) is the inverse of x, and the size of A is 1*2. There are a total of 4 128*256 feature map calculations, and a total of 4*2 linear parameters can be obtained.

B14. The electronic device obtains the target edge line of the target card in the currently processed image frame according to the initial edge line parameter and the four initial reference points;

In a possible example, the obtaining the target edge line of the target card in the currently processed image frame according to the initial edge line parameter and the four initial reference points includes: the electronic device according to the The four initial reference points determine four edge regions of the target card in the currently processed image frame; the electronic device determines multiple target vertices corresponding to each edge region of the four edge regions; The electronic device determines the target edge line corresponding to each edge region according to the multiple target vertices corresponding to each edge region, and obtains the target edge line of the target card in the currently processed image frame.

Wherein, the size of the four edge regions may be the same or different, and the size of the four edge regions is not specifically limited.

The electronic device determining multiple target vertices corresponding to each of the four edge regions includes: the electronic device divides the currently processed region into n segments to obtain n subregions, where n is a positive value greater than or equal to 3. Integer; the electronic device obtains binarized images of the n sub-regions according to the n sub-regions; the electronic device determines n targets in the n sub-regions according to the binarized images of the n sub-regions A straight line, the target straight line is the straight line with the smallest area in the sub-region; the electronic device determines the vertices at both ends of each target straight line in the n target straight lines to obtain multiple target vertices corresponding to the currently processed region.

Among them, the user can set the value of n as needed, and the value of n is not specifically limited.

Wherein, the electronic device divides the currently processed area into n segments to obtain n sub-areas. The implementation manner of obtaining n sub-areas may be: the electronic device Sobel edge detection algorithm divides the currently processed area into n segments to obtain n sub-areas.

Wherein, the implementation manner for the electronic device to obtain the binarized image of the n sub-regions according to the n sub-regions may be: the electronic device adopts the Otsu method OTSU to perform adaptive binary image on the pictures of the n sub-regions To obtain the binarized image of the n sub-regions.

It can be understood that the multiple target vertices corresponding to each of the four edge regions will get 2n vertices.

The electronic device determines the target edge line corresponding to each edge region according to the multiple target vertices corresponding to each edge region, and obtains the implementation manner of the target edge line of the target card in the currently processed image frame It may be: a random sampling ransac algorithm is used for the two vertices corresponding to each edge region to fit the edge straight line corresponding to each edge region.

B15. The electronic device determines the vertex of the target card in the currently processed image frame corresponding to the target edge straight line.

A13. If the currently processed image frame is not the first image frame in the multi-frame image frame, the electronic device determines whether the currently processed image frame exists in the previous image frame. Target card

A14. If the target card exists in the previous image frame of the currently processed image frame, the electronic device acquires the card key of the target card in the currently processed image frame according to a second preset algorithm Point information, the card key point information is used to reflect the key point situation of the target card;

The acquiring card key point information of the target card in the currently processed image frame according to the second preset algorithm includes the following steps B21 to B24:

B21. The electronic device obtains the initialization image frame of the currently processed image frame according to the card information of the target card of the previous image frame of the currently processed image frame;

B22. The electronic device determines multiple key points in the initialization image frame;

The multiple key points may be multiple feature points in the card, and the number of the key points is not specifically limited.

B23. The electronic device obtains the original coordinates of the multiple key points;

B24. The electronic device obtains the target feature values of the multiple key points according to the original coordinates of the multiple key points.

Wherein, the electronic device obtaining the target feature value of the multiple key points according to the original coordinates of the multiple key points includes: the electronic device determines according to the mth abscissa corresponding to the currently processed key point The m-th direction gradient histogram hog feature corresponding to the m-th abscissa is used to obtain the m-th eigenvalue of the currently processed key point, and the m-th abscissa is the m-th characteristic value of the currently processed image frame. The abscissa of the key point corresponding to the currently processed key point in the subconvolved image frame; the electronic device determines the currently processed key point corresponding to the m-th eigenvalue according to the m-th eigenvalue The m-th abscissa change of the key point, the m-th abscissa change is the change value from the m-th abscissa to the m+1-th abscissa, and the m+1-th abscissa is the current The abscissa of the key point corresponding to the currently processed key point in the image frame obtained after the m-th convolution of the processed image frame; the electronic device according to the m-th abscissa and the m-th abscissa The amount of coordinate change obtains the m+1th abscissa.

Wherein, the corresponding relationship among the m-th abscissa change delta_x _m , the m+1- _{th abscissa x m+1} , the m- _{th abscissa x m} , and the m- _{th eigenvalue F m may be: delta_x} _m+1 =w*F _m +b, x _m+1 =x _m +delta_x _m+1 , where the parameters w and b are obtained by training in advance using the least square method.

Among them, the number of m is not specifically limited, that is, the number of convolutions of the image frame is not specifically limited. In specific implementation, m can be 4. When m is 4, the calculation speed and the accuracy of the calculation result meet the requirements of use.

A15. If the target card does not exist in the previous image frame of the currently processed image frame, acquire the card edge information of the target card in the currently processed image frame according to the first preset algorithm .

S103: The electronic device judges whether the target card is a real card according to the card information of the target card.

Optionally, an svm classifier may be used to determine whether the target card is a real card according to the card information of the target card.

Optionally, the electronic device in this application may refer to any node device in the blockchain. The so-called blockchain is a computer technology such as distributed data storage, peer-to-peer transmission (P2P transmission), consensus mechanism, encryption algorithm, etc. The new type of application model is essentially a decentralized database; the blockchain can be composed of multiple serial transaction records (also known as blocks) that are connected by cryptography and protect the content. The connected distributed ledger allows multiple parties to effectively record the transaction, and the transaction can be checked permanently (not tampered with). Among them, the consensus mechanism refers to the mathematical algorithm that realizes the establishment of trust between different nodes and the acquisition of rights and interests in the blockchain network; that is to say, the consensus mechanism is a mathematical algorithm recognized by all network nodes of the blockchain. This application can use the consensus mechanism of the blockchain to realize the restoration of the target image to the target card recognition, which can improve the accuracy of the restoration of the target card recognition.

For example, each node device in the blockchain performs consensus verification on the execution results of the above steps S101 to S103, and the execution results of each step are passed by the consensus verification, it can be determined that the accuracy of the generated target card recognition is relatively high; if there are steps If the execution result of is not passed by the consensus verification, it can be determined that the accuracy of the generated target card recognition is relatively low, and the node device may perform the above steps S101 to S103 again to obtain the target card recognition again. Alternatively, each node device in the blockchain can perform consensus verification on the target card identification (that is, only the execution result of step S103). If the consensus verification is passed, it is determined that the accuracy of the target card identification is relatively high; if the consensus verification fails , It is determined that the accuracy of the target card recognition is relatively low, and the node device can perform the above steps S101 to S103 again to obtain the target card recognition again.

Please refer to FIG. 4, which is a schematic structural diagram of a card recognition device provided by an embodiment of the present application. The card recognition device of the embodiment of the present application may be in the above-mentioned electronic equipment. In this embodiment, the card recognition device includes an acquisition module 401, a determination module 402, and a judgment module 403:

The obtaining module 401 is configured to obtain a multi-frame image frame corresponding to the target card when it is detected that the card detection service is started, and each image frame in the multi-frame image frame is an image frame in red, green, and blue RGB format;

The determining module 402 is configured to determine the card information of the target card according to the multi-frame image frame, and the card information is used to reflect the edge condition and key point condition of the target card;

The judging module 403 is used for judging whether the target card is a real card according to the card information of the target card.

Wherein, in terms of determining the card information of the target card according to the multi-frame image frame, the determining module 402 is specifically configured to: determine whether the currently processed image frame is the first frame of the multi-frame image frame Image frame; if the currently processed image frame is the first image frame in the multi-frame image frame, acquire the card edge information of the target card in the currently processed image frame according to the first preset algorithm , The card edge information is used to reflect the edge condition of the target card; if the currently processed image frame is not the first image frame in the multi-frame image frame, the currently processed image frame is determined Whether the target card exists in the previous image frame of the currently processed image frame; if the target card exists in the previous image frame of the currently processed image frame, the currently processed image frame is acquired according to the second preset algorithm The card key point information of the target card in the card key point information, the card key point information is used to reflect the key point situation of the target card; if the target card does not exist in the previous image frame of the currently processed image frame , The card edge information of the target card in the currently processed image frame is acquired according to the first preset algorithm.

Wherein, in terms of obtaining the card edge information of the target card in the currently processed image frame according to the first preset algorithm, the determining module 402 is specifically configured to: obtain the first Reference image frame, the size of the first reference image frame is a first preset size; a second reference image frame is obtained according to the first reference image frame, and the pixel value of the first reference image frame is the second 255 times the pixel value of the reference image frame; import the second reference image frame into the target neural network model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame; according to The initial edge line parameter and the four initial reference points obtain the target edge line of the target card in the currently processed image frame; determine the target edge line corresponding to the target edge line in the currently processed image frame State the apex of the target card.

Wherein, the target neural network model includes a semantic segmentation model and a weighted least squares model, and the second reference image frame is imported into the target neural network model to obtain the image of the target card in the currently processed image frame In terms of initial edge line parameters and four initial reference points, the determining module 402 is specifically configured to: import the second reference image frame into the semantic segmentation model to obtain the target card information in the currently processed image frame Feature map; import the feature map into the weighted least squares model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame.

Wherein, in terms of obtaining the target edge line of the target card in the currently processed image frame according to the initial edge line parameter and the four initial reference points, the determining module 402 is specifically configured to: The four initial reference points determine the four edge regions of the target card in the currently processed image frame; determine multiple target vertices corresponding to each edge region of the four edge regions; according to each edge The multiple target vertices corresponding to the region determine the target edge line corresponding to each edge region to obtain the target edge line of the target card in the currently processed image frame.

Wherein, in the aspect of determining multiple target vertices corresponding to each of the four edge regions, the determining module 402 is specifically configured to: divide the currently processed region into n segments to obtain n subregions, where n is A positive integer greater than or equal to 3; obtain binarized images of the n sub-regions according to the n sub-regions; determine n target straight lines in the n sub-regions according to the binarized images of the n sub-regions, so The target straight line is the straight line with the smallest area in the subregion; the two end vertices of each target straight line in the n target straight lines are determined to obtain multiple target vertices corresponding to the currently processed region.

Wherein, in the aspect of acquiring the card key point information of the target card in the currently processed image frame according to the second preset algorithm, the determining module 402 is specifically configured to: according to the front of the currently processed image frame Obtain the initial image frame of the currently processed image frame from the card information of the target card in one image frame; determine multiple key points in the initial image frame; obtain the original coordinates of the multiple key points; The original coordinates of the multiple key points obtain the target feature values of the multiple key points.

Wherein, in terms of obtaining the target feature values of the plurality of key points according to the original coordinates of the plurality of key points, the determining module 402 is specifically configured to: according to the mth horizontal line corresponding to the currently processed key point Coordinates, determine the m-th direction gradient histogram hog feature corresponding to the m-th abscissa, and obtain the m-th eigenvalue for the currently processed key point, and the m-th abscissa is the currently processed image frame The abscissa of the key point corresponding to the currently processed key point in the m-th convolutional image frame, m is a positive integer; the m-th eigenvalue is determined according to the m-th eigenvalue. The m-th abscissa change of the currently processed key point, the m-th abscissa change is the change value from the m-th abscissa to the m+1-th abscissa, the m+1-th abscissa Is the abscissa of the key point corresponding to the currently processed key point in the image frame obtained after the mth convolution of the currently processed image frame; according to the mth abscissa and the mth abscissa The amount of coordinate change obtains the m+1th abscissa.

Please refer to FIG. 5, which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 5, the electronic device in this embodiment may include: one or more processors 501; one or more input devices 502, one or more output devices 503 and storage 504. The aforementioned processor 501, input device 502, output device 503, and memory 504 are connected via a bus 505.

The processor 501 may be a central processing unit (Central Processing Unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits (ASICs). ), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The input device 502 can include a touch panel, a fingerprint sensor (used to collect user fingerprint information and fingerprint orientation information), a microphone, etc., the output device 503 can include a display (LCD, etc.), a speaker, etc., and the output device 503 can output calibration The processed data sheet.

The memory 504 may include a read-only memory and a random access memory, and provides instructions and data to the processor 501. A part of the memory 504 may also include a non-volatile random access memory. The memory 504 is used to store a computer program. The computer program includes program instructions. The processor 501 is used to execute the program instructions stored in the memory 504 to execute a program. A method of card recognition, which is used to perform the following operations:

The processor 501, input device 502, and output device 503 described in the embodiment of this application can perform the implementation described in the first embodiment of the card identification method provided in the embodiment of this application, and can also perform the implementation described in the embodiment of this application. The implementation method of the electronic device of, I will not repeat it here.

An embodiment of the present application also provides a computer-readable storage medium that stores a computer program for electronic data exchange, where the computer program implements the card recognition method shown in the embodiment of FIG. 1 when the computer program is executed by a computer.

The computer-readable storage medium may be an internal storage unit of the electronic device described in any of the foregoing embodiments, such as a hard disk or a memory of a control device. The computer-readable storage medium may also be an external storage device of the control device, such as a plug-in hard disk equipped on the control device, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) ) Card, Flash Card, etc. Further, the computer-readable storage medium may also include both an internal storage unit of the control device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the control device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output. Wherein, the computer-readable storage medium may be non-volatile or volatile.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A card identification method, which includes:

When it is detected that the card detection service is started, acquiring a multi-frame image frame corresponding to the target card, each of the multi-frame image frames is an image frame in a red, green, and blue RGB format;

Determining the card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

Determine whether the target card is a real card according to the card information of the target card.
The method according to claim 1, wherein the determining the card information of the target card according to the multi-frame image frame comprises:

Judging whether the currently processed image frame is the first image frame among the multiple image frames;

If the currently processed image frame is the first image frame in the multi-frame image frame, acquiring the card edge information of the target card in the currently processed image frame according to a first preset algorithm, and Card edge information is used to reflect the edge situation of the target card;

If the currently processed image frame is not the first image frame in the multi-frame image frame, determining whether the target card exists in the previous image frame of the currently processed image frame;

If the target card exists in the previous frame of the currently processed image frame, the currently processed image frame is obtained according to the card information of the target card in the previous frame of the currently processed image frame The initial image frame of the image frame;

Determining multiple key points in the initialization image frame;

Acquiring the original coordinates of the multiple key points;

Obtaining target feature values of the multiple key points according to the original coordinates of the multiple key points;

If the target card does not exist in the previous image frame of the currently processed image frame, acquiring the card edge information of the target card in the currently processed image frame according to the first preset algorithm.
The method according to claim 2, wherein said acquiring card edge information of said target card in said currently processed image frame according to a first preset algorithm comprises:

Obtaining a first reference image frame according to the currently processed image frame, and the size of the first reference image frame is a first preset size;

Obtaining a second reference image frame according to the first reference image frame, where the pixel value of the first reference image frame is 255 times the pixel value of the second reference image frame;

Importing the second reference image frame into the target neural network model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame;

Obtaining, according to the initial edge line parameter and the four initial reference points, the target edge line of the target card in the currently processed image frame;

Determine the vertex of the target card in the currently processed image frame corresponding to the target edge straight line.
The method according to claim 3, wherein the target neural network model includes a semantic segmentation model and a weighted least squares model, and the second reference image frame is imported into the target neural network model to obtain the currently processed The initial edge line parameters and four initial reference points of the target card in the image frame include:

Importing the second reference image frame into the semantic segmentation model to obtain a feature map of the target card in the currently processed image frame;

The feature map is imported into the weighted least squares model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame.
The method according to claim 3 or 4, wherein the obtaining the target edge line of the target card in the currently processed image frame according to the initial edge line parameter and the four initial reference points comprises:

Determining four edge regions of the target card in the currently processed image frame according to the four initial reference points;

Determining multiple target vertices corresponding to each of the four edge regions;

The target edge line corresponding to each edge area is determined according to the multiple target vertices corresponding to each edge area to obtain the target edge line of the target card in the currently processed image frame.
The method according to claim 5, wherein the determining multiple target vertices corresponding to each of the four edge regions comprises:

Divide the currently processed area into n segments to obtain n sub-areas, where n is a positive integer greater than or equal to 3;

Obtaining the binarized image of the n sub-regions according to the n sub-regions;

Determining n target straight lines in the n sub-regions according to the binarized images of the n sub-regions, where the target straight lines are the straight lines with the smallest area in the sub-regions;

The two end vertices of each target straight line in the n target straight lines are determined, and multiple target vertices corresponding to the currently processed region are obtained.
The method according to claim 2, wherein the obtaining the target feature value of the plurality of key points according to the original coordinates of the plurality of key points comprises:

According to the m-th abscissa corresponding to the currently processed key point, determine the m-th direction gradient histogram hog feature corresponding to the m-th abscissa to obtain the m-th eigenvalue for the currently processed key point, so The m-th abscissa is the abscissa of the key point corresponding to the currently processed key point in the m-th convolutional image frame of the currently processed image frame, and m is a positive integer;

Determine the m-th abscissa change amount of the currently processed key point corresponding to the m-th eigenvalue according to the m-th characteristic value, where the m-th abscissa change amount is the m-th abscissa to the m-th The change value between +1 abscissa, the m+1th abscissa is the key corresponding to the currently processed key point in the image frame obtained after the mth convolution of the currently processed image frame The abscissa of the point;

The m+1th abscissa is obtained according to the amount of change of the mth abscissa and the mth abscissa.
A card recognition device, which includes:

The first acquisition module is configured to acquire a multi-frame image frame corresponding to the target card when it is detected that the card detection service is started, and each of the multi-frame image frames is an image frame in a red, green, and blue RGB format;

The first determining module is configured to determine the card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

The judging module is used for judging whether the target card is a real card according to the card information of the target card.
An electronic device, wherein the electronic device includes a memory and a processor, the memory and the processor are connected to each other, the memory is used to store a computer program, and the computer program is configured to be used by the processor Execution, the computer program is configured to execute a card recognition method:

Wherein, the card identification method includes:

When it is detected that the card detection service is started, acquiring a multi-frame image frame corresponding to the target card, each of the multi-frame image frames is an image frame in a red, green, and blue RGB format;

Determining the card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

Determine whether the target card is a real card according to the card information of the target card.
9. The electronic device according to claim 9, wherein the determining the card information of the target card according to the multi-frame image frame comprises:

Judging whether the currently processed image frame is the first image frame among the multiple image frames;

If the currently processed image frame is the first image frame in the multi-frame image frame, acquiring the card edge information of the target card in the currently processed image frame according to a first preset algorithm, and Card edge information is used to reflect the edge situation of the target card;

If the currently processed image frame is not the first image frame in the multi-frame image frame, determining whether the target card exists in the previous image frame of the currently processed image frame;

If the target card exists in the previous frame of the currently processed image frame, the currently processed image frame is obtained according to the card information of the target card in the previous frame of the currently processed image frame The initial image frame of the image frame;

Determining multiple key points in the initialization image frame;

Acquiring the original coordinates of the multiple key points;

Obtaining target feature values of the multiple key points according to the original coordinates of the multiple key points;

If the target card does not exist in the previous image frame of the currently processed image frame, acquiring the card edge information of the target card in the currently processed image frame according to the first preset algorithm.
The electronic device according to claim 10, wherein said acquiring card edge information of said target card in said currently processed image frame according to a first preset algorithm comprises:

Obtaining a first reference image frame according to the currently processed image frame, and the size of the first reference image frame is a first preset size;

Obtaining a second reference image frame according to the first reference image frame, where the pixel value of the first reference image frame is 255 times the pixel value of the second reference image frame;

Importing the second reference image frame into the target neural network model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame;

Obtaining, according to the initial edge line parameter and the four initial reference points, the target edge line of the target card in the currently processed image frame;

Determine the vertex of the target card in the currently processed image frame corresponding to the target edge straight line.
The electronic device according to claim 11, wherein the target neural network model includes a semantic segmentation model and a weighted least squares model, and the second reference image frame is imported into the target neural network model to obtain the current processing The initial edge line parameters and four initial reference points of the target card in the image frame include:

Importing the second reference image frame into the semantic segmentation model to obtain a feature map of the target card in the currently processed image frame;

The feature map is imported into the weighted least squares model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame.
The electronic device according to claim 11 or 12, wherein the obtaining the target edge line of the target card in the currently processed image frame according to the initial edge line parameter and the four initial reference points comprises :

Determining four edge regions of the target card in the currently processed image frame according to the four initial reference points;

Determining multiple target vertices corresponding to each of the four edge regions;

The target edge line corresponding to each edge area is determined according to the multiple target vertices corresponding to each edge area to obtain the target edge line of the target card in the currently processed image frame.
The electronic device according to claim 13, wherein said determining a plurality of target vertices corresponding to each of the four edge regions comprises:

Divide the currently processed area into n segments to obtain n sub-areas, where n is a positive integer greater than or equal to 3;

Obtaining the binarized image of the n sub-regions according to the n sub-regions;

Determining n target straight lines in the n sub-regions according to the binarized images of the n sub-regions, where the target straight lines are the straight lines with the smallest area in the sub-regions;

The two end vertices of each target straight line in the n target straight lines are determined, and multiple target vertices corresponding to the currently processed region are obtained.
11. The electronic device according to claim 10, wherein the obtaining the target feature value of the plurality of key points according to the original coordinates of the plurality of key points comprises:

According to the m-th abscissa corresponding to the currently processed key point, determine the m-th direction gradient histogram hog feature corresponding to the m-th abscissa to obtain the m-th eigenvalue for the currently processed key point, so The m-th abscissa is the abscissa of the key point corresponding to the currently processed key point in the m-th convolutional image frame of the currently processed image frame, and m is a positive integer;

Determine the m-th abscissa change amount of the currently processed key point corresponding to the m-th eigenvalue according to the m-th characteristic value, where the m-th abscissa change amount is the m-th abscissa to the m-th The change value between +1 abscissa, the m+1th abscissa is the key corresponding to the currently processed key point in the image frame obtained after the mth convolution of the currently processed image frame The abscissa of the point;

The m+1th abscissa is obtained according to the amount of change of the mth abscissa and the mth abscissa.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that realizes card recognition when the computer program is executed by a processor, wherein the card recognition includes the following steps:

When it is detected that the card detection service is started, obtain a multi-frame image frame corresponding to the target card, and each of the multi-frame image frames is an image frame in a red, green, and blue RGB format;

Determining the card information of the target card according to the multi-frame image frame, where the card information is used to reflect the edge condition and key point condition of the target card;

Determine whether the target card is a real card according to the card information of the target card.
The computer-readable storage medium according to claim 16, wherein the determining the card information of the target card according to the multi-frame image frame comprises:

Judging whether the currently processed image frame is the first image frame among the multiple image frames;

If the currently processed image frame is the first image frame in the multi-frame image frame, acquiring the card edge information of the target card in the currently processed image frame according to a first preset algorithm, and Card edge information is used to reflect the edge situation of the target card;

If the currently processed image frame is not the first image frame in the multi-frame image frame, determining whether the target card exists in the previous image frame of the currently processed image frame;

If the target card exists in the previous frame of the currently processed image frame, the currently processed image frame is obtained according to the card information of the target card in the previous frame of the currently processed image frame The initial image frame of the image frame;

Determining multiple key points in the initialization image frame;

Acquiring the original coordinates of the multiple key points;

Obtaining target feature values of the multiple key points according to the original coordinates of the multiple key points;

If the target card does not exist in the previous image frame of the currently processed image frame, acquiring the card edge information of the target card in the currently processed image frame according to the first preset algorithm.
18. The computer-readable storage medium according to claim 17, wherein the acquiring card edge information of the target card in the currently processed image frame according to a first preset algorithm comprises:

Obtaining a first reference image frame according to the currently processed image frame, and the size of the first reference image frame is a first preset size;

Obtaining a second reference image frame according to the first reference image frame, where the pixel value of the first reference image frame is 255 times the pixel value of the second reference image frame;

Importing the second reference image frame into the target neural network model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame;

Obtaining, according to the initial edge line parameter and the four initial reference points, the target edge line of the target card in the currently processed image frame;

Determine the vertex of the target card in the currently processed image frame corresponding to the target edge straight line.
The computer-readable storage medium according to claim 18, wherein the target neural network model includes a semantic segmentation model and a weighted least squares model, and the second reference image frame is imported into the target neural network model to obtain the The initial edge line parameters and four initial reference points of the target card in the currently processed image frame include:

Importing the second reference image frame into the semantic segmentation model to obtain a feature map of the target card in the currently processed image frame;

The feature map is imported into the weighted least squares model to obtain the initial edge line parameters and four initial reference points of the target card in the currently processed image frame.
The computer-readable storage medium according to claim 18 or 19, wherein the target edge of the target card in the currently processed image frame is obtained according to the initial edge straight line parameter and the four initial reference points Straight line, including:

Determining four edge regions of the target card in the currently processed image frame according to the four initial reference points;

Determining multiple target vertices corresponding to each of the four edge regions;

The target edge line corresponding to each edge area is determined according to the multiple target vertices corresponding to each edge area to obtain the target edge line of the target card in the currently processed image frame.