WO2021151319A1

WO2021151319A1 - Card edge detection method, apparatus, and device, and readable storage medium

Info

Publication number: WO2021151319A1
Application number: PCT/CN2020/122132
Authority: WO
Inventors: 张国辉; 雷晨雨; 宋晨
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-07-30
Filing date: 2020-10-20
Publication date: 2021-08-05
Also published as: CN111899270A; CN111899270B

Abstract

The present application relates to artificial intelligence and provides a card edge detection method, apparatus, and device, and a readable storage medium. The method comprises: obtaining an original card image and performing pre-processing; by means of a detection model, obtaining edge straight line parameter of the card; on the basis of the edge straight line parameter, performing post-processing of direct segmented detection and fitting, so as to obtain the card edges in the image. In addition, the present application also relates to the blockchain technology, as the original card image and/or card edges can be stored in a blockchain. The present application utilizes image boundary detection to detect and obtain the edges of a card, reducing mis-determination in situations with complex backgrounds or vague boundaries, enhancing the accuracy of card edge detection, and facilitating the follow-up processing such as card information extraction.

Description

Card frame detection method, device, equipment and readable storage medium

This application is required to be submitted to the Chinese Patent Office on July 30, 2020, the application number is CN 202010756730.X, the priority of the Chinese patent application titled "Card Frame Detection Method, Device, Equipment, and Readable Storage Medium", the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the field of image processing, and in particular to a method, device, equipment and readable storage medium for detecting a card frame.

Background technique

With the massive use of various cards such as ID cards, social security cards, and bank cards, related card identification services will follow. Among them, the border (edge) detection algorithm of various cards is a very important part of the card recognition algorithm. The existing card border detection algorithm mainly sets various conditions to filter out some edge information to obtain the card border.

technical problem

The inventor realizes that this method is prone to misjudgment in the case of a complex background or blurred edges, resulting in frame detection errors and affecting the subsequent operation of other services such as card information extraction.

Technical solutions

An embodiment of the present application provides a method for detecting a card frame, and the method for detecting a card frame includes:

Acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

Inputting the image to be recognized into a preset detection model, extracting corresponding image feature information from the preset detection model, and calculating frame line parameters according to the image feature information;

Acquiring the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge regions of each side of the card in the original card image according to the detection frame line;

Cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area;

Obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

Combine the borders corresponding to each side to fit a straight line to obtain the card border.

The embodiment of the present application also provides a card frame detection device, the card frame detection device includes:

An image acquisition module for acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

A model input module, configured to input the image to be recognized into a preset detection model, extract corresponding image feature information from the preset detection model, and calculate frame line parameters according to the image feature information;

An area interception module, configured to obtain the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge area of each side of the card in the original card image according to the detection frame line;

The straight line detection module is used to cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area;

The straight line fitting module is used to obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and to fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

The straight line combination module is used to combine the borders corresponding to each side to fit a straight line to obtain the card border.

An embodiment of the present application also provides a card frame detection device. The card frame detection device includes a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program is When the processor executes, the following steps are implemented:

The embodiment of the present application also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

Description of the drawings

FIG. 1 is a schematic diagram of the hardware structure of the card frame detection device involved in the solution of the embodiment of the application;

2 is a schematic flowchart of a first embodiment of a method for detecting a card frame according to this application;

FIG. 3 is a schematic diagram of the functional modules of the first embodiment of the card frame detection device according to the present application.

The following specific embodiments will further illustrate this application in conjunction with the above-mentioned drawings.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

The card frame detection method involved in the embodiments of this application is mainly applied to a card frame detection device, which can be a mobile phone, a tablet computer, a personal computer (PC), a notebook computer, a server, and other devices with data processing functions. .

Referring to FIG. 1, FIG. 1 is a schematic diagram of the hardware structure of the card frame detection device involved in the solution of the embodiment of the application. In the embodiment of the present application, the card frame detection device may include a processor 1001 (for example, a central processing unit, a CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (Such as wireless fidelity WIreless-FIdelity, WI-FI interface); memory 1005 can be high-speed random access memory (random access memory, RAM), can also be stable memory (non-volatile memory), such as disk memory, memory Optionally, 1005 may also be a storage device independent of the aforementioned processor 1001. The memory 1005 may be volatile or non-volatile. Those skilled in the art can understand that the hardware structure shown in FIG. 1 does not constitute a limitation to the present application, and may include more or less components than those shown in the figure, or a combination of certain components, or different component arrangements.

Continuing to refer to FIG. 1, the memory 1005 as a computer-readable storage medium in FIG. 1 may include an operating system, a network communication module, and a computer program. In FIG. 1, the network communication module can be used to connect to a preset database and perform data communication with the database; and the processor 1001 can call a computer program stored in the memory 1005 and execute the card frame detection method provided in the embodiment of the present application.

Based on the foregoing hardware architecture, various embodiments of the card frame detection method of the present application are proposed.

The embodiment of the present application provides a method for detecting a card frame.

Referring to FIG. 2, FIG. 2 is a schematic flowchart of a first embodiment of a method for detecting a card frame according to the present application.

In this embodiment, the card frame detection method includes the following steps:

Step S10, acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

With the massive use of various cards such as ID cards, social security cards, and bank cards, related card identification services will follow. Among them, the border (edge) detection algorithm of various cards is a very important part of the card recognition algorithm. The existing card border detection algorithm is mainly to set various conditions to filter out some edge information to obtain the card border. This method is prone to misjudgment in the case of a complex background or blurred edges, leading to error detection of the border and affecting subsequent evaluation. The operation of other services such as the extraction of card information. In this regard, this embodiment proposes a card border detection method. After obtaining the original card image, preprocessing is performed, and then the border line parameters of the card are obtained through the detection model, and then the direct detection and fitting are performed in segments based on the border line parameters. Processing to obtain the card frame in the image, and then the card frame can be detected by image edge detection, which is helpful to avoid misjudgment in the case of complex background or blurred edges, improves the accuracy of card frame detection, and serves as a follow-up card Information extraction and other processing provide convenience.

The card edge detection method in this embodiment is implemented by a card edge detection device. The card frame detection device can be a mobile phone, a tablet computer, a personal computer, a server, and other devices. In this embodiment, a mobile phone is used as an example for description. In this embodiment, before the inspection, a network model for detecting the outline of the card from the graph will be obtained (built) first, and this model may be called a detection model. The detection model of this embodiment is improved based on the deeplab v3 model; the original deeplab v3 model is relatively large, and the process of data processing (from input to output) takes a certain amount of time and has high requirements for hardware performance. In this embodiment, the detection is performed through a mobile phone, so the deeplab v3 model is improved to obtain a detection model. The detection model includes a codec and arithmetic unit, and the codec includes an encoder and a decoder. Of course, when building a detection model, a certain number of image samples need to be trained to obtain a detection model that meets certain requirements.

When the detection model is obtained, the mobile phone can first obtain the original card image to be detected. The original card image can be obtained by calling the camera of the mobile phone, or downloaded from the network or transmitted from other terminals; of course, the original card image There are cards included. When the mobile phone obtains the original card image, it will first preprocess the original card image so that the original card image meets the input requirements of the detection model, which is also beneficial to improve the processing efficiency of the detection model and reduce the performance pressure of the mobile phone.

Further, the step S10 includes:

Step S11: Obtain an RGB color original card image through a camera, and scale the RGB color original card image to obtain a scaled image of a preset size;

The original card image in this embodiment may be taken by a mobile phone through a camera, and the taken image may be an original card image in RGB color. For the RGB color original card image obtained by shooting, the user may not necessarily take it according to the input size requirements of the detection model when shooting, so the RGB color original card image needs to be scaled to obtain a scaled image of a preset size. For example, the RGB color original card image is reduced (or enlarged) or 128*256 size zoomed image.

In step S12, the pixel value of the zoomed image is reduced to obtain the image to be recognized.

After the zoomed image is obtained, the pixel value of the zoomed image will be adjusted, and the pixel value of the zoomed image will be reduced to obtain the image to be recognized for later image processing. The specific reduction method can be to divide the pixel value of the zoomed image by 255 , Get the new pixel value; of course, it can also be processed by binarization. Through the above method, the original card image can be obtained by shooting with a mobile phone, and the original card image can be preprocessed to zoom and reduce the pixels, so that the original card image meets the input requirements of the detection model, so that the image processing can be carried out later and the mobile phone can be reduced. Performance pressure.

Step S20, input the image to be recognized into a preset detection model, extract corresponding image feature information from the preset detection model, and calculate frame line parameters according to the image feature information;

In this embodiment, when the image to be recognized is obtained, the image to be recognized can be input to the detection model, and the corresponding image feature information can be obtained through the detection model. The process of feature extraction may include convolution and pooling of the image to be recognized. And other processing to extract image feature information that can represent image features from the image to be recognized. When the image feature information is obtained, the frame line parameter will be calculated according to the image feature information. The frame line parameter is used to describe the outline information of the card frame in the image. For example, it can be the line expression of the card frame, the coordinates of the vertices of the card, and the area where the card is located. Location information, etc.

Correspondingly, the preset detection model includes a codec and an arithmetic unit, and the step S20 includes:

Step S21, input the image to be recognized into a preset detection model, and perform feature extraction on the image to be recognized through a codec to obtain feature images of each side of the card;

The detection model in this embodiment includes a codec and an arithmetic unit, and the codec includes an encoder (Encoder) and a decoder (Decoder), and the image feature feature information may be in the form of a feature image. Specifically, the Encoder can use the shufflenet_0.5 network. Shufflenet is a lightweight network model that includes a grouped convolution part and a channel confusion part to extract image features from images; because this embodiment is applied to mobile phones , And the processing performance of the mobile phone has certain limitations. In order to improve the processing efficiency and reduce the performance pressure of the mobile phone, the shufflenet is reduced in this embodiment, and the width is set to 0.5 times the original. Data processing capacity; after the image to be recognized is encoded by the encoder, the size of the primary feature image obtained is reduced compared with the image to be recognized. For example, the size of the image to be recognized is 128*256, and the primary feature image is 32* 64 size. The decoder is based on the deeplab v3 model. In the ASPP (core module) of the deeplab v3 model, there are five branches. The decoder in this embodiment can take only two branches to reduce the amount of data processing; After obtaining a feature image, the first feature image will be transformed by Decoder (including convolution and pooling), so as to obtain the corresponding feature image (feature map); Compared with the primary image, the feature map has a certain enlargement in size. For example, the primary feature image is 32*64 size, and the feature image is 128*256 size.

In step S22, a weighted least square operation is performed on the feature image by the arithmetic unit to obtain the corresponding frame line parameter.

In this embodiment, after obtaining the feature When map, the feature can be The map performs a weighted least squares operation (Weighted_least_squares) to analyze the feature map to obtain the frame line parameters corresponding to the feature image. In this embodiment, the frame line parameters are used to describe the outline information of the card frame in the image. Among them, because the card has four borders, in order to improve the accuracy of the analysis, when extracting the feature image, you can output the corresponding feature map for each border, and then perform the weighted least square operation on each feature map to obtain The corresponding border line parameter. Specifically, for each feature map, you can first establish a coordinate system with a certain point as the origin of the coordinates, such as the center of the feature map, and then determine the feature coordinates of each feature point of the feature map, and establish the corresponding equation set: W*[y_map, 1] =A* W*x_map; In this equation system, W represents the preset eigenvalue, x_map is the eigenvalue of the x-axis coordinate, y_map is the eigenvalue of the y-axis coordinate, and A represents the border line parameter (matrix or collection method, For example, a and b in the straight line y=ax+b); by solving the above equations, A = inv(T(WY)*WY)*(T(WY)*WX), where T(WY) is the transpose of WY, inv(T(WY)*WY) is the inverse of (T(WY)*WY) .

Through the above method, the mobile phone can analyze the image to be recognized, and obtain the frame line parameters, roughly determine the position of the card frame according to the frame line parameters, and obtain the detection frame line in the original card image; in addition, because the detection model is end-to-end The end model is therefore also conducive to improving the accuracy of detection.

Step S30: Obtain the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge area of each side of the card in the original card image according to the detection frame line;

In this embodiment, when the frame line parameters are obtained, the mobile phone can roughly determine the position of the card frame according to the frame line parameters, and obtain the detection frame line in the original card image. It is worth noting that the detection frame line in this embodiment is obtained according to the approximate position of the card frame. In order to further improve the accuracy of frame detection, this embodiment will perform further analysis based on the detection frame line to more accurately locate Card border. When the detection frame line is obtained, the edge area of each side of the card can be respectively intercepted in the original card image based on the detection frame line. For example, the detection frame line is used as the baseline, and the line is translated a distance to both sides of the line to obtain the edge area. ; It can be considered that the actual sides of the card are located in the edge area. Of course, when obtaining the edge area, each detection frame line obtains the corresponding edge area, and a card is usually rectangular with four sides, that is, there are four detection frame lines, and then four edges are obtained. area.

Step S40, cutting the edge area of each side to obtain the cutting area of each side, and performing straight line detection on each cutting area to obtain the effective straight line of each cutting area;

In this embodiment, when obtaining the edge regions of each side, for each edge region, transverse cutting can be performed separately to obtain several cutting regions. Since each side of the card is actually located in the edge area, when cutting the edge area, it can also be considered as cutting the line where the card frame is located to obtain several straight lines (line segments); then, these cutting areas can be separately Perform straight line detection to obtain effective straight lines in each cutting area. Among them, when performing straight line detection, a suitable algorithm can be selected according to actual needs. For example, the detection can be based on Hough transform. Specifically, the points on the image (region) can be transformed into a set of parameter spaces. , According to the cumulative result of the parameter space points to find a solution corresponding to the maximum value, then this solution corresponds to the slope k and the constant b corresponding to the preset coordinates of the straight line, so as to detect the point; of course, it can also be based on LSD fast line detection algorithm and other methods for line detection.

Step S50, obtaining the vertices of the edge area of the same side according to the effective straight lines of the same side, and fitting the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

In this embodiment, when the effective line of each cutting area is obtained, the intersection of the effective line of each cutting area and the edge of the area will form two edge area vertices; if the edge area of one side is cut as x (x> 1) A cutting area, the edge area of the side corresponds to 2x edge area vertices. For each edge area, the vertices of the edge area can be fitted separately to obtain the corresponding frame fitting straight line. Since there are 4 edge regions, 4 border fitting straight lines will be obtained.

Step S60: Combine the borders corresponding to each side to fit a straight line to obtain a card border.

In this embodiment, when a straight line fitted to each frame is obtained, the straight line fitted to each frame can be combined to obtain a closed frame, which is the card frame.

Further, when the card frame is obtained, the card image in the frame can be recognized based on the card frame, and then the card information can be obtained. For example, the OCR (Optical Character Recognition, optical character recognition) module can be used to Each single-character image is recognized, and the OCR module can classify and recognize the single-character image through CNN (Convolutional Neural Networks, convolutional neural network) and other technologies. In this way, the card information in the card is recognized. The optical character recognition algorithm is used to recognize the card image in the card frame to obtain the character information in the card image, so that the user does not need to actively input the card information, which reduces the user's operations.

It should be emphasized that, in order to further ensure the privacy and security of the original card image and/or the card, the frame position information corresponding to the original card image and/or the card frame may also be stored in a node of a blockchain. Of course, when the card information is obtained, the card information can also be stored in a node of a blockchain.

The blockchain referred to in this embodiment is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In this embodiment, an original card image is obtained, and the original card image is preprocessed to obtain an image to be recognized; the image to be recognized is input to a preset detection model, and corresponding image features are extracted by the preset detection model Information, and calculate the frame line parameters according to the image feature information; obtain the detection frame line of the original card image according to the frame line parameters, and capture each card in the original card image according to the detection frame line Edge area of the side; cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area; obtain the same edge according to the effective straight line of the same side The area vertices are fitted to the edge area vertices of the same side to obtain the corresponding frame fitting straight line; the frame fitting straight lines corresponding to each side are combined to obtain the card frame. Through the above method, this embodiment performs preprocessing after acquiring the original card image, and then obtains the border line parameters of the card through the detection model, and then performs the post-processing of segmented direct detection and fitting based on the border line parameters to obtain the card in the image Border, so that the border of the card is detected by image edge detection, which helps to avoid misjudgment in the case of complex background or blurred edges, improves the accuracy of card border detection, and provides processing for subsequent card information extraction and other processing convenient.

Based on the first embodiment of the card frame detection method described above, a second embodiment of the card frame detection method of the present application is proposed.

In this embodiment, the step S40 includes:

Step S41: Perform edge detection on the edge area of each side respectively to obtain the edge contour of each side;

In this embodiment, when the edge regions of each side are obtained, edge detection is performed on each edge region to obtain the edge contour of each side; the edge detection may be implemented based on the sobel operator. The Sobel operator includes two sets of horizontal and vertical matrices, which are convolved with the edge area (image) in a plane to obtain the horizontal and vertical brightness difference approximations respectively, and then calculate the gradient based on the brightness difference approximation value at a certain point. If the gradient is greater than a certain A threshold, the point is an edge point. Of course, in order to reduce the amount of calculation and the space used by the built-in functions, it is possible to transform all the edge regions into the same direction, and then use only one operator to detect the edge contour (for example, the horizontal operator to detect the vertical border contour) .

Step S42, cutting the edge area of each side to obtain the cutting area of each side;

In this embodiment, after the edge contour is obtained, for each edge area, transverse cutting can be performed separately to obtain several cutting areas. It is worth noting that in this embodiment, the edge detection is performed first, and then the segmentation is performed; in practice, it may also be cut first, and then the edge detection is performed.

Step S43: Determine the gray scale threshold of each cut area by Otsu method and the edge contour, and perform binarization processing on each cut area according to the gray threshold;

After each cut area is obtained, the gray threshold of each cut area will be determined by the Otsu method (OTSU) and the edge contour of each edge area. The gray threshold can be regarded as the segmentation threshold between the foreground and the background. Specifically, for a cut area, suppose the proportion of foreground pixels in the cut area is w0, the average gray scale of foreground pixels u0, the proportion of background pixels in the cut area is w1, and the average gray scale of foreground pixels Degree u1, the variance between classes is g, then there is

g=w0*w1*(μ0-μ1)²

The traversal method can be used to continuously transform w0, u0, w1, and u1. When g is the largest, g is the gray threshold. When the gray threshold is obtained, the cutting area can be binarized according to the gray threshold, and the cutting area can be divided into white and gray (or black) areas.

In step S44, linear detection is performed on each of the binarized cutting areas to obtain the effective straight lines of each cutting area.

After the binarization, the straight line detection can be performed on each cutting area respectively, and the effective straight line of each cutting area can be obtained. It is worth noting that in some cutting areas, multiple straight lines may be detected, which may be caused by the complex background of the card or other interference factors. At this time, it is necessary to select one of them as the effective straight line of the cutting area. Specifically, if a certain cutting area is detected by a straight line, the number of area lines detected in the area is greater than one, then the cutting area can be recorded as the target cutting area; at this time, for each area line of the target cutting area, You can first determine the gray-scale area enclosed by the edge of the target cutting area, and determine the area of each gray-scale area; when the background or interference factors overlap with the card, the detected card area is generally larger than the actual avoidance. The gray area with the smallest gray area area can be selected (that is, the side with the smallest area is considered to be the actual edge of the card), and the area line corresponding to the gray area is determined as the effective line of the area.

Through the above method, the present embodiment can obtain the effective straight line of the corresponding area in each cutting area, thereby facilitating the subsequent fitting of the corresponding frame fitting line according to the effective line fitting of multiple areas, and improving the accuracy of the card frame detection.

Based on the first or second embodiment of the above-mentioned card frame detection method, a third embodiment of the card frame detection method of the present application is proposed.

In this embodiment, the step S50 includes:

Step S51, randomly selecting two test points from the edge area vertices of the same side to construct a test line, and calculating the test distance from each edge area vertex of the same side to the test line;

In this embodiment, when the effective line of each cutting area is obtained, the intersection of the effective line of each cutting area and the edge of the area will form two edge area vertices; if the edge area of one side is cut as x (x>1) A cutting area, the edge area of the side corresponds to a total of 2x edge area vertices. For each edge area, the vertices of the edge area can be fitted separately to obtain the corresponding frame fitting straight line. The straight line fitting in this embodiment will use the ransac straight line fitting method, which is beneficial to improve the anti-noise ability and reduce the adverse effect of the complex background of the card image or other interference factors on the frame recognition. Since the card has four edges, when fitting, the vertices of each edge area are respectively fitted. The following takes one of the edges as an example for description. For the clicks composed of the vertices of the edge area, two test points can be randomly selected to construct the corresponding test line, and then the test distances from the vertices of the other edge areas to the test line can be calculated respectively.

Step S52: Determine the edge area vertices whose test distance is less than the distance threshold, and obtain a test point set according to the edge area vertices whose test distance is less than the distance threshold;

When the test distance between the vertices of other edge areas and the test line is obtained, the edge area vertices whose test distance is less than the distance threshold will be further determined. These points can form a test point set, and the number of test points included in the test point set will be recorded.

Step S53: Repeat the above steps until n test point sets are obtained, where n is a positive integer greater than 1;

Repeat the above steps S51 and S52 (that is, reselect test points and construct a straight line, and then obtain test point sets) until n test point sets are obtained, where n is a positive integer greater than 1.

Step S54: Determine the number of test points in each test point set, and determine the test point set with the largest number of test points as the target point set;

When n test point sets are obtained, the number of test points in each test point set can be determined respectively, and the test point set with the largest number of test points is determined as the target point set.

In step S55, a straight line fitting is performed according to the test points of the target point set to obtain a corresponding frame fitting straight line.

When the target point set is obtained, a straight line fitting can be performed based on the test points of the target point set to obtain the corresponding frame fitting straight line. For the vertices of the edge region that do not belong to the target point set, they can be considered as noise data.

Through the above methods, this embodiment adopts the ransac straight line fitting method, which is beneficial to improve the anti-noise ability and reduce the adverse effect of the complex background of the card image or other interference factors on the frame recognition.

In addition, the embodiment of the present application also provides a card frame detection device.

Referring to FIG. 3, FIG. 3 is a schematic diagram of the functional modules of the first embodiment of the card frame detection device of the present application.

In this embodiment, the card frame detection device includes:

The image acquisition module 10 is configured to acquire an original card image, and preprocess the original card image to obtain an image to be recognized;

The model input module 20 is configured to input the image to be recognized into a preset detection model, extract corresponding image feature information from the preset detection model, and calculate frame line parameters according to the image feature information;

The area interception module 30 is configured to obtain the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge area of each side of the card in the original card image according to the detection frame line;

The straight line detection module 40 is used for cutting the edge area of each side to obtain the cutting area of each side, and detecting the straight line of each cutting area to obtain the effective straight line of each cutting area;

The straight line fitting module 50 is used to obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and to fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

The straight line combination module 60 is used to combine the frame corresponding to each side to fit a straight line to obtain the card frame.

Among them, each virtual function module of the above-mentioned card frame detection device is stored in the memory 1005 of the card frame detection device shown in FIG. Function.

Further, the image acquisition module 10 includes:

The image scaling unit is used to obtain the RGB color original card image through the camera, and scale the RGB color original card image to obtain a scaled image of a preset size;

The pixel reduction unit is used to reduce the pixel value of the zoomed image to obtain the image to be recognized.

Further, the preset detection model includes a codec and an arithmetic unit, and the model input module 20 includes:

The feature extraction unit is configured to input the image to be recognized into a preset detection model, and perform feature extraction on the image to be recognized through a codec to obtain feature images of each side of the card;

The weighting operation unit is used to perform a weighted least square operation on the feature image through an arithmetic unit to obtain the corresponding frame line parameter.

Further, the weighting operation unit is specifically configured to establish a rectangular coordinate system according to the characteristic image, and obtain the characteristic point coordinates of each characteristic point in the characteristic image according to the rectangular coordinate system; according to the characteristic of each characteristic point The point coordinates establish an equation set, and the equation set is solved to obtain the border line parameters of the original card image.

Further, the straight line detection module 40 includes:

The edge detection unit is used to perform edge detection on the edge area of each side to obtain the edge contour of each side;

The edge cutting unit is used to cut the edge area of each side to obtain the cutting area of each side;

A binary processing unit, configured to determine the gray-scale threshold of each cut area through the Otsu method and the edge contour, and perform binarization processing on each cut area according to the gray-scale threshold;

The straight line detection unit is used to perform straight line detection on each of the binarized cutting areas to obtain the effective straight line of each cutting area.

Further, the straight line detection unit is specifically configured to perform straight line detection on each of the binarized cutting areas to obtain the area straight lines of each cutting area; if there is a target cutting area with a number of area lines greater than one, the target cutting area is obtained The area of the gray area enclosed by the straight lines in each area; the area line corresponding to the area of the gray area with the smallest area is determined as the effective line of the target cutting area.

Further, the straight line fitting module 50 includes:

The point set acquisition unit is used to randomly select two test points from the edge area vertices of the same side to construct a test line, and calculate the test distance from each edge area vertex of the same side to the test line; determine that the test distance is less than the distance threshold The edge area vertices of, and the test point set is obtained according to the edge area vertices whose test distance is less than the distance threshold; the above steps are repeated until n test point sets are obtained, and n is a positive integer greater than 1;

The target determination unit is used to determine the number of test points in each test point set, and determine the test point set with the largest number of test points as the target point set;

The straight line fitting unit is used to perform straight line fitting according to the test points of the target point set to obtain the corresponding frame fitting straight line.

Among them, the function realization of each module in the above-mentioned card frame detection device corresponds to each step in the embodiment of the above-mentioned card frame detection method, and the functions and realization processes thereof will not be repeated here.

In addition, the embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium may be volatile or non-volatile.

A computer program is stored on the computer-readable storage medium of the present application, and when the computer program is executed by a processor, the steps of the method for detecting the card frame as described above are realized.

The method implemented when the computer program is executed can refer to the various embodiments of the card frame detection method of the present application, which will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disk, optical disk), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A method for detecting a card frame, wherein the method for detecting a card frame includes:

Acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

Inputting the image to be recognized into a preset detection model, extracting corresponding image feature information from the preset detection model, and calculating frame line parameters according to the image feature information;

Acquiring the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge regions of each side of the card in the original card image according to the detection frame line;

Cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area;

Obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

Combine the borders corresponding to each side to fit a straight line to obtain the card border.
The card frame detection method according to claim 1, wherein the preset detection model includes a codec and an arithmetic unit, and the image to be recognized is input to the preset detection model, and the preset detection model The steps of extracting and obtaining corresponding image feature information, and calculating frame line parameters according to the image feature information include:

Input the image to be recognized into a preset detection model, and perform feature extraction on the image to be recognized through a codec to obtain feature images of each side of the card;

The weighted least squares operation is performed on the feature image through the arithmetic unit to obtain the corresponding border line parameters.
3. The card frame detection method according to claim 2, wherein the step of performing a weighted least squares operation on the characteristic image by an arithmetic unit to obtain the corresponding frame line parameters comprises:

Establishing a rectangular coordinate system according to the characteristic image, and acquiring characteristic point coordinates of each characteristic point in the characteristic image according to the rectangular coordinate system;

An equation set is established according to the feature point coordinates of each feature point, and the equation set is solved to obtain the border line parameters of the original card image.
The card frame detection method according to claim 1, wherein the edge area of each side is cut to obtain the cutting area of each side, and the linear detection of each cutting area is performed to obtain the effective straight line of the area of each cutting area. The steps include:

Perform edge detection on the edge area of each side respectively to obtain the edge contour of each side;

Cut the edge area of each side to obtain the cutting area of each side;

Determine the gray-scale threshold of each cut area by Otsu method and the edge contour, and perform binarization processing on each cut area according to the gray-scale threshold;

The linear detection is performed on each of the binarized cutting areas, and the effective straight lines of each cutting area are obtained.
5. The card frame detection method according to claim 4, wherein the step of performing linear detection on each of the binarized cutting areas to obtain the effective straight line of each cutting area comprises:

Perform straight line detection on each of the binarized cutting areas to obtain the area straight line of each cutting area;

If there is a target cutting area in which the number of area straight lines is greater than one, obtain the gray-scale area area enclosed by the straight lines of each area in the target cutting area;

The area line corresponding to the area of the gray area with the smallest area is determined as the effective line of the target cutting area.
5. The card border detection method according to claim 1, wherein the step of fitting the vertices of the edge area of the same side to obtain the corresponding border fitting straight line comprises:

Randomly select two test points from the vertices of the edge area on the same side to construct a test line, and calculate the test distance from the vertices of each edge area on the same side to the test line;

Determine the edge area vertices whose test distance is less than the distance threshold, and obtain the test point set according to the edge area vertices whose test distance is less than the distance threshold;

Repeat the above steps until n test point sets are obtained, where n is a positive integer greater than 1;

Determine the number of test points in each test point set, and determine the test point set with the largest number of test points as the target point set;

Perform straight line fitting according to the test points of the target point set, and obtain the corresponding frame fitting straight line.
8. The card frame detection method according to claim 1, wherein the original card image and/or frame position information corresponding to the card frame is stored in a blockchain.
A card frame detection device, wherein the card frame detection device includes:

An image acquisition module for acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

A model input module, configured to input the image to be recognized into a preset detection model, extract corresponding image feature information from the preset detection model, and calculate frame line parameters according to the image feature information;

An area interception module, configured to obtain the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge area of each side of the card in the original card image according to the detection frame line;

The straight line detection module is used to cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area;

The straight line fitting module is used to obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and to fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

The straight line combination module is used to combine the borders corresponding to each side to fit a straight line to obtain the card border.
A card frame detection device, wherein the card frame detection device includes a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program is executed by the processor When executing, implement the following steps:

Acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

Inputting the image to be recognized into a preset detection model, extracting corresponding image feature information from the preset detection model, and calculating frame line parameters according to the image feature information;

Acquiring the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge regions of each side of the card in the original card image according to the detection frame line;

Cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area;

Obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

Combine the borders corresponding to each side to fit a straight line to obtain the card border.
The card frame detection device according to claim 9, wherein the preset detection model includes a codec and an arithmetic unit, and the image to be recognized is input to the preset detection model, and the preset detection model The steps of extracting and obtaining corresponding image feature information, and calculating frame line parameters according to the image feature information include:

Input the image to be recognized into a preset detection model, and perform feature extraction on the image to be recognized through a codec to obtain feature images of each side of the card;

The weighted least squares operation is performed on the feature image through the arithmetic unit to obtain the corresponding border line parameters.
10. The card frame detection device according to claim 10, wherein the step of performing a weighted least squares operation on the characteristic image by an arithmetic unit to obtain the corresponding frame line parameters comprises:

Establishing a rectangular coordinate system according to the characteristic image, and acquiring characteristic point coordinates of each characteristic point in the characteristic image according to the rectangular coordinate system;

An equation set is established according to the feature point coordinates of each feature point, and the equation set is solved to obtain the border line parameters of the original card image.
The card frame detection device according to claim 9, wherein the edge area of each side is cut to obtain the cutting area of each side, and the linear detection of each cutting area is performed to obtain the effective straight line of each cutting area. The steps include:

Perform edge detection on the edge area of each side respectively to obtain the edge contour of each side;

Cut the edge area of each side to obtain the cutting area of each side;

Determine the gray-scale threshold of each cut area by Otsu method and the edge contour, and perform binarization processing on each cut area according to the gray-scale threshold;

The linear detection is performed on each of the binarized cutting areas, and the effective straight lines of each cutting area are obtained.
12. The card frame detection device according to claim 12, wherein the step of performing linear detection on each of the binarized cutting areas to obtain an effective straight line of each cutting area comprises:

Perform straight line detection on each of the binarized cutting areas to obtain the area straight line of each cutting area;

If there is a target cutting area in which the number of area straight lines is greater than one, obtain the gray-scale area area enclosed by the straight lines of each area in the target cutting area;

The area line corresponding to the area of the gray area with the smallest area is determined as the effective line of the target cutting area.
9. The card frame detection device according to claim 9, wherein the step of fitting the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line comprises:

Randomly select two test points from the vertices of the edge area on the same side to construct a test line, and calculate the test distance from the vertices of each edge area on the same side to the test line;

Determine the edge area vertices whose test distance is less than the distance threshold, and obtain the test point set according to the edge area vertices whose test distance is less than the distance threshold;

Repeat the above steps until n test point sets are obtained, where n is a positive integer greater than 1;

Determine the number of test points in each test point set, and determine the test point set with the largest number of test points as the target point set;

Perform straight line fitting according to the test points of the target point set, and obtain the corresponding frame fitting straight line.
9. The card frame detection device according to claim 9, wherein the original card image and/or frame position information corresponding to the card frame is stored in a blockchain.
A readable storage medium, wherein a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the following steps are implemented:

Acquiring an original card image, and preprocessing the original card image to obtain an image to be recognized;

Inputting the image to be recognized into a preset detection model, extracting corresponding image feature information from the preset detection model, and calculating frame line parameters according to the image feature information;

Acquiring the detection frame line of the original card image according to the frame line parameter, and respectively intercept the edge regions of each side of the card in the original card image according to the detection frame line;

Cut the edge area of each side to obtain the cutting area of each side, and perform straight line detection on each cutting area to obtain the effective straight line of each cutting area;

Obtain the vertices of the edge area of the same side according to the effective straight lines of the same side, and fit the vertices of the edge area of the same side to obtain the corresponding frame fitting straight line;

Combine the borders corresponding to each side to fit a straight line to obtain the card border.
The readable storage medium of claim 16, wherein the preset detection model includes a codec and an arithmetic unit, and the image to be recognized is input to the preset detection model, and the preset detection model The steps of extracting and obtaining corresponding image feature information, and calculating frame line parameters according to the image feature information include:

Input the image to be recognized into a preset detection model, and perform feature extraction on the image to be recognized through a codec to obtain feature images of each side of the card;

The weighted least squares operation is performed on the feature image through the arithmetic unit to obtain the corresponding border line parameters.
17. The readable storage medium according to claim 17, wherein the step of performing a weighted least squares operation on the characteristic image by an arithmetic unit to obtain the corresponding frame line parameter comprises:

Establishing a rectangular coordinate system according to the characteristic image, and acquiring characteristic point coordinates of each characteristic point in the characteristic image according to the rectangular coordinate system;

An equation set is established according to the feature point coordinates of each feature point, and the equation set is solved to obtain the border line parameters of the original card image.
The readable storage medium according to claim 16, wherein said cutting the edge area of each side to obtain the cutting area of each side, and performing linear detection on each cutting area to obtain the effective straight line of the area of each cutting area The steps include:

Perform edge detection on the edge area of each side respectively to obtain the edge contour of each side;

Cut the edge area of each side to obtain the cutting area of each side;

Determine the gray-scale threshold of each cut area by Otsu method and the edge contour, and perform binarization processing on each cut area according to the gray-scale threshold;

The linear detection is performed on each of the binarized cutting areas, and the effective straight lines of each cutting area are obtained.
19. The readable storage medium according to claim 19, wherein the step of performing linear detection on each of the binarized cutting areas to obtain a valid straight line of each cutting area comprises:

Perform straight line detection on each of the binarized cutting areas to obtain the area straight line of each cutting area;

If there is a target cutting area in which the number of area straight lines is greater than one, then obtain the gray-scale area area enclosed by the straight lines of each area in the target cutting area;

The area line corresponding to the area of the gray area with the smallest area is determined as the effective line of the target cutting area.