CN112364675B - Off-line reading method and device based on three-dimensional code and image recognition - Google Patents
Off-line reading method and device based on three-dimensional code and image recognition Download PDFInfo
- Publication number
- CN112364675B CN112364675B CN202011171971.4A CN202011171971A CN112364675B CN 112364675 B CN112364675 B CN 112364675B CN 202011171971 A CN202011171971 A CN 202011171971A CN 112364675 B CN112364675 B CN 112364675B
- Authority
- CN
- China
- Prior art keywords
- image
- dimensional code
- matrix
- model
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1439—Methods for optical code recognition including a method step for retrieval of the optical code
- G06K7/1443—Methods for optical code recognition including a method step for retrieval of the optical code locating of the code in an image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K19/00—Record carriers for use with machines and with at least a part designed to carry digital markings
- G06K19/06—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
- G06K19/06009—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code with optically detectable marking
- G06K19/06037—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code with optically detectable marking multi-dimensional coding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K19/00—Record carriers for use with machines and with at least a part designed to carry digital markings
- G06K19/06—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
- G06K19/06009—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code with optically detectable marking
- G06K19/06046—Constructional details
- G06K19/06131—Constructional details the marking comprising a target pattern, e.g. for indicating the center of the bar code or for helping a bar code reader to properly orient the scanner or to retrieve the bar code inside of an image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Electromagnetism (AREA)
- Toxicology (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an off-line reading method and device based on three-dimensional code and image recognition, wherein the method comprises the following steps: establishing an original image set; the method comprises the following steps of (1) trimming an original image into a standardized image material and putting the standardized image material into a material set; extracting a characteristic matrix through a pixel matrix of the image material, and putting the characteristic matrix into a vector set; training an image reconstruction model capable of restoring the corresponding image material through the feature matrix through a convolutional neural network; identifying each feature matrix in the vector set through the character string to obtain a relation set of the character string identification and the feature matrix; coding the character string identification and the corresponding image material to obtain a three-dimensional code; training by using a deep learning network model and obtaining an image recognition model; and calling the vector set, the relation set, the image reconstruction model and the image identification model, importing the vector set, the relation set, the image reconstruction model and the image identification model into a three-dimensional code identification module, and identifying, reading and verifying the image containing the three-dimensional code.
Description
Technical Field
The invention relates to an off-line reading method and device based on three-dimensional codes and image recognition, and belongs to the technical field of three-dimensional code technology and image recognition.
Background
At present, many mobile phone applications have a Quick Response Code (QR Code) Code scanning function, and the applications of the Quick Response Code are more and more extensive, including but not limited to health codes, payment codes, collection codes, merchandise anti-counterfeiting codes, and certificate codes printed on certificates. The authenticity of these codes cannot be identified using conventional code scanning software or networking is required for identification. Meanwhile, the quick response code only contains information with limited capacity, but cannot store information such as pictures and texts, and the data which can be acquired in an off-line state is very limited. If the pictures or texts are directly stored in the code scanning software, the risk of leakage exists. An image recognition technology refers to a technology for processing, analyzing and understanding images by using a computer to recognize various different modes of targets and objects, and is a practical application of applying a deep learning algorithm.
The three-dimensional code is obtained by encoding an image and data information into a set of image symbols with larger information capacity through a specific algorithm (such as a three-dimensional code encoding method and a system disclosed in CN 201610080300.4) in combination with a two-dimensional bar code system and the overall color content of the image. The three-dimensional code is convenient for a user to visually observe visual graphic image information of the three-dimensional code, can read combination information of related modules through quick response code reading equipment and software, and can be matched with special equipment and software to expand functions based on coding and decoding.
In the prior art, information such as pictures and texts can be provided in an off-line state through a three-dimensional code, and authenticity identification can be performed through the pictures in the three-dimensional code through an image identification technology. However, the prior art lacks a scheme for reading and verifying the three-dimensional code and the image recognition technology in an off-line environment.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides an off-line reading method and device based on three-dimensional code and image identification, which solve the problems of identification and verification of the three-dimensional code in an off-line environment and solve the problem of leakage prevention of original data.
The technical scheme of the invention is as follows:
the first technical scheme is as follows:
an off-line reading method based on three-dimensional codes and image recognition comprises the following steps:
establishing an original image set, collecting a plurality of original images containing objects to be identified and putting the original images into the original image set;
carrying out image preprocessing, preprocessing each original image in the original image set, adjusting the original image into an image material set according to a required format and putting the image material into the material set;
carrying out picture reconstruction, processing each image material in the material set to obtain a pixel matrix of each image material, extracting a characteristic matrix in a vector matrix, and storing the characteristic matrix extracted from each image material in the vector set; meanwhile, training an image reconstruction model capable of restoring the corresponding image material through the feature matrix through a convolutional neural network;
compiling a three-dimensional code, and identifying each characteristic matrix in the vector set through a character string to obtain a relation set of the character string identification and the characteristic matrix; coding the character string identification and image materials corresponding to the character string-associated feature matrix, wherein the image materials are used as visual feature elements recognizable to human eyes and are displayed in a designated area, and the character string is converted into code points, so that a three-dimensional code is obtained;
training an image recognition model, and training the deep learning network model by using image materials in the material set to obtain the image recognition model;
and reading and verifying, calling the vector set, the relation set, the image reconstruction model and the image identification model, importing the vector set, the relation set, the image reconstruction model and the image identification model into a three-dimensional code identification module, and reading and verifying the image containing the three-dimensional code.
Further, the objects to be identified include, but are not limited to: faces, objects, and text; and after the original images are put into the original image set, a verification step is also carried out, each original image is confirmed to contain the object to be identified through a target detection algorithm, and the original images without the detected object are moved out of the original image set.
Further, the step of adjusting the original image into an image material set according to a required format and placing the image material into the material set specifically comprises:
detecting the position of an object to be recognized in an original image through a target detection algorithm, and making a minimum rectangular frame capable of containing the object to be recognized;
taking the minimum rectangular frame as a center, and cutting out a square in the original image to enable the area of the square to be N times of the area of the minimum rectangular frame;
and zooming the intercepted image, storing the image into a picture with a required size and a required format, and putting the picture into a material set.
Further, the step of obtaining the pixel matrix of each image material, extracting the feature matrix in the vector matrix, and storing the feature matrix extracted from each image material in the vector set specifically comprises:
obtaining a pixel matrix A of the image material of m-n order according to a formula
Singular value operation is carried out on the pixel matrix to obtain the matrixes U, sigma and VT；
Extracting K singular values from the matrix sigma by a principal component analysis method to enable the proportion of the sum of the K singular values in the sum of all the singular values to be P, and obtaining a new matrix U according to the positions of the K singular values1、∑1And V1 T；
Will matrix U1The feature matrix as the image material is put into a vector set.
Further, the step of training, by using the convolutional neural network, an image reconstruction model capable of restoring the corresponding image material by using the feature matrix specifically includes:
developing an image reconstruction model using a convolutional neural network, using ∑1And V1 TAs a calculation parameter of the model, a reconstructed picture formula U is constructed1∑1V1 T＝A1；
Calculating a pixel matrix A of a reconstructed picture through an image reconstruction model and a reconstructed picture formula by using the vector set as a training set1Adjust sigma1And V1 TSo that the pixel matrix A of the picture is reconstructed1The sum of the pixel values of the pixel matrix A corresponding to the image material is minimum;
saving the adjusted sigma1And V1 TAnd outputting the trained image reconstruction model.
Further, the deep learning network model is a feature extraction model, the feature extraction model is subjected to reinforced training by taking the image pixel material set as a training set, and the trained feature extraction model is stored as an image recognition model.
Furthermore, after the image containing the three-dimensional code is read and verified, an information display step is also included, relevant information, including but not limited to introduction, object data and extension information of an object to be identified in the image material, is stored in the client corresponding to each image material, and after the three-dimensional code is verified successfully, the relevant information of the corresponding image material is displayed.
Further, the steps of reading and verifying the picture containing the three-dimensional code specifically include:
acquiring a three-dimensional code image through a camera, identifying a three-dimensional code in the image through a three-dimensional code identification module, analyzing a coded character string identifier of the three-dimensional code and acquiring a three-dimensional code picture P1 after distortion correction;
acquiring a characteristic matrix corresponding to the three-dimensional code through the character string identification, and putting the characteristic matrix into an image reconstruction model for reconstruction to acquire a reconstructed picture P2;
identifying whether the objects in the reconstructed picture P2 and the three-dimensional code picture P1 are the same or not through an image identification model;
if the three-dimensional code image is the same, restoring and displaying the corresponding image material through the feature matrix of the object and the image reconstruction model, and displaying the image material at the client to indicate that the three-dimensional code image is effective.
The second technical scheme is as follows:
an off-line reading device based on three-dimensional code and image recognition comprises a memory and a processor, wherein the memory stores instructions, and the instructions are suitable for being loaded by the processor and executing an off-line reading method based on three-dimensional code and image recognition according to the first technical scheme.
The invention has the following beneficial effects:
1. the invention relates to an off-line reading method based on three-dimensional codes and image recognition, which is characterized in that a feature matrix with image features is obtained by collecting and preprocessing an original image of an object to be recognized in advance, the three-dimensional codes are subjected to off-line recognition and verification through an image reconstruction model capable of reconstructing the image through the feature matrix and an image recognition model through training, only the feature matrix and the relationship between the feature matrix and the image are stored in a client, the original image cannot be directly obtained, and under the condition that the client is cracked, the data and the model are extremely difficult to crack, the content of the original image is difficult to leak, and therefore the safety of the original image is ensured.
2. According to the off-line recognizing and reading method based on the three-dimensional code and the image recognition, the singular value decomposition method is used for conducting principal component analysis on the image material, the feature matrix of the image material is stored, the high reducibility of the image can be guaranteed under the condition of saving space, the capacity is greatly reduced compared with the capacity of the image material, and the problem of large-capacity storage is solved.
3. According to the off-line identification and reading method based on the three-dimensional code and the image identification, after the image identification model is trained through the material set, the identification precision of objects in the material set is improved, in an actual use scene, the step of manually checking faces or objects is omitted, subjective factors are removed, and the checking efficiency and precision are improved.
4. According to the off-line reading method based on the three-dimensional code and the image recognition, the relevant information is stored in the client corresponding to each image material, and the display of the object information can be realized on the premise of not directly coding the information into the three-dimensional code.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention;
FIG. 2 is an exemplary diagram of an original image;
FIG. 3 is an exemplary diagram of obtaining a minimum rectangular box that can contain an object in an original image by an object detection algorithm;
FIG. 4 is an exemplary diagram of image material after preprocessing;
FIG. 5 is an exemplary diagram of a three-dimensional code;
FIG. 6 is an exemplary diagram of picture reconstruction;
FIG. 7 is a schematic diagram of feature matrix extraction and pixel matrix reduction using singular value decomposition.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
Example one
Referring to fig. 1, an off-line reading method based on three-dimensional code and image recognition includes the following steps:
The principle of extracting the feature matrix and restoring the pixel matrix by the singular value decomposition method is shown in fig. 7; in fig. 7, the extraction calculation for obtaining the feature matrix is performed from left to right, and the reduction calculation for obtaining the pixel matrix is performed from right to left. The pixel matrix a is decomposed into three matrix multiplications, where Σ is the feature matrix, which is also a diagonal matrix, and the values on the diagonal are singular values. After the singular values are sequenced, the first k singular values with the maximum value are obtained, and the corresponding U is obtainedkAnd Vk TSaving matrix U in model M1kAnd Vk TValue of each element, reuse ∑kAnd A is used as a training set to train the model M1, so that U in M1kAnd Vk TEach of (1)The fine tuning of the values of the elements brings the result of the reduction calculation close to a. U shapekIn which there are m x k elements, sigmakOnly k elements on the diagonal are saved, Vk TThere are n × k elements, so that m × k + k + n × k values are kept, i.e., (m + n + 1) × k values, to restore the pixel matrix close to a.
In the embodiment, the vector set S3 and the relation set S4 are stored in the client, the image materials in the material set S2 cannot be directly obtained by identifying and reconstructing the image materials through the image reconstruction model M1 and the image identification model M2, and under the condition that the APP is cracked, the data and the models are extremely difficult to crack, the content of the image materials is difficult to leak, so that the safety of the image materials is ensured; meanwhile, the extracted feature matrix is stored for the image materials, so that the capacity is greatly reduced compared with the image materials, and the problem of large-capacity storage is solved.
Example two:
further, the objects to be identified include, but are not limited to: in this embodiment, the original images collected by the original image set S1 are all portrait pictures; in step 101a, the original image is further checked after being placed in the original image set S1, the original image is converted into a pixel value of each pixel point, the pixel value is taken out and stored in a pixel array, and the pixel array is used as a parameter and is transmitted to a detectincyscale function of OpenCV for target detection. The function identifies key coordinate points with clear semantics on the face, such as the nose tip, the mouth corner, the canthus, etc. If the key coordinate points are not identified, the function considers that no human face exists in the picture; if the face is identified, the face is considered to exist in the picture; and the picture without the detected face is regarded as an unqualified picture, and is removed from the original image set S1, and the picture with the plurality of faces detected or the picture with the largest area is taken as the standard.
Further, referring to fig. 2 to 4, the step 102a specifically includes:
performing face alignment operation according to the key coordinate points by using a target detection algorithm, namely solving a spatial transformation matrix between the key coordinate points of the current face image and predefined standard face key coordinate points (usually obtained by statistics) by using a least square method, and performing transformation corresponding to the spatial transformation matrix on the face image, thereby calculating the position of the face and making a minimum rectangular frame R capable of containing the position, referring to fig. 2 and 3, wherein fig. 2 is an original image, and the frame in fig. 3 is the minimum rectangular frame R;
taking the minimum rectangular frame R as a center, cutting out a square in the original image to enable the area of the square to be N times of the area of the minimum rectangular frame, wherein the area of the square is 5 times of the area of the minimum rectangular frame, and if the side length of the square exceeds the width or the length of the original image, the original image is cut out after the original image is supplemented to the required width and length by white; in order to accelerate the processing speed of the picture in the subsequent steps and simplify the data processing method of the picture, the intercepted picture is subjected to gray processing;
zooming the intercepted image, storing the image into a JPEG-format picture of m multiplied by n, and putting the JPEG-format picture into a material set, wherein m and n both take the value of 256 in the embodiment; an example of image material is shown in fig. 4.
Further, in step 103a, a Singular Value Decomposition (SVD) is used to perform a Principal Component Analysis (PCA) on the picture in S2, so as to extract a feature matrix:
first, a pixel matrix a of m × n-level image materials is obtained, in this embodiment, a is a 256 × 256 matrix, and the type of each value in the matrix is int integer. Since the picture is grayed in step 102a, each value in the matrix is only the brightness value of the corresponding pixel.
According to the formula
Singular value operation is carried out on the pixel matrix to obtain the matrixes U, sigma and VT(ii) a Where Σ is all 0 except for the elements on the main diagonal, the elements on the diagonal are singular values. The vectors inside U and V are orthogonal, U and V are called left and right singular vectors in singular values, respectively, and are unitary matrices, VTIs the transpose of V.
Obtaining m singular values in total according to the size of the picture, extracting K singular values from the matrix sigma by a principal component analysis method to enable the proportion of the sum of the K singular values in the sum of all the singular values to be P, and obtaining a new matrix U according to the positions of the K singular values1、∑1And V1 T；
In this embodiment, the relationship between K and P is as follows:
| K | P |
| 10 | 90％ |
| 30 | 95％ |
| 50 | 99％ |
| 70 | 99.8％ |
in this embodiment, the size of the original image is 256 × 256, the pixel point value to be saved is 65536, only the first 10 singular values are taken by using the SVD method, and the sum of all singular values is 90%, at this time, only (256 + 1) × 10= 5130) of the pixel points per image needs to be saved, so that the storage space is saved. The usage method can simultaneously ensure high reducibility of the picture under the condition of saving space, specifically referring to fig. 6, where fig. 6 is a reconstruction case of this embodiment, and when a singular value of 50 is taken, the ratio of the number of values to be stored to the original picture is (512 + 1) × 50/(512 + 512) =19.55%, and thus a high reducibility effect can be achieved.
Will matrix U1The feature matrices are placed into vector sets as the image material.
Further, the step of training, by using the convolutional neural network, an image reconstruction model capable of restoring the corresponding image material by using the feature matrix specifically includes:
based on the tensoflow deep learning framework, an image reconstruction model M1 developed by using a Convolutional Neural Network (CNN) method uses sigma1And V1 TAs a calculation parameter of the model, a reconstructed picture formula U is constructed1∑1V1 T＝A1(ii) a Direction of useThe quantity set S3 is used as a training set, and the pixel matrix A of the reconstructed picture is calculated through an image reconstruction model and a reconstructed picture formula1By continuously fine-tuning the matrix sigma1And V1 TSuch that the pixel matrix a of the picture is reconstructed1The sum of gray differences of all pixels of a pixel matrix A corresponding to the image material is minimum; the formula for calculating the sum of the gray level differences of all the pixel points is as follows:
in this example, a total of 200 rounds of training were performed, and after training, the adjusted sigma was saved1And V1 TAnd outputting the trained image reconstruction model.
Furthermore, the feature extraction model used by the deep learning network model is an open-source facenet model, the facenet model adopts a deepID2+ deep learning network model, the face features can be extracted and converted into a 512-dimensional vector while the face is recognized, and the maximum face area is used as the standard when a plurality of faces exist in the picture. Each component in the vector is a 9-bit double type number reserved after the decimal point, such as [0.045051873,0.009796426, -0.01565706, 0.077138886 \8230 ];
in the present embodiment, the image recognition model M2 performs reinforcement training using the material set S2 as a training set. And then, judging whether the two faces are the same person or not by comparing Euclidean distances between the feature vectors of the two faces. The formula for calculating the euclidean distance of vectors X and Y is as follows:
and saving the trained feature extraction model as an image recognition model.
Furthermore, after the image printed with the three-dimensional code is read and verified through the client and the three-dimensional code recognition module, the method further comprises an information display step, relevant information is stored in the client corresponding to each image material, including but not limited to introduction, object data and extension information of an object to be recognized in the image material, and after the three-dimensional code is verified successfully, the relevant information corresponding to the image material is displayed.
Further, the step of performing offline reading on the picture printed with the three-dimensional code through the client and the three-dimensional code identification module specifically comprises:
acquiring a three-dimensional code image in a picture printed with a three-dimensional code by a camera of equipment loaded with a client, identifying the three-dimensional code in the image by a three-dimensional code identification module, analyzing a coded character string identifier CI of the three-dimensional code, and acquiring a three-dimensional code picture P1 after correcting distortion (the correction distortion is the function of the existing three-dimensional code identification module which decodes during identification by the following steps of binarization → code finding in positioning → correction distortion → sequence of read code points 01 → decoding);
searching a corresponding feature matrix in the relation set S4 according to the character string identification CI, and prompting an error when the character string identification CI cannot be searched in the relation set S4; after finding out a corresponding characteristic matrix, putting the characteristic matrix into an image reconstruction model M1 for reconstruction operation to obtain a reconstructed picture P2;
after face feature vectors of a reconstructed picture P2 and a three-dimensional code picture P1 are extracted through an image recognition model M2, whether Euclidean distance between the two feature vectors is smaller than a preset threshold value T or not is calculated, and therefore whether the faces in the P1 and the P2 are the same or not is judged;
if the images are different, an error is prompted, and if the images are the same, the corresponding image materials are restored and displayed through the feature matrix and the image reconstruction model of the object at the client side, so that the three-dimensional code image is effective.
In the embodiment, the singular value decomposition method is used for analyzing the principal components of the image materials, and the feature matrix of the image materials is stored, so that the high reducibility of the image can be ensured under the condition of saving space; after the image recognition model M2 is trained through the material set S2, the identification precision of the object in the material set S2 is improved, in an actual use scene, the step of manually checking a human face or an object is saved, subjective factors are removed, and the checking efficiency and precision are improved; the client stores the related information corresponding to each image material, so that the object information can be displayed on the premise of not directly coding the information into the three-dimensional code.
EXAMPLE III
An off-line reading device based on three-dimensional code and image recognition comprises a memory and a processor, wherein the memory stores instructions, and the instructions are suitable for being loaded by the processor and executing the off-line reading method based on three-dimensional code and image recognition as described in the first embodiment and the second embodiment.
The above description is only an embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are included in the scope of the present invention.
Claims (9)
1. An off-line reading method based on three-dimensional codes and image recognition is characterized by comprising the following steps:
establishing an original image set, collecting a plurality of original images containing objects to be identified and putting the original images into the original image set;
carrying out image preprocessing, preprocessing each original image in the original image set, adjusting the original image into an image material set according to a required format and putting the image material into the material set;
carrying out picture reconstruction, processing each image material in the material set to obtain a pixel matrix of each image material, extracting a characteristic matrix in the pixel matrix, and storing the characteristic matrix extracted from each image material into a vector set; meanwhile, training an image reconstruction model capable of restoring the corresponding image material through the feature matrix through a convolutional neural network;
compiling a three-dimensional code, and identifying each characteristic matrix in the vector set through a character string to obtain a relation set of the character string identification and the characteristic matrix; coding the character string identification and image materials corresponding to the character string-associated feature matrix, wherein the image materials are used as visual feature elements recognizable to human eyes and are displayed in a designated area, and the character string is converted into code points, so that a three-dimensional code is obtained;
training an image recognition model, and training the deep learning network model by using image materials in the material set to obtain the image recognition model;
and reading and verifying, calling the vector set, the relation set, the image reconstruction model and the image identification model, importing the vector set, the relation set, the image reconstruction model and the image identification model into a three-dimensional code identification module, and reading and verifying the image containing the three-dimensional code.
2. The off-line reading method based on three-dimensional code and image recognition as claimed in claim 1, wherein the object to be recognized includes but is not limited to: faces, objects and text; and after the original images are put into the original image set, a verification step is also carried out, each original image is confirmed to contain the object to be identified through a target detection algorithm, and the original images without the detected object are moved out of the original image set.
3. The off-line identification and reading method based on the three-dimensional code and the image recognition according to claim 1, wherein the step of adjusting the original image into the image material set according to the required format and putting the image material into the material set specifically comprises:
detecting the position of an object to be identified in an original image through a target detection algorithm, and making a minimum rectangular frame capable of containing the object to be identified;
taking the minimum rectangular frame as a center, and cutting a square in the original image to enable the area of the square to be N times of the area of the minimum rectangular frame;
and zooming the intercepted image, storing the image into a picture with a required size and a required format, and putting the picture into a material set.
4. The off-line reading method based on the three-dimensional code and the image recognition according to claim 1, wherein the steps of obtaining the pixel matrix of each image material, extracting the feature matrix in the vector matrix, and storing the feature matrix extracted from each image material in the vector set specifically include:
obtaining a pixel matrix A of the image material of m-n order according to a formula
Singular value operation is carried out on the pixel matrix to obtain the matrixes U, sigma and VT(ii) a Wherein the matrix U is a unitary matrix of order mxm; the matrix sigma is a semi-positive definite m multiplied by n diagonal matrix; matrix VTIs a unitary matrix of order n x n;
extracting K singular values from the matrix sigma by a principal component analysis method to enable the proportion of the sum of the K singular values in the sum of all the singular values to be P, and obtaining a new matrix U according to the positions of the K singular values1、∑1And V1 T；
Will matrix U1The feature matrices are placed into vector sets as the image material.
5. The off-line recognizing and reading method based on the three-dimensional code and the image recognition as claimed in claim 4, wherein the step of training the image reconstruction model capable of restoring the corresponding image material through the feature matrix by the convolutional neural network specifically comprises:
developing an image reconstruction model using a convolutional neural network, using ∑1And V1 TAs the calculation parameters of the model, a reconstructed picture formula U is constructed1∑1V1 T＝A1；
Calculating a pixel matrix A of a reconstructed picture through an image reconstruction model and a reconstructed picture formula by using the vector set as a training set1Adjust sigma1And V1 TSo that the pixel matrix A of the picture is reconstructed1The sum of the pixel values of the pixel matrix A corresponding to the image material is minimum;
saving the adjusted sigma1And V1 TAnd outputting the trained image reconstruction model.
6. The off-line recognition method based on the three-dimensional code and the image recognition as claimed in claim 1, wherein the deep learning network model is a feature extraction model, a pixel material set is used as a training set to perform enhanced training on the feature extraction model, and the trained feature extraction model is saved as the image recognition model.
7. The off-line reading method based on the three-dimensional code and the image identification as claimed in claim 1, further comprising an information displaying step after the image containing the three-dimensional code is read and verified, wherein the client stores relevant information corresponding to each image material, including but not limited to introduction, object data and extension information of an object to be identified in the image material, and when the three-dimensional code is successfully verified, the relevant information corresponding to the image material is displayed.
8. The off-line reading method based on the three-dimensional code and the image recognition as claimed in claim 1, wherein the step of reading and verifying the picture containing the three-dimensional code specifically comprises:
acquiring a three-dimensional code image through a camera, identifying a three-dimensional code in the image through a three-dimensional code identification module, analyzing a coded character string identifier of the three-dimensional code and acquiring a three-dimensional code picture P1 after distortion correction;
acquiring a characteristic matrix corresponding to the three-dimensional code through the character string identification, and putting the characteristic matrix into an image reconstruction model for reconstruction to acquire a reconstructed picture P2;
identifying whether the objects in the reconstructed picture P2 and the three-dimensional code picture P1 are the same or not through an image identification model;
if the three-dimensional code image is the same as the image, restoring and displaying the corresponding image material through the feature matrix of the object and the image reconstruction model, and displaying the image material at the client to show that the three-dimensional code image is effective.
9. An off-line recognition device based on three-dimensional code and image recognition, comprising a memory and a processor, wherein the memory stores instructions adapted to be loaded by the processor and execute an off-line recognition method based on three-dimensional code and image recognition according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011171971.4A CN112364675B (en) | 2020-10-28 | 2020-10-28 | Off-line reading method and device based on three-dimensional code and image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011171971.4A CN112364675B (en) | 2020-10-28 | 2020-10-28 | Off-line reading method and device based on three-dimensional code and image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112364675A CN112364675A (en) | 2021-02-12 |
| CN112364675B true CN112364675B (en) | 2022-11-01 |
Family
ID=74511326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011171971.4A Active CN112364675B (en) | 2020-10-28 | 2020-10-28 | Off-line reading method and device based on three-dimensional code and image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112364675B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101276404A (en) * | 2007-03-30 | 2008-10-01 | 李季檩 | System and method for quickly and exactly processing intelligent image |
| CN103106413A (en) * | 2013-02-07 | 2013-05-15 | 华中科技大学 | Autograph two-dimension code representing and authentication method |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10217179B2 (en) * | 2016-10-17 | 2019-02-26 | Facebook, Inc. | System and method for classification and authentication of identification documents using a machine learning based convolutional neural network |
| CN207938065U (en) * | 2018-01-25 | 2018-10-02 | 广州市英泽电子有限公司 | A kind of offline entrance guard device based on Quick Response Code, three-dimension code identification technology |
| CN110147864B (en) * | 2018-11-14 | 2022-02-22 | 腾讯科技（深圳）有限公司 | Method and device for processing coding pattern, storage medium and electronic device |
| CN111401396B (en) * | 2019-01-03 | 2023-04-18 | 阿里巴巴集团控股有限公司 | Image recognition method and device |
| CN110321830B (en) * | 2019-06-28 | 2020-11-13 | 北京邮电大学 | Chinese character string picture OCR recognition method based on neural network |
| KR102108549B1 (en) * | 2019-12-06 | 2020-05-07 | 최원일 | Method and apparatus for distribution management of tile and sanitary ware |
-
2020
- 2020-10-28 CN CN202011171971.4A patent/CN112364675B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101276404A (en) * | 2007-03-30 | 2008-10-01 | 李季檩 | System and method for quickly and exactly processing intelligent image |
| CN103106413A (en) * | 2013-02-07 | 2013-05-15 | 华中科技大学 | Autograph two-dimension code representing and authentication method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112364675A (en) | 2021-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3916627A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| US10635946B2 (en) | Eyeglass positioning method, apparatus and storage medium | |
| US20170243097A1 (en) | Method and apparatus for decoding or generating multi-layer color or code, method for recommending setting parameters in generation of multi-layer or code, and product comprising multi-layer color or code | |
| CN112052781A (en) | Feature extraction model training method, face recognition device, face recognition equipment and medium | |
| CN113283446B (en) | Method and device for identifying object in image, electronic equipment and storage medium | |
| US9489566B2 (en) | Image recognition apparatus and image recognition method for identifying object | |
| CN109948397A (en) | A kind of face image correcting method, system and terminal device | |
| CN109886223B (en) | Face recognition method, bottom library input method and device and electronic equipment | |
| CN111754396A (en) | Face image processing method and device, computer equipment and storage medium | |
| CN111339897B (en) | Living body identification method, living body identification device, computer device, and storage medium | |
| CN110838119A (en) | Human face image quality evaluation method, computer device and computer readable storage medium | |
| Ling et al. | Image quality assessment for free viewpoint video based on mid-level contours feature | |
| CN113449704B (en) | Face recognition model training method and device, electronic equipment and storage medium | |
| CN110766007A (en) | Certificate shielding detection method, device and equipment and readable storage medium | |
| CN112784741A (en) | Pet identity recognition method and device and nonvolatile storage medium | |
| CN112395503A (en) | Face recognition-based sharing platform intelligent recommendation method and system and readable storage medium | |
| CN114758145A (en) | Image desensitization method and device, electronic equipment and storage medium | |
| CN108664906A (en) | The detection method of content in a kind of fire scenario based on convolutional network | |
| CN112364675B (en) | Off-line reading method and device based on three-dimensional code and image recognition | |
| CN112633200A (en) | Human face image comparison method, device, equipment and medium based on artificial intelligence | |
| Shashidhar et al. | Novel framework for optimized digital forensic for mitigating complex image attacks | |
| CN111507119A (en) | Identification code identification method and device, electronic equipment and computer readable storage medium | |
| CN112613457B (en) | Image acquisition mode detection method, device, computer equipment and storage medium | |
| CN115205943A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113627324A (en) | Matching method and device of face images, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |