WO2023284670A1

WO2023284670A1 - Construction method and apparatus for graphic code extraction model, identification method and apparatus, and device and medium

Info

Publication number: WO2023284670A1
Application number: PCT/CN2022/104857
Authority: WO
Inventors: 吴虓杨; 张岳晨; 莫宇; 沈小勇; 吕江波
Original assignee: 深圳思谋信息科技有限公司; 上海思谋科技有限公司
Priority date: 2021-07-12
Filing date: 2022-07-11
Publication date: 2023-01-19
Also published as: CN113468906B; CN113468906A

Abstract

Provided in the present application are a construction method and apparatus for a graphic code extraction model, a graphic code identification method and apparatus, and a computer device and a storage medium. The construction method comprises: acquiring a graphic code original image sample and a material background image sample (S101); obtaining a corresponding graphic code standard image sample according to the graphic code original image sample (S102); obtaining an augmented image sample on the basis of the fusion of the graphic code standard image sample and the material background image sample (S103); and constructing a graphic code extraction model by using the augmented image sample, the graphic code original image sample and the graphic code standard image sample (S104).

Description

Image code extraction model construction method, recognition method, device, equipment and medium

This application claims the priority of the Chinese patent application filed on July 12, 2021 with the application number 2021107853537 and titled "Graphic Code Extraction Model Construction Method, Recognition Method, Device, Equipment and Medium", which is hereby incorporated in its entirety Reference.

technical field

The present application relates to the technical field of artificial intelligence, in particular to a method for constructing a pattern code extraction model, a pattern code recognition method, a device, a computer device and a storage medium.

Background technique

With the development of artificial intelligence technology, there have been technologies that use neural network models to scan and extract graphic codes such as QR codes in various scenarios.

However, when the neural network model used for graphic code extraction in the related art is constructed or trained, due to factors such as the low complexity and limited number of training samples used, it is difficult for the trained model to adapt to the actual application of the graphic code. Changeable and diverse application scenarios lead to low recognition accuracy of the model trained in this way for graphic codes in actual application scenarios.

Contents of the invention

Based on this, it is necessary to provide a method for constructing a pattern code extraction model, a pattern code recognition method, a device, a computer device and a storage medium for the above technical problems.

The first aspect provides a kind of method of building graphic code extraction model, described method comprises:

Obtain the original image sample of the graphics code and the material background image sample;

Obtaining a corresponding standard image sample of the graphic code according to the original image sample of the graphic code;

Obtaining an augmented image sample based on the fusion of the graphic code standard image sample and the material background image sample;

A graphic code extraction model is constructed by using the augmented image sample, the original image sample of the graphic code and the standard image sample of the graphic code.

In some embodiments, the obtaining the corresponding standard image sample of the graphic code according to the original image sample of the graphic code includes:

Generate an initial sample of a standard image of a graphic code according to the graphic code information carried by the original image sample of the graphic code;

Performing morphological transformation processing on the graphic code points of the initial sample of the standard graphic code image to obtain the corresponding standard image sample of the graphic code; wherein, the morphological transformation process includes at least one of expansion, erosion and random noise.

In some embodiments, the number of the material background image samples is multiple; the fusion of the standard image samples based on the graphic code and the material background image samples to obtain the augmented image samples includes:

Contrast-preserving fusion of the graphic code standard image sample and a plurality of the material background image samples to obtain an initial sample of the augmented image;

According to the semantic distance between the initial sample of the augmented image and the original image sample of the graphic code, the initial sample of the augmented image is screened to obtain the augmented image sample.

In some embodiments, the contrast-preserving fusion of the graphic code standard image sample and a plurality of the material background image samples is obtained to obtain the initial sample of the augmented image, including:

Contrast-preserving fusion is performed on the standard image sample of the graphic code with each material background image sample to obtain an initial sample of the augmented image.

The standard image samples of the graphic code are fused with different material background combinations for multiple times to obtain a plurality of initial samples of the augmented image; each of the material background combinations includes two or more material background image samples.

In some embodiments, the constructing a graphic code extraction model by using the augmented image sample, the original image sample of the graphic code and the standard image sample of the graphic code includes:

Inputting the augmented image sample and the original image sample of the graphic code into the graphic code extraction model to be trained;

Obtaining a graphic code extraction result sample output by the graphic code extraction model to be trained for the augmented image sample and the graphic code original image sample;

Using the graphic code extraction result samples and the graphic code standard image samples to perform loss function calculations to obtain loss calculation results;

The graphic code extraction model to be trained is trained by using the loss calculation result to construct a graphic code extraction model.

The second aspect provides a method for identifying a graphic code, the method comprising:

Obtain the graphic code extraction model constructed according to the above-mentioned method;

Inputting the pattern code image to be recognized into the pattern code extraction model;

According to the pattern code recognition result output by the pattern code extraction model for the pattern code image to be recognized, obtain the pattern code candidate area in the pattern code image to be recognized;

Perform graphic code edge fitting processing on the graphic code candidate area to determine the graphic code area in the graphic code image to be recognized.

In some embodiments, the performing graphic code edge fitting processing on the graphic code candidate area, and determining the graphic code area in the graphic code image to be recognized includes:

Extracting at least one layer of outer layer edge points of the graphic code candidate area to obtain an outer layer edge point set;

performing robust regression fitting on the outer layer edge point set to obtain an edge fitting result of the graphic code region in the graphic code image to be recognized;

According to the edge fitting result of the pattern code region, the pattern code region in the pattern code image to be recognized is determined.

In some embodiments, the extracting at least one layer of outer layer edge points of the graphic code candidate area to obtain the outer layer edge point set includes:

Applying an edge extraction algorithm to extract edge points from the graphic code candidate area;

Extracting at least one layer of edge points from the outside of the pattern code candidate region to the interior of the pattern code candidate region as an outer layer edge point set.

In some embodiments, the method also includes:

After the graphic code area is located, the image data corresponding to the graphic code area is output to the graphic code decoding module, so that the graphic code decoding module can complete the decoding process.

The third aspect provides a device for constructing a graphic code extraction model, including:

The sample acquisition module is configured to perform acquisition of graphic code original image samples and material background image samples;

The sample generation module is configured to obtain a corresponding standard image sample of the graphic code according to the original image sample of the graphic code;

The sample fusion module is configured to perform fusion based on the graphic code standard image sample and the material background image sample to obtain an augmented image sample;

A model building module configured to construct a graphic code extraction model by using the augmented image samples, the original image samples of the graphic code, and the standard image samples of the graphic code.

In some embodiments, the sample generation module is configured to execute:

Generate an initial sample of a standard image of a graphic code according to the graphic code information carried by the original image sample of the graphic code; and,

In some embodiments, the number of material background image samples is multiple; the sample fusion module is configured to execute:

Contrast-preserving fusion of the graphic code standard image sample and a plurality of the material background image samples to obtain an initial sample of the augmented image; and,

In some embodiments, the sample fusion module is configured to perform:

Contrast-preserving fusion is performed on the standard image sample of the graphic code and each of the material background image samples to obtain an initial sample of the augmented image.

The fourth aspect provides a graphic code recognition device, including:

The model acquisition module is configured to perform the acquisition of the graphic code extraction model constructed according to the method described above;

An image input module configured to input the graphic code image to be recognized into the graphic code extraction model;

The coarse positioning module is configured to execute the pattern code recognition result output for the pattern code image to be recognized according to the pattern code extraction model, and obtain the pattern code candidate area in the pattern code image to be recognized;

The fine positioning module is configured to perform graphic code edge fitting processing on the graphic code candidate area, and determine the graphic code area in the graphic code image to be recognized.

In some embodiments, the fine positioning module is configured to perform:

performing robust regression fitting on the outer edge point set to obtain an edge fitting result of the graphic code region in the graphic code image to be recognized; and,

In some embodiments, the fine positioning module is configured to perform, including:

A fifth aspect provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Obtain the original image sample of the graphic code and the background image sample of the material; obtain the standard image sample of the corresponding graphic code according to the original image sample of the graphic code; based on the fusion of the standard image sample of the graphic code and the background image sample of the material, obtain the widening image samples; using the augmented image samples, the original image samples of the graphic codes and the standard image samples of the graphic codes to construct a graphic code extraction model.

A sixth aspect provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Obtaining the pattern code extraction model constructed according to the above-mentioned method; inputting the pattern code image to be recognized into the pattern code extraction model; according to the pattern code extraction model output pattern code recognition for the pattern code image to be recognized As a result, the graphic code candidate area in the graphic code image to be recognized is obtained; the graphic code edge fitting process is performed on the graphic code candidate area, and the graphic code area in the graphic code image to be recognized is determined.

A seventh aspect provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

The eighth aspect provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

A ninth aspect provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method described in any one of the above-mentioned first aspects, or the steps of the method described in any one of the above-mentioned second aspects are implemented. step.

The above scheme can integrate standard graphic codes with various material backgrounds in the model training stage to form augmented image samples with various material background styles, and combine augmented image samples with various material background styles with original image samples of graphic codes and standard graphic codes. The image samples are used together as model training data for model training, and a graphic code extraction model suitable for various application scenarios can be constructed to improve the accuracy of the model's recognition of graphic codes.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features, objects and advantages of the present application will be apparent from the description, drawings and claims.

Description of drawings

For a better description and illustration of those embodiments and examples disclosed herein, reference may be made to one or more of the accompanying drawings. Additional details or examples used to describe the drawings should not be considered limitations on the scope of any of the disclosed inventions, the presently described embodiments and/or examples, and the best mode of these inventions currently understood.

Fig. 1 is the schematic flow chart of the method for constructing graphic code extraction model in an embodiment;

Fig. 2 is a schematic flow chart of a pattern code recognition method in an embodiment;

Fig. 3 is a schematic structural diagram of a device for constructing a graphic code extraction model in an embodiment;

Fig. 4 is a schematic structural diagram of a pattern code recognition device in an embodiment;

Fig. 5 is a schematic diagram of the internal structure of a computer device in an embodiment;

Fig. 6 is a schematic diagram of the internal structure of a computer device in an embodiment.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

The following part first introduces the method for constructing the pattern code extraction model provided by this application, and then introduces the pattern code recognition method provided by this application.

Wherein, the method for constructing the graphic code extraction model provided by the present application can be mainly executed by the server, for example, the server obtains the original image sample of the graphic code and the material background image sample, and the server obtains the corresponding standard image of the graphic code according to the original image sample of the graphic code Sample; the server obtains an augmented image sample based on the fusion of the aforementioned graphic code standard image sample and the aforementioned material background image sample; the server uses the aforementioned augmented image sample, graphic code original image sample, and aforementioned graphic code standard image sample to construct a graphic code extraction Model. The server can be implemented by an independent server or a server cluster composed of multiple servers.

The graphic code recognition method provided in the present application can be mainly executed by the terminal, for example, the terminal obtains the graphic code extraction model constructed according to the above-mentioned method of constructing the graphic code extraction model, and inputs the image of the graphic code to be recognized into the graphic code extraction model ; The terminal obtains the graphic code candidate area in the graphic code image to be recognized according to the graphic code recognition result output by the graphic code extraction model for the graphic code image to be recognized; the terminal performs graphic coding on the graphic code candidate area The edge fitting process determines the graphic code area in the image of the graphic code to be recognized. The terminal can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices.

In one embodiment, as shown in FIG. 1 , a method for constructing a graphic code extraction model is provided. The method is applied to a server as an example for illustration. It can be understood that when the method is applied to a terminal, the execution of corresponding steps The subject is changed to a terminal. The method may include the steps of:

Step S101, the server obtains the original image sample of the graphic code and the sample of the material background image;

Among them, in the model construction stage of this application, the description of related features such as images will be uniformly described by such as image samples, so as to distinguish it from the model application, that is, the graphic code recognition stage. In this step, for the original image sample of the graphic code and the sample of the background image of the material, the server can obtain it through data collection and web crawler technology. A background image of multiple materials, which may include various materials that can be used to print graphic codes, such as paper of various colors, various wooden and plastic materials, etc., and may further include various lighting conditions ( Such materials under different light and dark conditions) to cover the possible application scenarios of graphic codes as much as possible.

Step S102, the server obtains the corresponding standard image sample of the graphic code according to the original image sample of the graphic code;

In this step, the server mainly generates the corresponding graphic code standard image sample according to the graphic code information contained in the original image sample of the graphic code, that is, the original image sample is converted to a certain extent on the basis of basically retaining the original graphic code information, so that Form a new graphic code image for subsequent fusion processing. This new graphic code image is called a graphic code standard image sample. This transformation can be, for example, changing the position, direction or shape of the graphic code on the image, etc., which can be implemented in some In the example, it is used to enrich or standardize the shape of the graphic code on the image.

In one embodiment, this step S102 may include: the server generates the initial sample of the standard image of the graphic code according to the graphic code information carried by the original image sample of the graphic code; the server performs morphological transformation processing on the graphic code points of the initial sample of the standard image of the graphic code , to obtain the standard image sample of the graphic code.

In this embodiment, the server can first generate a corresponding initial sample of a standard image of a graphic code according to the graphic code information carried by the original image sample of the graphic code. Specifically, the graphic code information (text information) carried by the original image sample of the graphic code can be extracted first, and then Through the code system of graphic codes such as two-dimensional codes, the text information is changed into standard graphic code picture information, and then a quadrilateral area is set to perform affine transformation on it, so as to obtain the initial sample of the standard image of the graphic code.

On this basis, in order to enrich the shape of the graphic code so that the model can adapt to more diverse and changeable application scenarios, the server further performs morphological transformation processing on the graphic code points of the initial sample of the standard image of the graphic code to obtain the standard image sample of the graphic code , the morphological transformation process includes at least one of dilation, erosion or random noise. Specifically, the probability P can be set to perform random morphological transformation (expansion, erosion, random noise, etc.) on the shape of the graphic code points in the initial sample of the graphic code standard image to obtain the standard image sample of the graphic code.

Step S103, the server obtains an augmented image sample based on the fusion of the standard image sample of the graphic code and the background image sample of the material; the augmented image sample represents an image sample obtained by fusing the standard image sample of the graphic code and the background image sample of the material.

In this step, the server may use the material background image sample as the background of the graphic code standard image sample, and fuse the graphic code standard image sample into the material background image sample to obtain an augmented image sample. In a specific application, the number of material background image samples may be multiple, and for this, in one embodiment, step S103 may include:

The server performs contrast-preserving fusion of the graphic code standard image sample and multiple material background image samples to obtain the initial sample of the augmented image; the server performs an initial augmented image sample according to the semantic distance between the initial sample of the augmented image and the original image sample of the graphic code. Filter to obtain augmented image samples.

Among them, the contrast preserving fusion (histogram preserving blending) is an algorithm for performing contrast preserving fusion on two pictures by calculating the color distribution of the pictures. In this embodiment, the server performs contrast-preserving fusion of the graphic code standard image sample and multiple material background image samples, which may include performing contrast-preserving fusion of the graphic code standard image sample with each material background image sample, and may also include graphic Contrast-preserving fusion of code standard image samples with two or more material background image samples at the same time, and multiple fusions with different material background combinations (that is, the combination of two or more material background image samples), so as to enrich the samples form to obtain an initial sample of the augmented image, and the number of initial samples of the augmented image is also multiple.

Next, the server needs to select qualified augmented image initial samples from the augmented image initial samples as augmented image samples. The server in this embodiment can filter according to the semantic distance (D(I_AUG)-D(I_DM)) between the obtained augmented image initial sample (I_AUG) and the graphic code original image sample (I_DM). Exemplarily, if the semantic distance is lower than or equal to the semantic distance threshold, the server can filter the initial sample of the augmented image as the augmented image sample.

In step S104, the server uses the augmented image sample, the original image sample of the graphic code, and the standard image sample of the graphic code to construct a graphic code extraction model.

In this step, for the image code extraction model, ImageNet pre-trained ResNet34 can be used as the backbone network to form a decoder through skip-connection between layers. Specifically, the augmented image sample and the original image sample of the graphic code can be used as input data for model training, and the standard image sample of graphic code can be used as label data for model training, and a graphic code extraction model can be constructed using the input data and label data. Wherein, the graphic code standard image sample is mainly generated according to the graphic code information carried by the graphic code original image sample, and may be referred to as the graphic code standard image initial sample in the above embodiments, that is, the graphic code standard without morphological transformation Image samples.

In one embodiment, step S104 specifically includes: the server inputs the augmented image sample and the original image sample of the graphic code into the graphic code extraction model to be trained, and obtains the image code extraction model to be trained for the augmented image sample and the original image of the graphic code The sample output graphic code extraction result sample; the server uses the graphic code extraction result sample and the graphic code standard image sample to perform loss function calculation to obtain the loss calculation result; the server uses the loss calculation result to train the graphic code extraction model to be trained, and constructs the obtained graphic code extraction model.

In the training process of this embodiment, the augmented image sample and the original image sample of the graphic code as the input data for model training will be input into the graphic code extraction model to be trained, and the graphic code extraction model to be trained is aimed at the augmented image Samples and graphic codes The original image samples output corresponding graphic code extraction result samples. The graphic code standard image sample used as the label data for model training will be used to perform loss function calculation with the graphic code extraction result sample to obtain the loss calculation result or called the loss function calculation result. Then the loss calculation result can be used to adjust the model network parameters of the graphic code extraction model to be trained, so as to train the graphic code extraction model to be trained, and construct the graphic code extraction model.

In a specific application, after the server inputs the augmented image sample and the original image sample of the graphic code to the graphic code extraction model to be trained, the graphic code part in the graphic code extraction result sample output by the graphic code extraction model and the graphic code The graphic code part in the standard image sample is calculated by the loss function, and the loss weight is added to the graphic code quiet zone (quiet zone), and the graphic code extraction model is constructed after a certain iteration. For example, for a QR code, the graphic code quiet zone (quiet zone) can be a blank frame outside the QR code to ensure that the scanning device correctly recognizes the QR code. If there is no such frame, the QR code reader will be Due to the interference of external factors, it is impossible to determine what the QR code contains and does not contain. For barcodes, the graphic code quiet zone (quiet zone) can be a blank boundary located on one side of the barcode, which is used to ensure that the scanning device correctly recognizes the end mark of the barcode, and avoids obtaining information irrelevant to the barcode.

In the above method of constructing a graphic code extraction model, the server obtains the original image sample of the graphic code and the material background image sample, and then the server obtains the corresponding standard image sample of the graphic code according to the original image sample of the graphic code, and then the server obtains the corresponding standard image sample of the graphic code based on the standard image sample of the graphic code and the material background image sample. The image samples are fused to obtain augmented image samples, and the image code extraction model is constructed by using the augmented image samples, the original image samples of the graphic code and the standard image samples of the graphic code. This solution can integrate standard graphic codes with various material backgrounds in the model training stage to form augmented image samples with various material background styles, and combine augmented image samples with various material background styles, original image samples of graphic codes, and graphic codes Standard image samples are also used as model training data for model training, which can construct graphic code extraction models suitable for various application scenarios, and improve the accuracy of the model for graphic code recognition.

In one embodiment, as shown in FIG. 2 , a pattern code recognition method is provided, and the method is applied to a terminal as an example for illustration. It can be understood that when the method is applied to a server, the subject of execution of the corresponding steps will be changed. for the server. The method may include the steps of:

Step S201, the terminal obtains the graphic code extraction model constructed according to the method described in the above embodiment;

Step S202, the terminal inputs the graphic code image to be recognized into the graphic code extraction model;

In this step, after obtaining the image of the graphic code to be recognized, the image of the graphic code to be recognized can be scaled to the set scale space, and basic preprocessing can also be performed, such as a series of grayscale, sharpening, and denoising. Preprocessing operations before input into the pattern code extraction model.

Step S203, the terminal obtains the graphic code candidate area in the graphic code image to be recognized according to the graphic code recognition result output by the graphic code extraction model for the graphic code image to be recognized.

This step mainly uses the graphic code extraction model to roughly locate the area where the graphic code in the image of the graphic code to be recognized is located. Among them, after the graphic code image to be recognized is input to the graphic code extraction model, the graphic code extraction model will output the corresponding graphic code recognition result, so that the terminal can obtain the graphic code candidate area in the graphic code image to be recognized according to the graphic code recognition result, which is about to The region where the pattern code is located by the pattern code extraction model on the pattern code image to be recognized is used as the pattern code candidate region.

Step S204, the terminal performs graphic code edge fitting processing on the graphic code candidate area, and determines the graphic code area in the graphic code image to be recognized.

In this step, on the basis of rough positioning of the graphic code extraction model, an edge fitting process is performed on the graphic code candidate area to accurately locate the graphic code area in the graphic code image to be recognized.

In one embodiment, step S204 specifically includes: the terminal extracts at least one layer of outer edge points of the graphic code candidate area to obtain the outer edge point set; then the terminal performs robust regression fitting on the outer edge point set to obtain the pattern to be recognized The edge fitting result of the graphic code area in the code image; then the terminal determines the graphic code area in the graphic code image to be recognized according to the edge fitting result of the graphic code area.

This embodiment is mainly to extract at least one layer of outer layer edge points from the graphic code candidate area roughly positioned by the graphic code extraction model to form an outer layer edge point set, and then perform a robust regression fitting process on the outer layer edge point set so that according to the graphic code area edge The fitting result determines the edge of the graphic code area, and precisely locates the graphic code area in the graphic code image to be recognized according to the edge. Among them, the robust regression fitting refers to the statistical robust regression theory, such as Theil-Sen estimation to complete the fitting of the edges of the graphic code area. In specific applications, for the extraction of outer layer edge points, an edge extraction algorithm can be applied, such as the canny operator to extract edge points from the graphic code candidate area, and extract such as a layer from the outside of the graphic code candidate area to the graphic code candidate area. Edge points (corresponding to the outer layer edge points), as the outer layer edge point set of the area boundary of the fitting graphic code, can then use Theil-Sen estimation to fit the outer layer edge point set to obtain the graphic code area edge fitting result, according to This locates the graphic code area in the graphic code image to be recognized. After the graphic code area is located, the image data corresponding to the graphic code area can be output to the graphic code decoding module for the graphic code decoding module to complete the decoding process.

The above pattern code recognition method, based on the rough positioning of the model, uses the statistical method of edge extraction and robust regression to realize the high-precision optimization of the edge of the rough positioning of the pattern code candidate area, improve the accuracy of pattern code recognition, and can adapt to continuous changes It has strong robustness and is conducive to improving the readability and decoding rate of graphic codes such as two-dimensional codes (QR, DM codes).

It should be understood that although the various steps in the above flow chart are displayed in sequence according to the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the above flowchart may include multiple steps or multiple stages, these steps or stages are not necessarily executed at the same time, but may be executed at different times, the execution order of these steps or stages It does not necessarily have to be performed sequentially, but can be performed alternately or alternately with other steps or at least a part of steps or stages in other steps.

In one embodiment, as shown in FIG. 3 , a device for constructing a graphic code extraction model is provided, and the device 300 can be applied to a server. The device 300 may include:

The sample acquisition module 301 is configured to perform acquisition of graphic code original image samples and material background image samples;

The sample generating module 302 is configured to obtain a corresponding standard image sample of the graphic code according to the original image sample of the graphic code;

The sample fusion module 303 is configured to perform fusion based on the graphic code standard image sample and the material background image sample to obtain an augmented image sample;

The model building module 304 is configured to execute using the augmented image sample, the original image sample of the graphic code and the standard image sample of the graphic code to construct a graphic code extraction model.

In one embodiment, the sample generating module 302 is configured to generate an initial sample of a standard image of a graphic code according to the graphic code information carried by the original image sample of the graphic code; Performing morphological transformation processing to obtain the standard image sample of the graphic code; the morphological transformation processing includes at least one of expansion, erosion or random noise.

In one embodiment, the number of material background image samples is multiple; the sample fusion module 303 is configured to perform contrast-preserving fusion of the graphic code standard image sample and multiple material background image samples to obtain an augmented An initial sample of the image: according to the semantic distance between the initial sample of the augmented image and the original image sample of the graphic code, the initial sample of the augmented image is screened to obtain the sample of the augmented image.

In one embodiment, the model building module 304 is configured to input the augmented image sample and the original image sample of the graphic code into the graphic code extraction model to be trained, and obtain the graphic code extraction model to be trained for the The graphic code extraction result sample output by the augmented image sample and the original image sample of the graphic code; use the graphic code extraction result sample and the graphic code standard image sample to perform loss function calculation to obtain the loss calculation result; use the loss calculation result to calculate the The graphic code extraction model to be trained is trained, and the graphic code extraction model is constructed.

In one embodiment, as shown in FIG. 4 , a pattern code recognition device is provided, and the device 400 can be applied to a terminal. The device 400 may include:

The model acquisition module 401 is configured to perform acquisition of the graphic code extraction model constructed according to the method described above;

The image input module 402 is configured to input the graphic code image to be recognized into the graphic code extraction model;

The coarse positioning module 403 is configured to execute the pattern code recognition result output for the pattern code image to be recognized according to the pattern code extraction model, and obtain the pattern code candidate area in the pattern code image to be recognized;

The fine positioning module 404 is configured to perform pattern code edge fitting processing on the pattern code candidate region, and determine the pattern code region in the pattern code image to be recognized.

In one embodiment, the fine positioning module 404 is configured to extract at least one layer of outer edge points of the graphic code candidate area to obtain an outer edge point set; perform robust regression simulation on the outer edge point set According to the edge fitting result of the pattern code region, the pattern code region in the pattern code image to be recognized is determined.

For the specific limitations of the device for constructing the pattern code extraction model and the pattern code recognition device, please refer to the limitations of the method for constructing the pattern code extraction model and the pattern code recognition method above respectively, and will not repeat them here. Each module in the above-mentioned device for constructing a pattern code extraction model and the pattern code recognition device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure may be shown in FIG. 5 . The computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs and databases. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store image sample data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for constructing a pattern code extraction model is realized.

In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in FIG. 6 . The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, an operator network, NFC (Near Field Communication) or other technologies. When the computer program is executed by the processor, a pattern code recognition method is realized. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad, or mouse.

Those skilled in the art can understand that the structures shown in Figures 5 and 6 are only block diagrams of partial structures related to the solution of this application, and do not constitute a limitation to the computer equipment on which the solution of this application is applied. The specific computer Devices may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In one embodiment, there is also provided a computer device, including a memory and a processor, where a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

Based on this, in one embodiment, a kind of computer program product is provided, comprises computer program, and when this computer program is executed by processor, realizes the step: acquire graphic code original image sample and material background image sample; According to described graphic code original image sample, to obtain the corresponding graphic code standard image sample; based on the fusion of the graphic code standard image sample and the material background image sample, an augmented image sample is obtained; using the augmented image sample, the graphic code original image sample As well as the standard image samples of the graphic code, a graphic code extraction model is constructed.

In another embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of: obtaining the graphic code extraction model constructed according to the above-mentioned method; Input to the pattern code extraction model; according to the pattern code recognition result output by the pattern code extraction model for the pattern code image to be recognized, obtain the pattern code candidate area in the pattern code image to be recognized; for the pattern The code candidate area is subjected to graphic code edge fitting processing to determine the graphic code area in the graphic code image to be recognized.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only represent several implementation modes of the present application, and the description thereof is relatively specific and detailed, but it should not be construed as limiting the scope of the patent for the invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims

A method for constructing a pattern code extraction model, characterized in that it comprises:

Obtain the original image sample of the graphics code and the material background image sample;

Obtaining a corresponding standard image sample of the graphic code according to the original image sample of the graphic code;

Obtaining an augmented image sample based on the fusion of the graphic code standard image sample and the material background image sample;

A graphic code extraction model is constructed by using the augmented image sample, the original image sample of the graphic code and the standard image sample of the graphic code.
The method according to claim 1, wherein said obtaining a corresponding graphic code standard image sample according to said graphic code original image sample comprises:

Generate an initial sample of a standard image of a graphic code according to the graphic code information carried by the original image sample of the graphic code;

Performing morphological transformation processing on the graphic code points of the initial sample of the standard graphic code image to obtain the corresponding standard image sample of the graphic code; wherein, the morphological transformation process includes at least one of expansion, erosion and random noise.
The method according to claim 1, wherein the quantity of the material background image samples is multiple; the fusion of the standard image samples based on the graphic code and the material background image samples obtains the augmented image samples ,include:

Contrast-preserving fusion of the graphic code standard image sample and a plurality of the material background image samples to obtain an initial sample of the augmented image;

According to the semantic distance between the initial sample of the augmented image and the original image sample of the graphic code, the initial sample of the augmented image is screened to obtain the augmented image sample.
The method according to claim 3, wherein said performing contrast-preserving fusion of said graphic code standard image sample and a plurality of said material background image samples to obtain an initial sample of an augmented image, comprising:

Contrast-preserving fusion is performed on the standard image sample of the graphic code and each of the material background image samples to obtain an initial sample of the augmented image.
The method according to claim 3, wherein said performing contrast-preserving fusion of said graphic code standard image sample and a plurality of said material background image samples to obtain an initial sample of an augmented image, comprising:

Multiple fusions of the graphic code standard image samples with different material background combinations respectively, to obtain a plurality of augmented image initial samples; wherein, each of the material background combinations includes two or more material background image samples .
The method according to any one of claims 1 to 5, wherein said use of said augmented image sample, said graphic code original image sample and said graphic code standard image sample to construct a graphic code extraction model, include:

Inputting the augmented image sample and the original image sample of the graphic code into the graphic code extraction model to be trained;

Obtaining a graphic code extraction result sample output by the graphic code extraction model to be trained for the augmented image sample and the graphic code original image sample;

Using the graphic code extraction result samples and the graphic code standard image samples to perform loss function calculations to obtain loss calculation results;

The graphic code extraction model to be trained is trained by using the loss calculation result to construct a graphic code extraction model.
A pattern code recognition method, characterized in that, comprising:

Obtaining a graphic code extraction model constructed according to the method according to any one of claims 1 to 6;

Inputting the pattern code image to be recognized into the pattern code extraction model;

According to the pattern code recognition result output by the pattern code extraction model for the pattern code image to be recognized, the pattern code candidate area in the pattern code image to be recognized is obtained;

Perform graphic code edge fitting processing on the graphic code candidate area to determine the graphic code area in the graphic code image to be recognized.
The method according to claim 7, wherein said performing graphic code edge fitting processing on said graphic code candidate area, and determining the graphic code area in said graphic code image to be recognized comprises:

Extracting at least one layer of outer layer edge points of the graphic code candidate area to obtain an outer layer edge point set;

performing robust regression fitting on the outer layer edge point set to obtain an edge fitting result of the graphic code region in the graphic code image to be recognized;

According to the edge fitting result of the pattern code region, the pattern code region in the pattern code image to be recognized is determined.
The method according to claim 8, wherein said extracting at least one layer of outer layer edge points of said graphic code candidate area to obtain an outer layer edge point set comprises:

Applying an edge extraction algorithm to extract edge points from the graphic code candidate area;

Extracting at least one layer of edge points from the outside of the pattern code candidate region to the interior of the pattern code candidate region as an outer layer edge point set.
The method according to claim 7, wherein the method further comprises:

After the graphic code area is located, the image data corresponding to the graphic code area is output to the graphic code decoding module, so that the graphic code decoding module can complete the decoding process.
A device for constructing a graphic code extraction model, characterized in that it comprises:

The sample acquisition module is configured to perform acquisition of graphic code original image samples and material background image samples;

The sample generation module is configured to obtain a corresponding standard image sample of the graphic code according to the original image sample of the graphic code;

The sample fusion module is configured to perform fusion based on the graphic code standard image sample and the material background image sample to obtain an augmented image sample;

A model building module configured to construct a graphic code extraction model by using the augmented image samples, the original image samples of the graphic code, and the standard image samples of the graphic code.
The device according to claim 11, wherein the sample generation module is configured to execute:

Generate an initial sample of a standard image of a graphic code according to the graphic code information carried by the original image sample of the graphic code; and,

Performing morphological transformation processing on the graphic code points of the initial sample of the standard graphic code image to obtain the corresponding standard image sample of the graphic code; wherein, the morphological transformation process includes at least one of expansion, erosion and random noise.
The device according to claim 11, wherein the quantity of the material background image samples is multiple; the sample fusion module is configured to execute:

Contrast-preserving fusion of the graphic code standard image sample and a plurality of the material background image samples to obtain an initial sample of the augmented image; and,

According to the semantic distance between the initial sample of the augmented image and the original image sample of the graphic code, the initial sample of the augmented image is screened to obtain the augmented image sample.
The device according to claim 13, characterized in that, the sample fusion module is configured to execute:

Contrast-preserving fusion is performed on the standard image sample of the graphic code and each of the material background image samples to obtain an initial sample of the augmented image.
A graphic code recognition device is characterized in that it comprises:

A model acquisition module configured to perform acquisition according to the graphic code extraction model constructed by the method according to any one of claims 1 to 6;

An image input module configured to input the graphic code image to be recognized into the graphic code extraction model;

The coarse positioning module is configured to execute the pattern code recognition result output for the pattern code image to be recognized according to the pattern code extraction model, and obtain the pattern code candidate area in the pattern code image to be recognized;

The fine positioning module is configured to perform graphic code edge fitting processing on the graphic code candidate area, and determine the graphic code area in the graphic code image to be recognized.
The device according to claim 15, wherein the fine positioning module is configured to perform:

Extracting at least one layer of outer layer edge points of the graphic code candidate area to obtain an outer layer edge point set;

performing robust regression fitting on the outer edge point set to obtain an edge fitting result of the graphic code region in the graphic code image to be recognized; and,

According to the edge fitting result of the pattern code region, the pattern code region in the pattern code image to be recognized is determined.
The device according to claim 16, wherein the fine positioning module is configured to perform:

Applying an edge extraction algorithm to extract edge points from the graphic code candidate area; and,

Extracting at least one layer of edge points from the outside of the pattern code candidate region to the interior of the pattern code candidate region as an outer layer edge point set.
A computer device, comprising a memory and a processor, the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 10 when executing the computer program.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are realized.
A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.