CN112651256A - Two-dimensional code identification method and device, computer equipment and storage medium - Google Patents
Two-dimensional code identification method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112651256A CN112651256A CN201910969469.9A CN201910969469A CN112651256A CN 112651256 A CN112651256 A CN 112651256A CN 201910969469 A CN201910969469 A CN 201910969469A CN 112651256 A CN112651256 A CN 112651256A
- Authority
- CN
- China
- Prior art keywords
- image
- dimensional code
- decoded
- initial
- contour
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
- G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
- G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
- G06K7/1404—Methods for optical code recognition
- G06K7/1439—Methods for optical code recognition including a method step for retrieval of the optical code
- G06K7/1452—Methods for optical code recognition including a method step for retrieval of the optical code detecting bar code edges
Abstract
The invention relates to the field of image processing, and discloses a two-dimensional code identification method, a device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring an initial image; processing the initial image according to a first preprocessing method to obtain the contour information of the initial image; screening a target two-dimensional code outline from the outline information according to a preset screening method; extracting a two-dimensional code area image from the initial image according to the outline of the target two-dimensional code; and processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information. The scheme provided by the invention can improve the recognition rate of the two-dimensional code image so as to meet the recognition requirements in application scenes with large recognition range, high-speed dynamic recognition and high reliability.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to a two-dimensional code recognition method and apparatus, a computer device, and a storage medium.
Background
The two-dimensional code is a geometric figure which converts data symbol information into a two-dimensional plane according to a certain rule, has the advantages of high capacity, high decoding reliability, strong error correction capability and the like, and has a very wide application range.
The two-dimension code can be divided into daily two-dimension codes and industrial two-dimension codes according to the actual application of the two-dimension codes. The daily two-dimensional code refers to a two-dimensional code used in scenes such as payment and identity recognition which are common in life. Daily two-dimensional code is when using, goes to search for the two-dimensional code through the mobile device initiative to make the two-dimensional code be in effective range, even there is the discernment mistake and also can not have big influence, owing to can go on repeatedly, it is not high to discernment scope, speed, reliability requirement. The identification of the industrial two-dimensional code has high requirements on identification equipment, and the identification equipment has the characteristics of quick automatic identification, suitability for complex working environments, high reliability and the like.
At present, a large amount of work such as incoming material identification, label authentication, material classification and the like is carried out by adopting two-dimensional codes in the field of industrial automation. But still has the problem for some specific application scenarios that need to satisfy a large recognition range, high-speed dynamic recognition and high reliability at the same time. In practical application, a plurality of two-dimensional codes on a workpiece in a wide-format production line may be randomly distributed in the whole format, so that the requirement of identifying a plurality of randomly distributed small two-dimensional codes in a large view field range (more than 1m) needs to be met; meanwhile, in order to improve the production efficiency, the running speed of an automatic production line is higher and higher (higher than 60m/min), and higher requirements are put forward on the identification speed of the two-dimensional code; and the application occasion of industrial two-dimensional code identification has very high requirement on reliability, and the condition of missing detection or error detection is not allowed (the missing detection may cause that the current workpiece is not processed to cause disorder of the subsequent production flow, and the error detection may cause that the workpiece is not processed according to a normal program or is classified to be disordered, and the like, which all cause great loss).
Therefore, a two-dimensional code identification method is needed to be found to meet the identification requirements in the application scene of large identification range, high-speed dynamic identification and high reliability, and to eliminate missing detection and error detection.
Disclosure of Invention
In view of the above, it is necessary to provide a two-dimensional code recognition method, device, computer device and storage medium to improve the recognition capability of the recognition system in a large recognition range and high-speed dynamic state and improve the accuracy of the recognition result.
A two-dimensional code identification method comprises the following steps:
acquiring an initial image;
processing the initial image according to a first preprocessing method to obtain the contour information of the initial image;
screening a target two-dimensional code outline from the outline information according to a preset screening method;
extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
A two-dimensional code recognition device includes:
the image acquisition module is used for acquiring an initial image;
the contour acquisition module is used for processing the initial image according to a first preprocessing method to acquire contour information of the initial image;
the contour screening module is used for screening a target two-dimensional code contour from the contour information according to a preset screening method;
the image extraction module is used for extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and the image decoding module is used for processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the two-dimensional code recognition method when executing the computer program.
A computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the above-described two-dimensional code recognition method.
According to the two-dimensional code identification method, the two-dimensional code identification device, the computer equipment and the storage medium, the image to be processed is obtained by obtaining the initial image. And processing the initial image according to a first preprocessing method to obtain the contour information of the initial image so as to convert the initial image into a contour image and reduce the processing amount of image data. And screening the outline of the target two-dimensional code from the outline information according to a preset screening method so as to accurately position the area of the two-dimensional code. And extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline so as to obtain a local image needing to be decoded. And processing the two-dimensional code area image according to a second preprocessing method to obtain an image to be decoded, decoding the image to be decoded to obtain two-dimensional code information so as to optimize the image quality of the two-dimensional code area image and improve the identification speed. The scheme provided by the invention can improve the recognition rate of the two-dimensional code image so as to meet the recognition requirements in application scenes with large recognition range, high-speed dynamic recognition and high reliability.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
Fig. 1 is a schematic flow chart of a two-dimensional code recognition method according to an embodiment of the invention;
FIG. 2 is a plurality of images generated during processing by a two-dimensional code recognition method according to an embodiment of the present invention;
fig. 3 is a flow chart illustrating a two-dimensional code recognition method according to an embodiment of the invention;
FIG. 4 is a schematic diagram of a two-dimensional code recognition applied to an automated pipeline according to an embodiment of the present invention;
fig. 5 is a flow chart illustrating a two-dimensional code recognition method according to an embodiment of the invention;
fig. 6 is a flow chart illustrating a two-dimensional code recognition method according to an embodiment of the invention;
fig. 7 is a flow chart illustrating a two-dimensional code recognition method according to an embodiment of the invention;
fig. 8 is a flow chart illustrating a two-dimensional code recognition method according to an embodiment of the invention;
fig. 9 is a flow chart illustrating a two-dimensional code recognition method according to an embodiment of the invention;
fig. 10 is a schematic structural diagram of a two-dimensional code recognition apparatus according to an embodiment of the invention;
FIG. 11 is a diagram of a computing device in accordance with an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In an embodiment, as shown in fig. 1, a two-dimensional code identification method is provided, which includes the following steps:
s10, acquiring an initial image;
s20, processing the initial image according to a first preprocessing method to obtain the outline information of the initial image;
s30, screening a target two-dimensional code outline from the outline information according to a preset screening method;
s40, extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and S50, processing the two-dimensional code area image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
In this embodiment, the brightness of the environment can be set according to the preset environment parameters, and the field range of the shooting device can be set as required. And after the setting is finished, acquiring an initial image of the shot object through the shooting device. Therefore, the quality of the obtained initial image is higher, and the identification rate of the two-dimensional code in the initial image is improved. In a specific application example, the two-dimensional code recognition method is applied to automatic recognition on a production line, the recognition effective field range is 1.3m by 1m, and the moving speed of the shot product or material is 1 m/s.
The first preprocessing method includes several processing steps that can process the initial image into a contour image to generate contour information. The processing steps involved here include, but are not limited to, sharpening, adaptive binarization, median filtering, morphological on-operations. The contour information may include a plurality of contour data. Each contour data may be represented using a series of points in the image that appear as a closed trajectory in the contour image. Specifically, a profile data may be expressed as: [ (32, 32), (33, 32), (34, 32), (35, 32), (35, 33), …. ]. Specifically, in fig. 2, fig. 2-1 is an initial image, fig. 2-2 is an image of fig. 2-1 after sharpening and adaptive binarization processing, and fig. 2-3 is an image of fig. 2-2 after median filtering and morphological on operation processing.
The preset screening method is used for eliminating the outlines which are not similar to the outline (two-dimensional code outline) shape to be identified and the color arrangement of which does not meet the specification (color specification of the two-dimensional code). In the preset screening method, any one or more of parameters such as the length-width ratio of the contour, the area of the contour, the ratio of black and white pixels, and the distance between the edge and the central point of the contour may be defined to obtain one or more defined ranges. And then screening all the profile data in the profile information according to the limited ranges, and selecting a proper target two-dimensional code profile. If the initial image contains more than one two-dimensional code, the number of the target two-dimensional code outlines may be more than one.
After the target two-dimensional code outline is determined, a two-dimensional code area image can be extracted from the initial image according to the coordinate data of each edge point contained in the target two-dimensional code outline. In fig. 2, through screening, an image containing the outline of the target two-dimensional code is selected from fig. 2-3, namely fig. 2-4. And then, cutting the images in the figures 2-4 to obtain target two-dimensional code area images, namely the images in the figures 2-5.
The second preprocessing method comprises a plurality of image optimization steps so as to improve the recognition rate of the two-dimensional code. Wherein, the image optimization step includes but is not limited to contrast enhancement and binarization. And obtaining the optimized image to be decoded after the optimization by the second preprocessing method. The two-dimensional code information contained in the image to be decoded can be analyzed by selecting an applicable standard decoding method. In some embodiments, standard decoding methods that may be used include QRcode decoding, DataMatrix decoding, and the like.
In steps S10-S60, an initial image is acquired to obtain an image to be processed. And processing the initial image according to a first preprocessing method to obtain the contour information of the initial image so as to convert the initial image into a contour image and reduce the processing amount of image data. And screening the outline of the target two-dimensional code from the outline information according to a preset screening method so as to accurately position the area of the two-dimensional code. And extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline so as to obtain a local image needing to be decoded. And processing the two-dimensional code area image according to a second preprocessing method to obtain an image to be decoded, decoding the image to be decoded to obtain two-dimensional code information so as to optimize the image quality of the two-dimensional code area image and improve the identification speed.
Optionally, as shown in fig. 3, step S10 includes:
s101, acquiring preset setting parameters, wherein the preset setting parameters comprise environment setting parameters and shooting setting parameters;
s102, checking whether the current environment is matched with the environment setting parameters;
s103, if the current environment is matched with the environment setting parameters, shooting the target object according to the shooting setting parameters to obtain the initial image.
In this embodiment, a better shooting environment needs to be provided, so that the shooting device can obtain a clearer initial image.
As shown in fig. 4, the two-dimensional code recognition device 01 on the automatic production line may be connected to the light source controller 06, the motion control device 07, and the photographing apparatus 02, respectively. The light source controller 06 is connected to the industrial-grade light source 03 and is configured to control the brightness of the industrial-grade light source 03. The motion control device 07 is used to control the speed of motion of the conveyor belt 05. The subject 04 is placed on the conveyor belt 05. When the subject 04 comes within the effective field of view of the photographing apparatus 02, the photographing apparatus 02 acquires an initial image of the subject 04 and then transmits the initial image to the two-dimensional code recognition device 01. The two-dimensional code recognition device 01 includes a plurality of image recognition units, and can recognize two-dimensional code information from an initial image.
The two-dimensional code recognition device 01 may acquire preset setting parameters from an input device (not shown) (the input device herein refers to a provider of the preset setting parameters). The preset setting parameters include, but are not limited to, environment setting parameters and photographing setting parameters. In some cases, the preset setting parameters may further include a motion setting parameter, an image processing parameter (e.g., a setting parameter of a two-dimensional code outline). The two-dimensional code recognition device 01 can transmit the environment setting parameters to the light source controller 06, and the light source controller 06 adjusts the brightness of the industrial-grade light source 03 according to the environment setting parameters. In some cases, a light sensor (not shown) may be disposed near the industrial-grade light source 03 to check whether the luminous intensity of the industrial-grade light source 03 meets a preset requirement. And if the luminous intensity of the industrial-grade light source 03 meets the preset requirement, judging that the current environment is matched with the environment setting parameter.
When the current environment is matched with the environment setting parameters, the two-dimensional code recognition device 01 can send the shooting setting parameters to the shooting equipment 02, the shooting equipment 02 shoots the target object, an initial image is generated, and the initial image is sent back to the two-dimensional code recognition device 01.
In steps S101 to S103, preset setting parameters are obtained, where the preset setting parameters include environment setting parameters and shooting setting parameters, so as to obtain appropriate preset setting parameters and improve the imaging quality of the initial image. And checking whether the current environment is matched with the environment setting parameters or not to ensure that the initial image is shot under the environment controlled by the preset setting parameters, and if the environment is not matched with the environment setting parameters, generating error reporting information. And if the current environment is matched with the environment setting parameters, shooting the target object according to the shooting setting parameters to obtain the initial image so as to finish the shooting action and obtain the initial image to be processed.
Alternatively, as shown in fig. 5, step S20 includes:
s201, sharpening the initial image according to a preset sharpening parameter to obtain a sharpened image;
s202, performing self-adaptive binarization processing on the sharpened image to obtain a binarized image;
s203, carrying out median filtering processing on the binary image to obtain a median filtering image;
s204, performing morphological opening operation processing on the median filtering image to obtain at least one connected region;
s205, generating the contour information according to the at least one connected region.
In this embodiment, the preset sharpening parameter may be from the preset setting parameter. And sharpening the initial image by adopting a Laplacian operator to generate a sharpened image. For example, the discrete laplacian of the two variables used may be:
in some cases, other sharpening algorithms may also be used to sharpen the initial image.
After the sharpened image is obtained, the sharpened image can be subjected to binarization processing by adopting an adaptive binarization algorithm. The adaptive binarization algorithm includes, but is not limited to, an OTSU adaptive binarization algorithm. And (5) converting the sharpened image into a binary image through binarization processing. The Image Binarization (Image Binarization) can set the gray value of a pixel point on an Image to be 0 or 255, so that the Image presents an obvious black-and-white effect.
Then, the binarized image may be median filtered to generate a median filtered image. Median filtering is a nonlinear smoothing technique, which sets the gray value of each pixel point as the median of all the gray values of the pixel points in a certain neighborhood window of the point.
The median filtered image may then be subjected to morphological opening operations to generate an image comprising at least one connected region. The morphological opening operation can smooth contours in the median filtered image, eliminating thin and narrow protrusions. The convolution window of the morphological opening operation of this step can set larger parameters.
After morphological opening operation processing, at least one connected region can be obtained, and each connected region corresponds to one outline. And performing edge processing on the connected region, extracting the coordinates of each edge point, and generating contour data. The contour information includes all the contour data.
In steps S201 to S205, the initial image is sharpened according to a preset sharpening parameter to obtain a sharpened image, so as to enhance the edge contour of the two-dimensional code. And carrying out self-adaptive binarization processing on the sharpened image to obtain a binarized image so as to reduce the data processing amount of the initial image. And carrying out median filtering processing on the binary image to obtain a median filtering image so as to eliminate noise interference. And performing morphological open operation processing on the median filtering image to obtain at least one connected region so as to smooth the contour of the two-dimensional code image. And generating the contour information according to the at least one connected region to generate the contour information for screening.
Alternatively, as shown in fig. 6, step S30 includes:
s301, performing outer surrounding rectangle processing on the contour edge of the contour data in the contour information, and calculating the length and width of the outer surrounding rectangle;
s302, selecting the contour data with the length-width ratio within a preset length-width ratio range as first selected contour data;
s303, calculating the area of each first selected contour data;
s304, selecting the first selected contour data with the area within the preset area range as second selected contour data;
s305, extracting a second selected area image from the initial image according to the second selected contour data, and calculating the black-white pixel ratio of the second selected area image;
s306, selecting second selected contour data with the black-white pixel ratio within the preset black-white pixel ratio range as third selected contour data;
s307, calculating the contour center point of the third selected contour data, calculating the longest distance and the shortest distance between the contour center point and the contour edge point, and calculating the ratio of the longest distance and the shortest distance;
and S308, selecting third selected contour data with the ratio of the longest distance to the shortest distance within a preset proportion range as the target two-dimensional code contour.
In this embodiment, a plurality of screening conditions may be set, and when the screened contour data meets all the screening conditions, it may be determined that the contour data is a target two-dimensional code contour. In the screening process, these screening conditions may be arranged in parallel. For example, the contour data meeting the preset length-width ratio range can be screened out according to the steps S301 to S302, and then the contour data meeting the preset area range can be screened out according to the steps S303 to S304; or the contour data meeting the preset area range can be screened out according to the steps S303 to S304, and then the contour data meeting the preset length-width ratio range can be screened out according to the steps S301 to S302.
In one embodiment, the predetermined aspect ratio may be in the range of [0.6-1.4 ].
In one embodiment, the predetermined black-to-white pixel ratio range may be [0.5-1.5 ].
In one embodiment, the predetermined ratio range (ratio of the longest distance to the shortest distance) may be [0.86-2.0 ].
In steps S301 to S308, the outer bounding rectangle processing is performed on the contour edge of the contour data in the contour information, and the length and width of the outer bounding rectangle are calculated to estimate the aspect ratio of the contour data. And selecting the contour data with the length-width ratio within the preset length-width ratio range as first selected contour data so as to eliminate the contour data which do not conform to the length-width ratio and reduce the calculation amount of decoding the two-dimensional code. And calculating the area of each first selected contour data to calculate the area of the contour data. And selecting the first selected contour data with the area within the preset area range as the second selected contour data so as to eliminate the contour data which do not conform to the area and reduce the calculation amount of two-dimensional code decoding. And extracting a second selected area image from the initial image according to the second selected contour data, and calculating the black-white pixel ratio of the second selected area image so as to calculate the black-white pixel ratio of the image corresponding to the contour data. And selecting the second selected contour data with the black-white pixel ratio within the preset black-white pixel ratio range as third selected contour data so as to eliminate the unmatched contour data according to the black-white pixel ratio and reduce the calculation amount of two-dimensional code decoding. And calculating the contour center point of the third selected contour data, calculating the longest distance and the shortest distance between the contour center point and the contour edge point, and calculating the ratio of the longest distance and the shortest distance to calculate the ratio of the longest distance and the shortest distance between the contour center point and the contour edge point in the contour data. And selecting third selected contour data with the ratio of the longest distance to the shortest distance within a preset proportion range as the target two-dimensional code contour so as to eliminate inconsistent contour data according to the center distance ratio and reduce the calculated amount of two-dimensional code decoding.
Alternatively, as shown in fig. 7, step S40 includes:
s401, extracting an initial two-dimensional code area image from the initial image according to the target two-dimensional code outline;
s402, acquiring four two-dimensional code graphic vertexes of the initial two-dimensional code area image;
s403, selecting three two-dimensional code graphic vertexes from the four two-dimensional code graphic vertexes, and calculating angles of angles formed by the three two-dimensional code graphic vertexes, wherein the angles and the two-dimensional code graphics share two edges;
s404, judging whether the angle is within a preset angle range;
s405, if the angle is within a preset angle range, determining that the initial two-dimensional code area image is the two-dimensional code area image;
and S406, if the angle is not within the preset angle range, performing perspective transformation on the initial two-dimensional code area image to obtain the two-dimensional code area image.
In this embodiment, it can be determined whether a two-dimensional code graph composed of four two-dimensional code graph vertices is a rectangle-like graph. And if the two-dimension code graph is similar to a rectangle, the initial two-dimension code area image is the two-dimension code area image. The two-dimensional code region image can be decoded by adopting a standard decoding method. If the two-dimensional code graph is not similar to a rectangle, the initial two-dimensional code area image can be corrected by using perspective transformation, and the recognition rate of the initial image is improved. The perspective transformation is to project the picture to a new view plane, a transformation matrix can be calculated according to four two-dimensional code graphic vertexes of the initial two-dimensional code area image and four vertexes of the image to be transformed, and the two-dimensional code area image after the perspective transformation can be calculated by inputting the initial two-dimensional code area image and the calculated transformation matrix.
In steps S401-S406, extracting an initial two-dimensional code area image from the initial image according to the target two-dimensional code outline; acquiring four two-dimensional code graphic vertexes of the initial two-dimensional code area image; selecting three two-dimensional code pattern vertexes from the four two-dimensional code pattern vertexes, and calculating angles of angles formed by the three two-dimensional code pattern vertexes, wherein the angles and the two-dimensional code patterns share two edges; and judging whether the angle is within a preset angle range or not so as to determine whether the two-dimensional code graph in the initial two-dimensional code area image is a rectangle-like shape or not. And if the angle is within a preset angle range, determining that the initial two-dimensional code area image is the two-dimensional code area image, and at the moment, not carrying out deformation processing on the initial two-dimensional code area image. And if the angle is not within the preset angle range, performing perspective transformation on the initial two-dimensional code area image to obtain the two-dimensional code area image so as to correct the two-dimensional code graph and improve the recognition rate of the two-dimensional code.
Alternatively, as shown in fig. 8, step S50 includes:
s501, performing contrast enhancement processing on the two-dimensional code region image according to a preset contrast enhancement parameter to obtain a contrast optimized image;
s502, carrying out binarization processing on the contrast optimization image according to a preset gray threshold value to obtain the image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
In this embodiment, the contrast enhancement may use a 3 × 3 block second-order laplacian algorithm to facilitate separation of pixel information. The preset gray threshold can be set according to actual needs. After contrast enhancement processing, a contrast optimized image can be obtained.
The preset gray threshold value may include a plurality of gray threshold values. The starting grayscale threshold may be 20, 30, etc. And carrying out binarization processing on the contrast optimization image by using the initial gray threshold value to generate an image to be decoded, decoding the image to be decoded, and obtaining two-dimensional code information if the decoding is successful. If the decoding fails, the other gray threshold in the preset gray thresholds is used for continuing to optimize the contrast until the two-dimensional code information in the initial image is analyzed. And if all the preset gray level threshold values are used up and the two-dimensional code information cannot be obtained, outputting a prompt for identifying errors.
In steps S501 to S502, contrast enhancement processing is performed on the two-dimensional code region image according to a preset contrast enhancement parameter to obtain a contrast optimized image, so as to facilitate separation of pixel information in the two-dimensional code region image. And carrying out binarization processing on the contrast optimization image according to a preset gray threshold value to obtain the image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information so as to obtain the finally required two-dimensional code information.
Optionally, as shown in fig. 9, step S502 includes:
s5021, carrying out binarization processing on the contrast optimization image according to an initial gray threshold value to obtain a first image to be decoded;
s5022, decoding the first image to be decoded and judging whether decoding is successful or not;
s5023, if the first image to be decoded fails to be decoded, modifying the initial gray level threshold value by a specified step length to obtain a gray level change threshold value;
s5024, carrying out binarization processing on the contrast ratio optimized image according to the gray level change threshold value to obtain a second image to be decoded;
s5025, decoding the second image to be decoded;
s5026, if the second image to be decoded is decoded successfully, obtaining a decoding result of the second image to be decoded, namely the two-dimensional code information;
s5027, if the second image to be decoded fails to be decoded, the gray level change threshold value is continuously modified according to the specified step length until the modified gray level change threshold value exceeds a preset gray level range.
In this embodiment, different gray thresholds may be used to perform binarization processing on the contrast-optimized image, so as to obtain a corresponding binarized image. The binarized image is decoded using standard decoding methods. And if the decoding is successful, returning the identified two-dimensional code information. And if the decoding fails, replacing the gray threshold value and continuously generating the binary image. Replacing the grayscale threshold, a specified step size, such as 10, may be used. For example, the contrast-optimized image may be binarized using grayscale thresholds of 20, 30, and 40 … …, respectively. The preset gray scale range may be set as desired, such as may be [20,180 ].
In steps S5021-S5027, binarization processing is carried out on the contrast optimization image according to an initial gray threshold value to obtain a first image to be decoded; and decoding the first image to be decoded, judging whether the decoding is successful, outputting two-dimensional code information if the decoding is successful, and retrying if the decoding is failed. If the decoding of the first image to be decoded fails, modifying the initial gray level threshold value by a specified step length to obtain a gray level change threshold value; carrying out binarization processing on the contrast optimization image according to the gray level change threshold value to obtain a second image to be decoded; s5025, decoding the second image to be decoded; and if the second image to be decoded is successfully decoded, obtaining a decoding result of the second image to be decoded, namely the two-dimensional code information, and improving the recognition rate of the binary image by modifying the gray threshold value. And if the second image to be decoded fails to be decoded, the gray level change threshold value is continuously modified according to the specified step length until the modified gray level change threshold value exceeds a preset gray level range, so that the binary image is decoded in a wider gray level threshold value range, and the accuracy of two-dimensional code identification is improved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment, a two-dimensional code recognition device is provided, and the two-dimensional code recognition device corresponds to the two-dimensional code recognition method in the above embodiments one to one. As shown in fig. 10, the two-dimensional code recognition apparatus includes an image acquisition module 10, an outline acquisition module 20, an outline filtering module 30, an image extraction module 40, and an image decoding module 50. The functional modules are explained in detail as follows:
an image acquisition module 10, configured to acquire an initial image;
a contour obtaining module 20, configured to process the initial image according to a first preprocessing method, so as to obtain contour information of the initial image;
the contour screening module 30 is used for screening a target two-dimensional code contour from the contour information according to a preset screening method;
an image extraction module 40, configured to extract a two-dimensional code region image from the initial image according to the target two-dimensional code profile;
and the image decoding module 50 is configured to process the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decode the image to be decoded to obtain two-dimensional code information.
Optionally, the image obtaining module 10 includes:
the device comprises a parameter acquiring unit, a parameter setting unit and a parameter setting unit, wherein the parameter acquiring unit is used for acquiring preset setting parameters which comprise environment setting parameters and shooting setting parameters;
an environment checking unit for checking whether the current environment matches the environment setting parameter;
and the shooting unit is used for shooting the target object according to the shooting setting parameters to obtain the initial image if the current environment is matched with the environment setting parameters.
Optionally, the contour obtaining module 20 includes:
the sharpening unit is used for sharpening the initial image according to a preset sharpening parameter to obtain a sharpened image;
a binarization unit, configured to perform adaptive binarization processing on the sharpened image to obtain a binarized image;
a median filtering unit, configured to perform median filtering processing on the binarized image to obtain a median filtered image;
the opening operation unit is used for carrying out morphological opening operation processing on the median filtering image to obtain at least one connected region;
and the contour generating unit is used for generating the contour information according to the at least one connected region.
Optionally, the contour filtering module 30 includes:
a length and width calculation unit, configured to perform outer bounding rectangle processing on a contour edge of contour data in the contour information, and calculate a length and a width of the outer bounding rectangle;
the first screening unit is used for selecting the contour data with the length-width ratio within the preset length-width ratio range as first selected contour data;
the area calculation unit is used for calculating the area of each first selected contour data;
the second screening unit is used for selecting the first selected contour data with the area within the preset area range as second selected contour data;
the pixel ratio calculating unit is used for extracting a second selected area image from the initial image according to the second selected contour data and calculating the black-white pixel ratio of the second selected area image;
the third screening unit is used for selecting second selected contour data with the black-white pixel ratio within the preset black-white pixel ratio range as third selected contour data;
the distance ratio calculation unit is used for calculating the contour center point of the third selected contour data, calculating the longest distance and the shortest distance between the contour center point and the contour edge point, and calculating the ratio of the longest distance and the shortest distance;
and the fourth screening unit is used for selecting third selected contour data with the ratio of the longest distance to the shortest distance within a preset proportion range as the target two-dimensional code contour.
Optionally, the image extraction module 40 includes:
a preliminary extraction unit, configured to extract an initial two-dimensional code region image from the initial image according to the target two-dimensional code profile;
a vertex determining unit, configured to obtain four two-dimensional code graphic vertices of the initial two-dimensional code region image;
the angle calculation unit is used for selecting three two-dimensional code pattern vertexes from the four two-dimensional code pattern vertexes and calculating angles of angles formed by the three two-dimensional code pattern vertexes, wherein the angles and the two-dimensional code patterns share two edges;
the angle judging unit is used for judging whether the angle is within a preset angle range or not;
a first determining unit, configured to determine that the initial two-dimensional code region image is the two-dimensional code region image if the angle is within a preset angle range;
and the second determining unit is used for performing perspective transformation on the initial two-dimensional code area image to obtain the two-dimensional code area image if the angle is not within a preset angle range.
Optionally, the image decoding module 50 includes:
the contrast unit is used for carrying out contrast enhancement processing on the two-dimensional code region image according to a preset contrast enhancement parameter to obtain a contrast optimized image;
and the decoding unit is used for carrying out binarization processing on the contrast optimization image according to a preset gray threshold value to obtain the image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
Optionally, the decoding unit includes:
the first binarization subunit is used for carrying out binarization processing on the contrast optimization image according to an initial gray threshold value to obtain a first image to be decoded;
the first decoding subunit is used for decoding the first image to be decoded and judging whether the decoding is successful;
the gray modification subunit is used for modifying the initial gray threshold value by a specified step length to obtain a gray modification threshold value if the decoding of the first image to be decoded fails;
the second binarization subunit is used for carrying out binarization processing on the contrast optimization image according to the gray level change threshold value to obtain a second image to be decoded;
the second decoding subunit is used for decoding the second image to be decoded;
a decoding success subunit, configured to, if the second image to be decoded is decoded successfully, obtain a decoding result of the second image to be decoded, that is, the two-dimensional code information;
and the repeated decoding subunit is used for continuously modifying the gray level change threshold according to the specified step length if the second image to be decoded fails to be decoded until the modified gray level change threshold exceeds a preset gray level range.
For specific limitations of the two-dimensional code recognition device, reference may be made to the above limitations of the two-dimensional code recognition method, which are not described herein again. All or part of the modules in the two-dimensional code recognition device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 11. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for storing data related to the two-dimension code identification method. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a two-dimensional code recognition method.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring an initial image;
processing the initial image according to a first preprocessing method to obtain the contour information of the initial image;
screening a target two-dimensional code outline from the outline information according to a preset screening method;
extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring an initial image;
processing the initial image according to a first preprocessing method to obtain the contour information of the initial image;
screening a target two-dimensional code outline from the outline information according to a preset screening method;
extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A two-dimensional code recognition method is characterized by comprising the following steps:
acquiring an initial image;
processing the initial image according to a first preprocessing method to obtain the contour information of the initial image;
screening a target two-dimensional code outline from the outline information according to a preset screening method;
extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
2. The two-dimensional code recognition method of claim 1, wherein the acquiring an initial image comprises:
acquiring preset setting parameters, wherein the preset setting parameters comprise environment setting parameters and shooting setting parameters;
checking whether the current environment is matched with the environment setting parameters;
and if the current environment is matched with the environment setting parameters, shooting the target object according to the shooting setting parameters to obtain the initial image.
3. The two-dimensional code recognition method of claim 1, wherein the processing the initial image according to the first preprocessing method to obtain the contour information of the initial image comprises:
sharpening the initial image according to a preset sharpening parameter to obtain a sharpened image;
carrying out self-adaptive binarization processing on the sharpened image to obtain a binarized image;
carrying out median filtering processing on the binary image to obtain a median filtering image;
performing morphological opening operation processing on the median filtering image to obtain at least one connected region;
generating the contour information from the at least one connected region.
4. The two-dimensional code recognition method of claim 1, wherein the screening of the target two-dimensional code profile from the profile information according to a preset screening method comprises:
performing outer surrounding rectangle processing on the contour edge of the contour data in the contour information, and calculating the length and width of the outer surrounding rectangle;
selecting profile data with the length-width ratio within a preset length-width ratio range as first selected profile data;
calculating the area of each first selected contour data;
selecting first selected contour data with the area within a preset area range as second selected contour data;
extracting a second selected area image from the initial image according to the second selected contour data, and calculating the black-white pixel ratio of the second selected area image;
selecting second selected contour data with the black-white pixel ratio within a preset black-white pixel ratio range as third selected contour data;
calculating the contour center point of the third selected contour data, calculating the longest distance and the shortest distance between the contour center point and the contour edge point, and calculating the ratio of the longest distance and the shortest distance;
and selecting third selected contour data with the ratio of the longest distance to the shortest distance within a preset proportion range as the target two-dimensional code contour.
5. The two-dimensional code recognition method according to claim 1, wherein the extracting a two-dimensional code area image from the initial image according to the target two-dimensional code profile comprises:
extracting an initial two-dimensional code area image from the initial image according to the target two-dimensional code outline;
acquiring four two-dimensional code graphic vertexes of the initial two-dimensional code area image;
selecting three two-dimensional code pattern vertexes from the four two-dimensional code pattern vertexes, and calculating angles of angles formed by the three two-dimensional code pattern vertexes, wherein the angles and the two-dimensional code patterns share two edges;
judging whether the angle is within a preset angle range or not;
if the angle is within a preset angle range, determining that the initial two-dimensional code area image is the two-dimensional code area image;
and if the angle is not within the preset angle range, carrying out perspective transformation on the initial two-dimensional code area image to obtain the two-dimensional code area image.
6. The two-dimensional code recognition method according to claim 1, wherein the processing the two-dimensional code region image according to the second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information includes:
performing contrast enhancement processing on the two-dimensional code region image according to a preset contrast enhancement parameter to obtain a contrast optimized image;
and carrying out binarization processing on the contrast optimization image according to a preset gray threshold value to obtain the image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
7. The two-dimensional code identification method according to claim 6, wherein the binarizing the contrast-optimized image according to a preset gray threshold to obtain the image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information comprises:
carrying out binarization processing on the contrast optimization image according to an initial gray threshold value to obtain a first image to be decoded;
decoding the first image to be decoded, and judging whether the decoding is successful;
if the decoding of the first image to be decoded fails, modifying the initial gray level threshold value by a specified step length to obtain a gray level change threshold value;
carrying out binarization processing on the contrast optimization image according to the gray level change threshold value to obtain a second image to be decoded;
decoding the second image to be decoded;
if the second image to be decoded is decoded successfully, acquiring a decoding result of the second image to be decoded, namely the two-dimensional code information;
and if the second image to be decoded fails to be decoded, continuously modifying the gray level change threshold according to the specified step length until the modified gray level change threshold exceeds a preset gray level range.
8. A two-dimensional code recognition device, comprising:
the image acquisition module is used for acquiring an initial image;
the contour acquisition module is used for processing the initial image according to a first preprocessing method to acquire contour information of the initial image;
the contour screening module is used for screening a target two-dimensional code contour from the contour information according to a preset screening method;
the image extraction module is used for extracting a two-dimensional code area image from the initial image according to the target two-dimensional code outline;
and the image decoding module is used for processing the two-dimensional code region image according to a second preprocessing method to obtain an image to be decoded, and decoding the image to be decoded to obtain two-dimensional code information.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the two-dimensional code recognition method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the two-dimensional code recognition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910969469.9A CN112651256A (en) | 2019-10-12 | 2019-10-12 | Two-dimensional code identification method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910969469.9A CN112651256A (en) | 2019-10-12 | 2019-10-12 | Two-dimensional code identification method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112651256A true CN112651256A (en) | 2021-04-13 |
Family
ID=75343332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910969469.9A Pending CN112651256A (en) | 2019-10-12 | 2019-10-12 | Two-dimensional code identification method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112651256A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113468905A (en) * | 2021-07-12 | 2021-10-01 | 深圳思谋信息科技有限公司 | Graphic code identification method and device, computer equipment and storage medium |
| CN114330407A (en) * | 2021-12-30 | 2022-04-12 | 深圳创维－Rgb电子有限公司 | Method, device, equipment and storage medium for detecting and identifying bar code |
| CN114723766A (en) * | 2022-04-14 | 2022-07-08 | 润芯微科技(江苏)有限公司 | Two-dimensional code extraction and display method and electronic equipment |
| CN115375625A (en) * | 2022-07-25 | 2022-11-22 | 北京数美时代科技有限公司 | Two-dimensional code detection decoding method, system, electronic device and medium |
| CN116468060A (en) * | 2023-04-23 | 2023-07-21 | 创新奇智(上海)科技有限公司 | DPM code processing method, device, storage medium and equipment |
| CN116542926A (en) * | 2023-05-04 | 2023-08-04 | 上海感图网络科技有限公司 | Method, device, equipment and storage medium for identifying defects of two-dimension codes of battery |
| CN116563048A (en) * | 2023-07-05 | 2023-08-08 | 江西科技学院 | Financial reimbursement method, system and computer |
| CN113689378B (en) * | 2021-07-07 | 2024-04-05 | 杭州未名信科科技有限公司 | Determination method and device for accurate positioning of test strip, storage medium and terminal |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009113231A1 (en) * | 2008-03-14 | 2009-09-17 | 株式会社ソニー・コンピュータエンタテインメント | Image processing device and image processing method |
| CN107545207A (en) * | 2017-09-28 | 2018-01-05 | 云南电网有限责任公司电力科学研究院 | DM two-dimensional code identification methods and device based on image procossing |
| CN109815763A (en) * | 2019-01-04 | 2019-05-28 | 广州广电研究院有限公司 | Detection method, device and the storage medium of two dimensional code |
-
2019
- 2019-10-12 CN CN201910969469.9A patent/CN112651256A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009113231A1 (en) * | 2008-03-14 | 2009-09-17 | 株式会社ソニー・コンピュータエンタテインメント | Image processing device and image processing method |
| CN107545207A (en) * | 2017-09-28 | 2018-01-05 | 云南电网有限责任公司电力科学研究院 | DM two-dimensional code identification methods and device based on image procossing |
| CN109815763A (en) * | 2019-01-04 | 2019-05-28 | 广州广电研究院有限公司 | Detection method, device and the storage medium of two dimensional code |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113689378B (en) * | 2021-07-07 | 2024-04-05 | 杭州未名信科科技有限公司 | Determination method and device for accurate positioning of test strip, storage medium and terminal |
| CN113468905A (en) * | 2021-07-12 | 2021-10-01 | 深圳思谋信息科技有限公司 | Graphic code identification method and device, computer equipment and storage medium |
| WO2023284609A1 (en) * | 2021-07-12 | 2023-01-19 | 深圳思谋信息科技有限公司 | Graphic code identification method and apparatus, computer device, and storage medium |
| CN113468905B (en) * | 2021-07-12 | 2024-03-26 | 深圳思谋信息科技有限公司 | Graphic code identification method, graphic code identification device, computer equipment and storage medium |
| CN114330407A (en) * | 2021-12-30 | 2022-04-12 | 深圳创维－Rgb电子有限公司 | Method, device, equipment and storage medium for detecting and identifying bar code |
| CN114723766A (en) * | 2022-04-14 | 2022-07-08 | 润芯微科技(江苏)有限公司 | Two-dimensional code extraction and display method and electronic equipment |
| CN115375625A (en) * | 2022-07-25 | 2022-11-22 | 北京数美时代科技有限公司 | Two-dimensional code detection decoding method, system, electronic device and medium |
| CN116468060A (en) * | 2023-04-23 | 2023-07-21 | 创新奇智(上海)科技有限公司 | DPM code processing method, device, storage medium and equipment |
| CN116542926A (en) * | 2023-05-04 | 2023-08-04 | 上海感图网络科技有限公司 | Method, device, equipment and storage medium for identifying defects of two-dimension codes of battery |
| CN116563048A (en) * | 2023-07-05 | 2023-08-08 | 江西科技学院 | Financial reimbursement method, system and computer |
| CN116563048B (en) * | 2023-07-05 | 2024-05-03 | 江西科技学院 | Financial reimbursement method, system and computer |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112651256A (en) | Two-dimensional code identification method and device, computer equipment and storage medium | |
| US9530069B2 (en) | Method and system for detecting and recognizing text in images | |
| US8590794B2 (en) | Barcode recognion method and computer product thereof | |
| CN110348264B (en) | QR two-dimensional code image correction method and system | |
| JP4531809B2 (en) | Code image processing method | |
| CN107886026B (en) | graphic code processing method and device | |
| CN111461100B (en) | Bill identification method and device, electronic equipment and storage medium | |
| US11341739B2 (en) | Image processing device, image processing method, and program recording medium | |
| US10990778B2 (en) | Apparatus and method for recognizing barcode based on image detection | |
| CN111681160B (en) | Curved image restoration method, device and equipment and readable storage medium | |
| CN111582000A (en) | Bar code positioning method and device and related equipment | |
| CN113963353A (en) | Character image processing and identifying method and device, computer equipment and storage medium | |
| CN111783795A (en) | Method, apparatus, device and medium for converting image into laser scanning path | |
| CN111860582A (en) | Image classification model construction method and device, computer equipment and storage medium | |
| CN111652011B (en) | Method and device for reading two-dimensional code | |
| CN112926695A (en) | Image recognition method and system based on template matching | |
| CN115221909A (en) | Two-dimensional code identification method, device and equipment and computer readable storage medium | |
| CN115331234A (en) | Computer-readable non-transitory recording medium storing image processing program, and image processing apparatus | |
| CN109448013B (en) | QR code image binarization processing method with local uneven illumination | |
| CN115311242A (en) | Industrial camera image sharpening method, device, equipment and storage medium | |
| CN114299299A (en) | Tree leaf feature extraction method and device, computer equipment and storage medium | |
| CN113283262A (en) | Two-dimensional code positioning method and device, electronic equipment and machine readable medium | |
| CN112927288B (en) | Image coordinate extraction method, image processing device and storage device | |
| CN107392205B (en) | Code value table generation method and device of remote controller | |
| CN115759146A (en) | Two-dimensional bar code identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |