CN112241640A - Graphic code determination method and device and industrial camera - Google Patents

Graphic code determination method and device and industrial camera Download PDF

Info

Publication number
CN112241640A
CN112241640A CN201910651699.0A CN201910651699A CN112241640A CN 112241640 A CN112241640 A CN 112241640A CN 201910651699 A CN201910651699 A CN 201910651699A CN 112241640 A CN112241640 A CN 112241640A
Authority
CN
China
Prior art keywords
graphic code
image
processing
code image
data format
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910651699.0A
Other languages
Chinese (zh)
Other versions
CN112241640B (en
Inventor
徐跃书
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201910651699.0A priority Critical patent/CN112241640B/en
Publication of CN112241640A publication Critical patent/CN112241640A/en
Application granted granted Critical
Publication of CN112241640B publication Critical patent/CN112241640B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1439—Methods for optical code recognition including a method step for retrieval of the optical code
    • G06K7/1443—Methods for optical code recognition including a method step for retrieval of the optical code locating of the code in an image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
    • G06K7/1413—1D bar codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
    • G06K7/1417—2D bar codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1439—Methods for optical code recognition including a method step for retrieval of the optical code
    • G06K7/1447—Methods for optical code recognition including a method step for retrieval of the optical code extracting optical codes from image or text carrying said optical code
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/146—Methods for optical code recognition the method including quality enhancement steps
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Toxicology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Quality & Reliability (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a method and a device for determining a graphic code and an industrial camera, wherein the method comprises the following steps: acquiring position information of a first graphic code; intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code; performing image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format. Since the first image in the first data format can retain more image information, the accuracy of locating the first graphic code image from the first data format is higher.

Description

Graphic code determination method and device and industrial camera
Technical Field
The present application relates to the field of image processing, and in particular, to a method and an apparatus for determining a graphic code, and an industrial camera.
Background
The determination of the graphic code means that the graphic code is determined from the acquired image.
The existing pattern code determination method is as follows: and performing enhancement processing on the image output by the camera, and then recognizing the image code from the image after the enhancement processing. However, the image output by the camera is already the image obtained by processing the image in the first data format acquired by the camera, and much original information is lost, so that the image output by the acquired camera is subjected to pattern code recognition, and the recognized pattern code has low accuracy.
Disclosure of Invention
In view of this, the present application provides a method and an apparatus for determining a graphic code, and an industrial camera.
Specifically, the method is realized through the following technical scheme:
according to a first aspect of the present application, there is provided a graphic code determination method, which is applied to an image acquisition device, and includes:
acquiring position information of a first graphic code;
intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code;
performing image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format.
Optionally, the image processing includes at least one or more of the following combinations:
brightness correction processing;
resolution correction processing;
and (5) blur restoration processing.
Optionally, if the image processing is brightness correction processing, the image processing on the first graphic code image includes:
determining the brightness mean value of the first graphic code image;
determining a brightness gain value based on the determined brightness mean value and the preset brightness mean value;
and performing brightness correction on the first graphic code image based on the brightness gain value to obtain the second graphic code image.
Optionally, if the image processing is resolution correction processing, the image processing on the first graphic code image includes:
inputting the first graphic code image to a trained first neural network; the first neural network at least realizes the resolution enhancement of the first graphic code image through the convolution layer and the correction linear layer which are combined according to a first combination mode to obtain a second graphic code image; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation;
and acquiring a second graphic code image output by the second neural network.
Optionally, if the image processing is a blur restoration processing, the performing image processing on the first graphic code image includes:
inputting the first graphic code image to a trained second neural network; the second neural network realizes the fuzzy restoration of the first graphic code image to obtain a second graphic code image at least through a convolution layer and a correction linear layer which are combined according to a second combination mode; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation;
and acquiring a second graphic code image which is output by the second neural network and is subjected to fuzzy restoration.
Optionally, the determining the position information of the graphic code from the acquired first image in the first data format includes:
preprocessing the first image to obtain a second image capable of detecting the graphic code;
and determining the position information of the graphic code in the second image, and taking the position information determined in the second image as the position information of the graphic code in the first image.
Optionally, the determining the position information of the graphic code in the second image includes:
inputting the second image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for executing normalization processing, a convolution layer for executing convolution, a pooling layer for executing downsampling, a full-connection layer for executing feature synthesis and a frame regression layer for executing coordinate transformation;
and taking the result output by the third neural network as the position information of the graphic code in the second image.
Optionally, the preprocessing the first image to obtain a second image capable of performing graphic code detection includes:
and preprocessing the first image by at least adopting curve mapping to obtain a second image capable of detecting the graphic code.
Optionally, the determining the position information of the graphic code from the acquired first image in the first data format includes:
inputting the first image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for normalization processing, a convolution layer for convolution execution, a pooling layer for downsampling execution, a full-link layer for feature synthesis execution and a regression layer for coordinate transformation execution;
and taking the output result of the third neural network as the position information of the first graphic code in the first image.
Optionally, before performing image processing on the first graphic code image to obtain a second graphic code image with an image quality higher than that of the first graphic code image and a data format different from that of the first data format, the method further includes:
detecting whether the appointed bottom edge of the first graphic code image is overlapped with the horizontal direction;
if not, calculating the deviation angle of the appointed bottom edge of the first graphic code image and the horizontal direction based on the position coordinates of each vertex of the first graphic code image;
and rotating the first graphic code image based on the position coordinates of each vertex of the first graphic code image and the deviation angle so as to enable the appointed bottom side of the first graphic code image to be consistent with the horizontal direction.
Optionally, the method further includes:
performing character recognition on the second graphic code image to obtain recognition information;
alternatively, the first and second electrodes may be,
and sending the second graphic code image to a graphic code recognition device for character recognition.
According to a second aspect of the present application, there is provided a graphic code determination apparatus, which is applied to an image capturing device, the apparatus including:
the first processing module is used for acquiring the position information of the first graphic code and intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code.
The second processing module is used for carrying out image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format.
According to a third aspect of the present application, there is provided an industrial camera comprising a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor, the processor being caused by the machine-executable instructions to perform the method of the first aspect.
As can be seen from the above description, the image capturing device captures a first graphic code image from a first image in a first data format based on the position information of the first graphic code image, performs image processing on the first graphic code image to obtain a second graphic code image in a second data format, and outputs the second graphic code image.
Since the first image in the first data format can retain more image information, the accuracy of locating the first graphic code image from the first data format is higher.
Drawings
FIG. 1a is a schematic diagram illustrating software modules of an image capture device according to an exemplary embodiment of the present application;
FIG. 1b is a diagram illustrating software modules in a first processing module according to an exemplary embodiment of the present application;
fig. 2 is a flowchart illustrating a method for determining a graphic code according to an exemplary embodiment of the present application;
fig. 3a is a logic block diagram illustrating a method for determining location information of a graphic code according to an exemplary embodiment of the present application;
fig. 3b is a logic block diagram of another method for determining location information of a graphic code according to an exemplary embodiment of the present application;
FIG. 4 is a schematic diagram illustrating a method of dead pixel correction according to an exemplary embodiment of the present application;
FIG. 5a is a schematic diagram of a third neural network shown in an exemplary embodiment of the present application;
FIG. 5b is a schematic diagram of another third neural network shown in an exemplary embodiment of the present application;
FIG. 6 is a logic block diagram of a method of position correction processing shown in an exemplary embodiment of the present application;
fig. 7a is a schematic diagram illustrating a first graphic code placement location according to an exemplary embodiment of the present application;
FIG. 7b is a schematic diagram illustrating another first graphic code placement location according to an exemplary embodiment of the present application;
FIG. 7c is a schematic diagram illustrating another first graphic code placement location according to an exemplary embodiment of the present application;
FIG. 7d is a schematic diagram illustrating another first graphic code placement location according to an exemplary embodiment of the present application;
FIG. 8 is a logic block diagram of a luminance correction processing method shown in an exemplary embodiment of the present application;
fig. 9a is a schematic diagram illustrating software modules in a second processing module according to an exemplary embodiment of the present application.
FIG. 9b is a schematic diagram of a first neural network shown in an exemplary embodiment of the present application;
FIG. 10a is a logic block diagram of a blur restoration process shown in an exemplary embodiment of the present application;
FIG. 10b is a schematic diagram of a second neural network shown in an exemplary embodiment of the present application;
fig. 11 is a block diagram of a graphic code determination apparatus according to an exemplary embodiment of the present application;
fig. 12 is a hardware configuration diagram of an industrial camera according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
The method comprises the steps that image acquisition equipment acquires position information of a first graphic code, intercepts a first graphic code image from a first image based on the position information, performs image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format, and outputs the second graphic code image.
On one hand, the first graphic code image can be positioned more accurately from the first data format because the first image in the first data format can keep more image information.
On the other hand, the method and the device also perform image processing on the first graphic code image which is cut from the first image in the first data format to obtain the second graphic code image which is higher in image quality and suitable for display and/or transmission in the data format, so that the image quality of the graphic code image transmitted or displayed by the image acquisition equipment is higher.
Before introducing the method for determining the graphic code, a software module in the image acquisition equipment is introduced.
Referring to fig. 1a, fig. 1a is a schematic diagram illustrating software modules of an image capturing device according to an exemplary embodiment of the present application.
The image capturing device refers to a device having an image capturing function, such as an industrial camera, a camera, and the like. The image capturing device is only exemplified and not particularly limited herein.
The software modules of the image capturing apparatus 100 may include: a first processing module 101 and a second processing module 102.
1) First processing module 101
The first processing module 101 is configured to obtain position information of a first graphic code, and intercept a first graphic code image from the first image based on the position information of the graphic code.
Referring to fig. 1b, fig. 1b is a schematic diagram illustrating software modules in a first processing module 101 according to an exemplary embodiment of the present application.
The first processing module 101 includes: a first processing sub-module 1011 and a second processing sub-module 1012.
And the first processing sub-module 1011 is configured to obtain the position information of the first graphic code.
And a second processing sub-module 1012, configured to intercept the first graphic code image from the first image based on the location information of the first graphic code, and output the first graphic code image in the first data format.
2) Second processing module 102
The second processing module 102 is configured to perform image processing on the first graphic code image, and the second processing module 102 outputs a second graphic code image with an image quality higher than that of the first graphic code image and a data format of a second data format.
It should be noted that, in practical applications, the image capturing apparatus may further include other software modules, and here, the software modules in the image capturing apparatus are only exemplarily illustrated and are not specifically limited.
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for determining a graphic code according to an exemplary embodiment of the present application, where the method is applicable to an image capturing apparatus and may include the following steps.
Step 201: and acquiring the position information of the first graphic code.
Step 201 may be applied to the first processing sub-module 1011 described above in fig. 1 b. The image sensor of the image capturing device may capture an image, and transmit the captured first image in the first data format to the first sub-processing module 1011 in the first processing module 101, and the first sub-processing module 1011 determines the position information of the first graphic code from the captured first image in the first data format.
It should be noted that:
1. the first data format referred to in this application refers to a raw data format of an image captured by an image capture sensor of an image capture device.
For example, the first data format refers to a data format in which an all-pass, color-insensitive image sensor converts captured light source signals into raw images of digital signals. The RAW image typically contains sensed data from one or more spectral bands, such as RAW image signals acquired by an RGB sensor.
2. Reference herein to a second data format is to any data format suitable for display and/or transmission.
3. The graphic code referred to in this application refers to a graphic identifier that expresses information. The graphical code may include a bar code (also referred to as a one-dimensional code), a two-dimensional code, and the like. The graphic code is only illustrated by way of example and is not particularly limited.
The implementation of step 201 is described in detail below.
The first method is as follows:
for example, referring to fig. 3a, fig. 3a is a logic block diagram of a method for determining location information of a graphic code according to an exemplary embodiment of the present application.
The first processing sub-module 1011 of the image acquisition apparatus may perform pre-processing on the acquired first image to obtain a second image capable of performing graphic code detection, and then perform graphic code detection on the second image to obtain the position information of the first graphic code in the second image, where the obtained position information of the graphic code is the position information of the graphic code in the first image.
The first mode will be described in detail through steps 2011 to 2012.
Step 2011: and preprocessing the first image to obtain a second image capable of detecting the graphic code.
When the method is realized, the first image can be preprocessed at least by adopting curve mapping to obtain a second image which can be used for detecting the graphic code.
The preprocessing method at least includes curve mapping, and certainly, in practical applications, other common image processing methods may also be included, such as dead pixel correction, and the like, where the preprocessing method is only exemplarily described and is not specifically limited.
The preprocessing method includes dead pixel correction and curve mapping as examples, and step 1011 is described.
As shown in fig. 3b, the first image is preprocessed by using a preprocessing method of dead pixel correction and curve mapping to obtain a second image capable of being detected, and then the second image is subjected to graphic code detection to obtain the position information of the first graphic code in the second image.
1) Dead pixel correction
The dead pixel correction is to remove some abnormal pixel points in the image of the image sensor.
Due to the defects of the array process on the image sensor or errors in the process of converting the optical signals, pixel values in the imaged image are inaccurate, and these inaccurate pixel points are called Bad pixels (Bad pixels). Since the pixel value of the image dead pixel is generally larger or smaller than the pixel values of the surrounding pixel points, a filter can be used to eliminate the dead pixel.
For example, a median filter of 3 x3 may be used to eliminate dead spots.
In particular, a3 x3 filtering window may be employed to slide over the image. For each sliding, 9 pixels are circled out by the 3 × 3 filtering window. And sequencing the 9 pixel points according to the sequence that the pixel values of the 9 pixel points are from large to small or from small to large. And selecting the pixel value of the pixel point ordered in the middle as the pixel value of the central pixel point of the 3 x3 filtering window because the pixel value of the image dead pixel is larger or smaller than the pixel value of the normal pixel point.
For example, as shown in fig. 4, fig. 4 is a schematic diagram of a dead pixel correction method according to an exemplary embodiment of the present application.
Suppose that the 9 pixels circled by the filter window of 3 × 3 are a1, a2, A3, a4, a5, a6, a7, A8, and a9 in sequence, and assume that the arrangement of the 9 pixels is as shown in the right diagram of fig. 4. Then, the 9 pixel points are sorted according to the sequence of the pixel values of the 9 pixel points from large to small or from small to large. Assuming that the sorted pixel point in the middle rank is A8, the pixel value of the A8 pixel point is used as the pixel value of the a4 pixel point (the central pixel point of the 3 × 3 filtering window).
2) Curve mapping
The curve mapping is to compress an image with a large bit width value into an image with a small bit width (for example, an image with a bit width of 12bits is mapped to an image with a bit width of 8 bits). Meanwhile, the curve mapping can also adjust the contrast of the image, so that the image with the adjusted contrast is suitable for the human eyes to watch.
In implementation, a Gamma curve may be used for curve mapping, and the specific formula is as follows:
Lout＝L08max*(Lin/L12max)1/γ
wherein L isinTo representInput image, LoutRepresenting the output image, and Gamma represents the Gamma curve parameter. When γ is 1, it indicates that an image with a bit width of 12bits is directly compressed into an image with a bit width of 8 bits.
Step 2012: and determining the position information of the graphic code in the second image, and taking the position information determined in the second image as the position information of the graphic code in the first image.
The first processing sub-module 1011 of fig. 1b may perform step 2012, and when it is executed, the second processing sub-module 1012 may determine the position information of the graphic code in the second image through the third neural network.
Referring to fig. 5a, the third neural network 500 may include at least: a normalization layer 501 for performing normalization processing, a convolution layer 502 for performing convolution, a pooling layer 503 for performing downsampling, a fully-connected layer 504 for performing feature synthesis, and a bounding box regression layer 505 for performing coordinate transformation.
As an embodiment of the third neural network, referring to fig. 5b, the first neural network 500 may include: normalization layer 511, convolution layer 512, convolution layer 513, pooling layers 514, …, convolution layer 515, pooling layer 516, fully-connected layer 517 and border regression layer 518 connected in sequence.
The first processing sub-module 1011 may input the second image into a third neural network, and the third neural network may output the position information of the first graphic code in the second image.
It should be noted that, the functions performed by each layer of the third neural network will be described below, and each layer may be adaptively changed, for example, the convolution kernels of different convolution layers may be different, and will not be described herein again. It is to be understood that the third neural network shown in fig. 5b is only an example, and is not particularly limited thereto, for example, convolutional layers, and/or pooling layers, and/or other layers may be reduced or added.
The specific functions of the layers in the third neural network are described in detail below, but should not be limited thereto.
1) Normalization layer
The normalization layer is to normalize the input image in the third neural network, and generally normalizes the pixel value of each pixel point in the image to the range of [ -1,1], and the calculation formula is as follows:
Figure BDA0002135445630000101
wherein L isinRepresenting inputs to a normalization layer in a third neural network, LoutRepresenting the output of the normalization layer.
2) Convolutional layer
Typically, each convolutional layer carries an activation function ReLU, so the operation for a convolutional layer can be represented by the following formula:
YCi(I)＝g(Wi*YCi-1(I)+Bi)
wherein I represents the coordinate set of all pixel points, YCi(I) Is the output of the current convolutional layer, YCi-1(I) For the input of the current convolution layer, denotes the convolution operation, WiWeight coefficients of the convolution filter for the current convolution layer, BiFor the offset coefficients of the convolution filter of the current convolution layer, g () represents the activation function, and when the activation function is ReLU, g (x) is max (0, x).
3) Pooling layer
The pooling layer is a special down-sampling layer, namely, the feature map obtained by convolution is reduced, and the specific formula is as follows:
YPj(I)＝maxpool(YPj-1(I))
wherein YPj-1(I) Is the input to the jth pooling layer, YPj(I) Is the output of the jth pooling layer.
4) Full connection layer
The full-link layer can be regarded as a convolutional layer with a filtering window of 1 × 1, and is implemented similarly to convolutional filtering, where the expression is as follows:
Figure BDA0002135445630000111
wherein, I represents the coordinate set of all pixel points, and (I, j) represents the coordinate of one pixel point. FkI(I) For input to the kth fully-connected layer, YFk(I) R, C are F for the k-th fully-connected layer outputkI(I) Width, height, WijAnd BijThe connection weights and bias coefficients of the fully connected layer, respectively, g () represent the activation function.
5) Frame regression layer
The frame regression layer (BBR) is used for searching a relation so that a window G' which is closer to a real graphic code frame G is obtained by mapping a window P output by the full connection layer; the regression is generally carried out by transforming the coordinates of the window P, including for example a translation transformation and/or a scaling transformation; let the coordinate of the window P of the full link layer output be (x)1，x2，y1，y2) Transformed coordinate (x) after window3，x4，y3，y4)；
If the translation is converted into translation transformation, the translation scale is (Δ x, Δ y), and the coordinate relationship before and after translation is as follows:
x3＝x1+Δx
x4＝x2+Δx
y3＝y1+Δy
y4＝y2+Δy
if the scale is transformed into scaling transformation, the scaling in the direction X, Y is dx and dy, and the coordinate relationship before and after transformation is:
x4-x3＝(x2-x1)*dx
y4-y3＝(y2-y1)*dy
it should be noted that, for the training of the third neural network in the first mode, the second image may be used as a sample, and the image code position information in the second image is used as a label to train the third neural network, and the training mode is the same as that of the existing training neural network, and is not repeated here.
The second method comprises the following steps:
the first processing sub-module 1011 of the image capturing apparatus may input the first image into the above-described third neural network, and the third neural network may determine the position information of the first graphic code in the first image through a normalization layer for performing normalization processing, a convolution layer for performing convolution, a pooling layer for performing downsampling, a full-link layer for performing feature synthesis, and a bounding box regression layer for performing coordinate transformation, and output the first graphic code position information.
In the second mode, the first image is used as a sample, and the position information of the first graphic code in the first image is used as a label to train the third neural network.
Step 202: and intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code.
Step 202 may be applied to the second processing sub-module 1012 in fig. 1b, and the second processing sub-module 1012 intercepts the first graphic code image from the first image in the first data format based on the position information of the first graphic code. The specific interception mode may refer to the existing mode for interception, and is not described herein again.
In order to obtain a more regular first graphic code image, the second processing sub-module 1012 may perform position correction processing on the first graphic code image after intercepting the first graphic code image.
As shown in fig. 6, the second processing sub-module 1012 performs S601 angle calculation on the first graphic code image, and then performs S602 position correction.
S601, angle calculation:
as shown in fig. 7a, fig. 7a is a schematic diagram illustrating a first graphic code placement position according to an exemplary embodiment of the present application. The first graphic code placing position shown in fig. 7a is an ideal placing position of the first graphic code, and according to the graphic code placed in fig. 7a, four vertexes, starting from the upper left corner, are vertex a, vertex B, vertex C and vertex D in turn in a counterclockwise order.
The coordinates of the vertex a, the vertex B, the vertex C, and the vertex D are (x0, y0), (x1, y1), (x2, y2), and (x3, y3) in this order.
Wherein, the designated bottom edge of the first graphic code image is the BC edge.
S601 is explained in detail by step 6011 and step 6012.
Step 6011: detecting whether the appointed bottom edge of the first graphic code image is overlapped with the horizontal direction
In implementation, it may be detected whether y1 and y2 are equal. If y1 is y2, it indicates that the bottom side of the first graphic code image coincides with the horizontal direction. If y1 ≠ y2, it indicates that the specified bottom edge of the first graphic code image does not coincide with the horizontal direction.
For example, when the position of the first graphic code image is as shown in fig. 7b and 7c, it indicates that the designated bottom side of the first graphic code image is not coincident with the horizontal direction.
Step 6012: if the designated bottom edge of the first graphic code image does not coincide with the horizontal direction, the deviation angle between the bottom edge of the first graphic code image and the horizontal direction can be calculated based on the position coordinates of each vertex of the first graphic code image.
In implementation, as shown in fig. 7b, if x0 is x3, it indicates that the designated base BC side is perpendicular to the horizontal direction, and the deviation angle is 90 °.
As shown in fig. 7c, if x0 ≠ x3, it indicates that the designated base BC is not perpendicular to the horizontal direction, and the deviation angle a can be calculated by the following formula.
Figure BDA0002135445630000141
In addition, if the specified bottom side of the first graphic code image coincides with the horizontal direction, the position of the first graphic code image does not need to be corrected.
S602, position correction:
when the method is realized, based on the position coordinates of each vertex of the first graphic code image and the deviation angle, the first graphic code image is rotated, so that the appointed bottom edge of the first graphic code image is consistent with the horizontal direction.
The specific procedure is as follows:
s6021: and calculating the length and the width of the rotated first graphic code according to the long edge, the wide edge and the deviation angle of the first graphic code before rotation, and constructing a rotated first graphic code image based on the calculated length and the calculated width of the rotated first graphic code.
For example, as shown in fig. 7D, assume that the first graphic code image before rotation is ABCD and the first graphic code image after rotation is a 'B' C 'D'.
And calculating the long side B 'C' of the rotated first graphic code according to the long side BC of the first graphic code before rotation and the deviation angle.
And calculating the broadside A 'B' edge of the rotated first graphic code according to the broadside AB edge of the first graphic code before rotation and the deviation angle.
Then, the vertex B is unchanged, and a rotated first graphic code image A 'B' C 'D' is constructed according to the calculated B 'C' side and A 'B' side.
S6022: and determining the pixel value of each pixel point of the formed first graphic code image to obtain the rotated first graphic code image.
When the method is implemented, for each point after rotation, the corresponding position coordinate of the point in the first graphic code image before rotation is determined.
And searching a plurality of adjacent points of the point in the first graphic code image before rotation based on the position coordinates of the point in the first graphic code image. And determining a nearest point with the smallest distance from the point among the plurality of nearest points, and then taking the pixel value of the nearest point with the smallest distance as the pixel value of the point in the rotated first graphic code image. According to the method, the pixel value of each pixel point can be determined in the constructed first graphic code image, so that the rotated first graphic code image is obtained.
For example, the following description will be given by taking the determination of the pixel value of the point M' in the first graphic code image after the rotation selection as an example.
Let M' have the coordinate of (i)a，ja) M' corresponds to a point M in the first graphic code image before rotation, assuming that the coordinate of M is (i)pf，jpf)，ipf、jpfAre all floating point numbers.
And searching adjacent points of the M points in the first graphic code image before rotation.
For example, 4 neighboring points are found, and the coordinates of the 4 neighboring points are (i)pi，jpi)、(ipi，jpi+1)、(ipi+1，jpi)、(ipi+1，jpi+1). The distances between the four neighboring points and the M point can then be calculated, and the neighboring point with the smallest distance from the M point is determined. Then, the pixel value of the adjacent point having the smallest distance from the M point is taken as the pixel value of the M' point.
According to the method for determining the pixel value of the point M', the pixel value of each point is determined in the constructed first graphic code image, which is not described herein again.
Step 203: and carrying out image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format.
The second processing module 102 in FIG. 1a can perform step 203.
Wherein the image processing may comprise one or a combination of more of:
brightness correction processing;
resolution correction processing;
and (5) blur restoration processing.
1) When the image processing is brightness correction processing, the first graphic code image may be subjected to image processing in the following manner
As shown in fig. 8, during the brightness correction process, the second processing module 102 may first perform brightness analysis on the first graphic code image (i.e., S801), then perform brightness adjustment on the first graphic code image (i.e., S802), and then output the second graphic code image with a brightness higher than the preset brightness average value and a data format of the second graphic code image.
S801: luminance analysis
The brightness analysis mainly analyzes whether the brightness of the bar code is reasonable or not, compares the brightness mean value of the first bar code image with a preset brightness mean value, and then calculates a brightness gain value required to be adjusted.
When the method is implemented, the brightness mean value of the first graphic code image can be determined, and the brightness gain value is determined based on the determined brightness mean value and the preset brightness mean value. The formula of the luminance gain value is as follows:
gain＝Lt/Lo
wherein L isoIs the mean value of brightness, L, of the first graphic code imagetThe gain is a brightness gain value for the preset brightness mean value.
S802: brightness adjustment
The brightness adjustment refers to brightness correction and contrast adjustment according to the brightness gain information obtained by the brightness analysis module, and the contrast adjustment and the data format conversion can be mapped by using a Gamma curve.
In implementation, the second processing module 102 may perform brightness correction on the first graphic code image based on the brightness gain value to obtain a second graphic code image with a data format of a second data format, so that a brightness average value of the second graphic code image after the brightness correction reaches a preset brightness average value.
The formula of the brightness correction is as follows:
p(ia，ja)＝p(ip，jp)*gain
wherein, the mean value of the brightness of the first graphic code image before being corrected is p (i)p，jp) The mean value of the brightness of the corrected first graphic code image is p (i)a，ja) And gain is the brightness gain value.
The contrast adjustment formula is as follows:
imggm(i，j)＝g(img(i，j))
wherein, imggm(i, j) is the first graphic code image after contrast adjustment and data format conversion, img (i, j) is the first graphic code image before contrast adjustment and data format conversion; g () is the mapping function of the Gamma curve.
2) When the image processing is resolution correction processing, the first graphic code image may be subjected to image processing in the following manner
As shown in fig. 9a, when the image processing is resolution correction processing, the second processing module 102 may include a first sub-processing unit 1021 and a second sub-processing unit 1022.
The first sub-processing unit 1021 is configured to input the first graphic code to the trained first neural network, so that the first neural network performs resolution enhancement and data format conversion on the first graphic code image, and outputs a second graphic code image with a resolution higher than that of the first graphic code image and a data format of the second graphic code image.
The second sub-processing unit 1022 is configured to train the first neural network by using the first graphic code image as a sample, and using the second graphic code image with the enhanced resolution and the data format of the second data format as a tag, so as to train parameters of each layer of the first neural network.
The second sub-processing unit 1022 may send the trained parameters of each layer to the first sub-processing unit 1021, and the first sub-processing unit 1021 may configure the first neural network based on the trained parameters of each layer, so as to obtain a trained neural network.
The network structure of the first neural network is described below.
The first neural network may include: a convolutional layer for performing a convolutional operation, a modified linear layer for performing a nonlinear operation, the convolutional layer and the modified linear layer being combined by a first combination.
For example, as shown in fig. 9b, a first neural network 900 includes: a convolutional layer 901, a convolutional layer 902, a modified linear layer 903, a convolutional layer 904, and a convolutional layer 905 connected in this order. The first neural network may have adaptive variations in each layer, for example, the convolution kernels of different convolutional layers may be different, and will not be described herein again. It is to be understood that the first neural network shown in fig. 9b is only an example, and is not particularly limited thereto, for example, convolutional layers, and/or pooling layers, and/or other layers may be reduced or added.
The first sub-processing unit 1021 may input the first graphic code image to a first neural network as shown in fig. 9b, and the first neural network may enhance the resolution of the first graphic code image by at least a convolution layer and a modified linear layer combined in a combination manner as shown in fig. 9b, and output a second graphic code image.
For the training of the first neural network, in order to optimize the deep neural network in advance, a large number of sample label pairs can be used for continuously training network parameters used in the training process of the first neural network until when a first graphic code image in a first data format is input, a second graphic code image which is higher in resolution than the first graphic code image and has a data format of a second data format can be output, and at the moment, the network parameters are output for the actual test and use of a third neural network. The sample is a first graphic code image in a first data format, and the label corresponding to the sample is a second graphic code image which has a resolution higher than that of the first graphic code image and has a data format of a second data format.
The training procedure for training the first neural network may include the steps of:
step 1: collecting training samples: and collecting the first graphic code image and the corresponding second graphic code image. Assume that n training sample label pairs { (x) have been obtained1，y1)，(x2，y2)，…，(xn，yn) In which xiRepresenting an input first graphic code image, yiRepresenting the corresponding second graphic code image.
Step 2: designing a structure of a first neural network; the network structure used for network training and the network structure used for testing are the same;
and step 3: initializing a training parameter; initializing network parameters of the first neural network structure, wherein random value initialization, fixed value initialization and the like can be adopted; setting training related parameters such as learning rate, iteration times and the like;
and 4, step 4: forward propagation; based on current network parameters, training sample x is adoptediForward propagating on the first neural network to obtain the output F (x) of the first neural networki) Calculating a Loss function Loss:
Loss＝(F(xi)-yi)2；
and 5: backward propagation: adjusting network parameters of the first neural network by utilizing backward propagation;
step 6: and (3) repeatedly iterating: and repeating the iteration steps 4 and 5 until the network is converged, and outputting the network parameters at the moment.
3) When the image processing is the blur restoration processing, the first graphic code image may be subjected to the image processing in the following manner
As shown in fig. 10a, when performing the blur restoration process, the second processing module 102 may perform blur estimation and sharpness restoration on the first graphic code image.
For example, the second processing module 102 may input the first graphic code image to the second neural network, and the first graphic code image is subjected to blur restoration by the second neural network.
The second neural network may include: a convolutional layer for performing a convolutional operation, a modified linear layer for performing a nonlinear operation, the convolutional layer and the modified linear layer being combined by a second combining means.
For example, as shown in fig. 10b, the second neural network 1000 sequentially includes: a convolutional layer 1001, a modified linear layer 1002, a convolutional layer 1003, a modified linear layer 1004, a convolutional layer 1005, a modified linear layer 1006, a convolutional layer 1007, a modified linear layer 1008, and a convolutional layer 1009 connected in this order. These convolutional layers and modified linear layers are combined in a combination manner as shown in fig. 10 b.
It should be noted that each layer of the second neural network may have adaptive variation, for example, the convolution kernels of different convolution layers may be different, and are not described herein again. It is to be understood that the second neural network shown in fig. 10b is only an example, and is not particularly limited thereto, for example, convolutional layers, and/or pooling layers, and/or other layers may be reduced or added.
When training the second neural network, the first graphic code image may be used as a sample, and the second graphic code image which is processed by the fuzzy recovery and has the data format of the second data format may be used as a label to train the second neural network.
Of course, the above fuzzy estimation and the clear restoration operation can be performed by two neural networks, which are only exemplary and not specifically limited.
It is also to be noted that the above-described image processing may further include various combinations of the luminance correction processing, the resolution correction processing, and the blur restoration processing. When a plurality of combinations are combined, only one format conversion is required in the processes of the luminance correction, the resolution correction processing, and the blur restoration processing.
For example, when the image processing includes resolution correction processing and blur restoration processing, only format conversion needs to be performed once.
For example, a first neural network may be trained to perform only resolution enhancement, and a second neural network may be trained to perform both fuzzy recovery and format conversion.
When the first neural network is trained, the first graphic code image is used as a sample, and the image with enhanced resolution is used as a label to train the first neural network, so that the trained first neural network only enhances the resolution of the first graphic code image.
When the second neural network is trained, the first graphic code image is used as a sample, and the second graphic code image which is subjected to fuzzy restoration processing and has a data format of a second data format is used as a label to train the second neural network, so that the trained second neural network can perform fuzzy restoration on the first graphic code image and can also perform data format conversion on the first graphic code image.
The second processing module 102 may input the first graphic code image into the first neural network, so that the first neural network performs resolution enhancement on the first graphic code image, and outputs the resolution-enhanced first graphic code image. Then, the second processing module 102 inputs the resolution-enhanced first graphic code image to the second neural network, so that the second neural network performs blur restoration and data format conversion on the resolution-enhanced first graphic code image to obtain a second graphic code image, and outputs the second graphic code image.
In addition, in the embodiment of the present application, after the first graphic code image is subjected to image processing to obtain a second graphic code image with a data format of a second data format, the image capturing device may further perform character recognition on the second graphic code image to obtain the recognition information. Or the image acquisition equipment can also send the second graphic code image to a graphic code recognition device so as to obtain recognition information by the graphic code recognition device through character recognition.
As can be seen from the above description, the image capturing device determines the position information of the graphic code from the captured first image in the first data format, intercepts the first graphic code image from the first image based on the position information, performs image processing on the first graphic code image, obtains the second graphic code image with the image quality higher than that of the first graphic code image and the data format different from that of the first data format, and outputs the second graphic code image.
On one hand, the first graphic code image can be positioned more accurately from the first data format because the first image in the first data format can keep more image information.
On the other hand, the method and the device also perform image processing on the first graphic code image which is cut from the first image in the first data format to obtain the second graphic code image which is higher in image quality and suitable for display and/or transmission in the data format, so that the image quality of the graphic code image transmitted or displayed by the image acquisition equipment is higher.
Referring to fig. 11, fig. 11 is a block diagram illustrating a graphic code determination apparatus according to an exemplary embodiment of the present application, which may include the following modules.
The first processing module 111 is configured to obtain location information of the first graphic code, and intercept a first graphic code image from a first image in a first data format based on the location information of the first graphic code.
A second processing module 112, configured to perform image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format.
Optionally, the second processing module 112, the image processing at least includes one or more of the following combinations:
brightness correction processing;
resolution correction processing;
and (5) blur restoration processing.
Optionally, if the image processing is brightness correction processing, the second processing module 112 is specifically configured to determine a brightness mean value of the first graphic code image when performing image processing on the first graphic code image; determining a brightness gain value based on the determined brightness mean value and the preset brightness mean value; and performing brightness correction on the first graphic code image based on the brightness gain value to obtain a second graphic code image with a data format of a second data format.
Optionally, if the image processing is resolution correction processing, the second processing module 112 is specifically configured to input the first graphic code image to a trained first neural network when performing image processing on the first graphic code image; the first neural network at least realizes the resolution enhancement of the first graphic code image through the convolution layer and the correction linear layer which are combined according to a first combination mode to obtain a second graphic code image; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation; and acquiring a second graphic code image output by the second neural network.
Optionally, if the image processing is a blur restoration processing, the second processing module 112 is specifically configured to input the first graphic code image to a trained second neural network when performing image processing on the first graphic code image; the second neural network realizes the fuzzy restoration of the first graphic code image to obtain a second graphic code image at least through a convolution layer and a correction linear layer which are combined according to a second combination mode; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation; and acquiring a second graphic code image which is output by the second neural network and is subjected to fuzzy restoration.
Optionally, the first processing module 111 includes: a first processing sub-module 1111 and a second processing sub-module 1112;
the first processing sub-module 1111 is configured to obtain location information of the first graphic code; the second processing sub-module 1112 is configured to intercept a first graphic code image from a first image in a first data format based on the location information of the first graphic code;
the first processing sub-module 1111 is specifically configured to perform preprocessing on the first image to obtain a second image capable of performing graphic code detection when acquiring the position information of the first graphic code; and determining the position information of the graphic code in the second image, and taking the position information determined in the second image as the position information of the graphic code in the first image.
Optionally, the first processing sub-module 1111, when determining the position information of the graphic code in the second image, is specifically configured to input the second image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for executing normalization processing, a convolution layer for executing convolution, a pooling layer for executing downsampling, a full-connection layer for executing feature synthesis and a frame regression layer for executing coordinate transformation; and taking the result output by the third neural network as the position information of the graphic code in the second image.
Optionally, the first processing sub-module 1111 is specifically configured to, when the first image is preprocessed to obtain the second image capable of performing the graphic code detection, preprocess the first image by using at least curve mapping to obtain the second image capable of performing the graphic code detection.
Optionally, the first processing sub-module 1111, when acquiring the position information of the first graphic code, is specifically configured to input the first image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for normalization processing, a convolution layer for convolution execution, a pooling layer for downsampling execution, a full-link layer for feature synthesis execution and a regression layer for coordinate transformation execution; and taking the output result of the third neural network as the position information of the first graphic code in the first image.
Optionally, the second processing sub-module 1112 is further specifically configured to detect whether a specified bottom edge of the first graphic code image coincides with the horizontal direction; if not, calculating the deviation angle of the appointed bottom edge of the first graphic code image and the horizontal direction based on the position coordinates of each vertex of the first graphic code image; and rotating the first graphic code image based on the position coordinates of each vertex of the first graphic code image and the deviation angle so as to enable the appointed bottom side of the first graphic code image to be consistent with the horizontal direction.
Optionally, the second processing module 112 is further specifically configured to perform character recognition on the second graphic code image to obtain recognition information; or sending the second graphic code image to a graphic code recognition device so that the graphic code recognition device performs character recognition to obtain recognition information.
Referring to fig. 12, fig. 12 is a hardware configuration diagram of an industrial camera according to an exemplary embodiment of the present disclosure.
The industrial camera includes: a communication interface 1201, a processor 1202, a machine-readable storage medium 1203, and a bus 1204; the communication interface 1201, the processor 1202, and the machine-readable storage medium 1203 are in communication with each other via a bus 1204. The processor 1202 may perform the graphical code determination method described above by reading and executing machine-executable instructions in the machine-readable storage medium 1203 corresponding to the graphical code determination control logic.
The machine-readable storage medium 1203 referred to herein may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: volatile memory, non-volatile memory, or similar storage media. In particular, the machine-readable storage medium 1203 may be a RAM (random Access Memory), a flash Memory, a storage drive (e.g., a hard disk drive), a solid state disk, any type of storage disk (e.g., a compact disk, a DVD, etc.), or similar storage medium, or a combination thereof.
The processor is caused by the machine executable instructions to perform:
acquiring position information of a first graphic code;
intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code;
performing image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format.
Optionally, the image processing includes at least one or more of the following combinations:
brightness correction processing;
resolution correction processing;
and (5) blur restoration processing.
Optionally, if the image processing is brightness correction processing, when the image processing is performed on the first graphic code image, the processor is caused by the machine executable instruction to perform:
determining the brightness mean value of the first graphic code image;
determining a brightness gain value based on the determined brightness mean value and the preset brightness mean value;
and performing brightness correction on the first graphic code image based on the brightness gain value to obtain a second graphic code image with a data format of a second data format.
Optionally, if the image processing is resolution correction processing, when the image processing is performed on the first graphic code image, the processor is caused by the machine executable instruction to perform:
inputting the first graphic code image to a trained first neural network; the first neural network at least realizes the resolution enhancement of the first graphic code image through the convolution layer and the correction linear layer which are combined according to a first combination mode to obtain a second graphic code image; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation;
and acquiring a second graphic code image output by the second neural network.
Optionally, if the image processing is a blur restoration processing, when the image processing is performed on the first graphic code image, the processor is caused by the machine executable instruction to perform:
inputting the first graphic code image to a trained second neural network; the second neural network realizes the fuzzy restoration of the first graphic code image to obtain a second graphic code image at least through a convolution layer and a correction linear layer which are combined according to a second combination mode; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation;
and acquiring a second graphic code image which is output by the second neural network and is subjected to fuzzy restoration.
Optionally, when obtaining the location information of the first graphic code, the processor is caused by the machine executable instructions to perform:
preprocessing the first image to obtain a second image capable of detecting the graphic code;
and determining the position information of the graphic code in the second image, and taking the position information determined in the second image as the position information of the graphic code in the first image.
Optionally, when the position information of the graphic code is determined in the second image, the processor is caused by the machine executable instructions to perform:
inputting the second image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for executing normalization processing, a convolution layer for executing convolution, a pooling layer for executing downsampling, a full-connection layer for executing feature synthesis and a frame regression layer for executing coordinate transformation;
and taking the result output by the third neural network as the position information of the graphic code in the second image.
Optionally, when the first image is preprocessed to obtain a second image capable of performing graphic code detection, the processor is caused by the machine executable instruction to perform:
and preprocessing the first image by at least adopting curve mapping to obtain a second image capable of detecting the graphic code.
Optionally, when obtaining the location information of the first graphic code, the processor is caused by the machine executable instructions to perform:
inputting the first image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for normalization processing, a convolution layer for convolution execution, a pooling layer for downsampling execution, a full-link layer for feature synthesis execution and a regression layer for coordinate transformation execution;
and taking the output result of the third neural network as the position information of the first graphic code in the first image.
Optionally, before the image processing is performed on the first graphic code image to obtain a second graphic code image with a data format of a second data format, the processor is caused by the machine executable instructions to perform:
detecting whether the appointed bottom edge of the first graphic code image is overlapped with the horizontal direction;
if not, calculating the deviation angle of the appointed bottom edge of the first graphic code image and the horizontal direction based on the position coordinates of each vertex of the first graphic code image;
and rotating the first graphic code image based on the position coordinates of each vertex of the first graphic code image and the deviation angle so as to enable the appointed bottom side of the first graphic code image to be consistent with the horizontal direction.
Optionally, the processor is further caused by the machine executable instructions to perform:
performing character recognition on the second graphic code image to obtain recognition information;
alternatively, the first and second electrodes may be,
and sending the second graphic code image to a graphic code recognition device so as to obtain recognition information by the graphic code recognition device through character recognition.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims (13)

1. A method for determining a graphic code is applied to an image acquisition device, and comprises the following steps:
acquiring position information of a first graphic code;
intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code;
performing image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format.
2. The method of claim 1, wherein the image processing comprises at least one or more of the following in combination:
brightness correction processing;
resolution correction processing;
and (5) blur restoration processing.
3. The method according to claim 2, wherein if the image processing is a luminance correction processing, the image processing on the first graphic code image comprises:
determining the brightness mean value of the first graphic code image;
determining a brightness gain value based on the determined brightness mean value and the preset brightness mean value;
and performing brightness correction on the first graphic code image based on the brightness gain value to obtain a second graphic code image with a data format of a second data format.
4. The method according to claim 2, wherein if the image processing is resolution correction processing, the image processing on the first graphic code image comprises:
inputting the first graphic code image to a trained first neural network; the first neural network at least realizes the resolution enhancement of the first graphic code image through the convolution layer and the correction linear layer which are combined according to a first combination mode to obtain a second graphic code image; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation;
and acquiring a second graphic code image output by the second neural network.
5. The method according to claim 2, wherein if the image processing is a blur restoration processing, the image processing on the first graphic code image comprises:
inputting the first graphic code image to a trained second neural network; the second neural network realizes the fuzzy restoration of the first graphic code image to obtain a second graphic code image at least through a convolution layer and a correction linear layer which are combined according to a second combination mode; the convolution layer is used for executing convolution operation, and the modified linear layer is used for executing nonlinear operation;
and acquiring a second graphic code image which is output by the second neural network and is subjected to fuzzy restoration.
6. The method according to claim 1, wherein the obtaining the location information of the first graphic code comprises:
preprocessing the first image to obtain a second image capable of detecting the graphic code;
and determining the position information of the graphic code in the second image, and taking the position information determined in the second image as the position information of the graphic code in the first image.
7. The method of claim 6, wherein determining the position information of the graphic code in the second image comprises:
inputting the second image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for executing normalization processing, a convolution layer for executing convolution, a pooling layer for executing downsampling, a full-connection layer for executing feature synthesis and a frame regression layer for executing coordinate transformation;
and taking the result output by the third neural network as the position information of the graphic code in the second image.
8. The method of claim 6, wherein the pre-processing the first image to obtain a second image capable of detecting a graphic code comprises:
and preprocessing the first image by at least adopting curve mapping to obtain a second image capable of detecting the graphic code.
9. The method according to claim 1, wherein the obtaining the location information of the first graphic code comprises:
inputting the first image to a trained third neural network; the third neural network realizes the positioning and output of the first graphic code at least through a normalization layer for normalization processing, a convolution layer for convolution execution, a pooling layer for downsampling execution, a full-link layer for feature synthesis execution and a regression layer for coordinate transformation execution;
and taking the output result of the third neural network as the position information of the first graphic code in the first image.
10. The method according to claim 1, wherein before the image processing of the first graphic code image to obtain the second graphic code image with the data format of the second data format, the method further comprises:
detecting whether the appointed bottom edge of the first graphic code image is overlapped with the horizontal direction;
if not, calculating the deviation angle of the appointed bottom edge of the first graphic code image and the horizontal direction based on the position coordinates of each vertex of the first graphic code image;
and rotating the first graphic code image based on the position coordinates of each vertex of the first graphic code image and the deviation angle so as to enable the appointed bottom side of the first graphic code image to be consistent with the horizontal direction.
11. The method of claim 1, further comprising:
performing character recognition on the second graphic code image to obtain recognition information;
alternatively, the first and second electrodes may be,
and sending the second graphic code image to a graphic code recognition device so as to obtain recognition information by the graphic code recognition device through character recognition.
12. A graphic code determination device is applied to an image acquisition device, and comprises:
the first processing module is used for acquiring the position information of a first graphic code and intercepting a first graphic code image from a first image in a first data format based on the position information of the first graphic code;
the second processing module is used for carrying out image processing on the first graphic code image to obtain a second graphic code image with a data format of a second data format; the second data format is different from the first data format.
13. An industrial camera comprising a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor, the processor being caused by the machine-executable instructions to perform the method of any one of claims 1 to 11.
CN201910651699.0A 2019-07-18 2019-07-18 Graphic code determining method and device and industrial camera Active CN112241640B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910651699.0A CN112241640B (en) 2019-07-18 2019-07-18 Graphic code determining method and device and industrial camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910651699.0A CN112241640B (en) 2019-07-18 2019-07-18 Graphic code determining method and device and industrial camera

Publications (2)

Publication Number Publication Date
CN112241640A true CN112241640A (en) 2021-01-19
CN112241640B CN112241640B (en) 2023-06-30

Family

ID=74168019

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910651699.0A Active CN112241640B (en) 2019-07-18 2019-07-18 Graphic code determining method and device and industrial camera

Country Status (1)

Country Link
CN (1) CN112241640B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105447529A (en) * 2015-12-30 2016-03-30 商汤集团有限公司 Costume detection and attribute value identification method and system
CN106980854A (en) * 2017-03-29 2017-07-25 珠海习悦信息技术有限公司 Number-plate number recognition methods, device, storage medium and processor
CN107483809A (en) * 2017-07-20 2017-12-15 努比亚技术有限公司 A kind of image capturing method, terminal and computer-readable recording medium
CN108121982A (en) * 2016-11-30 2018-06-05 杭州海康机器人技术有限公司 The acquisition methods and device of face single image
CN108229375A (en) * 2017-12-29 2018-06-29 百度在线网络技术（北京）有限公司 For detecting the method and apparatus of facial image
CN108563559A (en) * 2018-03-12 2018-09-21 平安普惠企业管理有限公司 A kind of test method of identifying code, device, terminal device and storage medium
CN108921782A (en) * 2018-05-17 2018-11-30 腾讯科技（深圳）有限公司 A kind of image processing method, device and storage medium
CN109409161A (en) * 2018-10-22 2019-03-01 腾讯科技（深圳）有限公司 Graphic code recognition methods, device, terminal and storage medium
CN109753838A (en) * 2018-12-12 2019-05-14 深圳市三宝创新智能有限公司 Two-dimensional code identification method, device, computer equipment and storage medium
CN109784186A (en) * 2018-12-18 2019-05-21 深圳云天励飞技术有限公司 A kind of pedestrian recognition methods, device, electronic equipment and computer readable storage medium again
CN109815765A (en) * 2019-01-21 2019-05-28 东南大学 A kind of method and device for extracting the business license information containing two dimensional code
CN109978792A (en) * 2019-03-28 2019-07-05 厦门美图之家科技有限公司 A method of generating image enhancement model

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105447529A (en) * 2015-12-30 2016-03-30 商汤集团有限公司 Costume detection and attribute value identification method and system
CN108121982A (en) * 2016-11-30 2018-06-05 杭州海康机器人技术有限公司 The acquisition methods and device of face single image
CN106980854A (en) * 2017-03-29 2017-07-25 珠海习悦信息技术有限公司 Number-plate number recognition methods, device, storage medium and processor
CN107483809A (en) * 2017-07-20 2017-12-15 努比亚技术有限公司 A kind of image capturing method, terminal and computer-readable recording medium
CN108229375A (en) * 2017-12-29 2018-06-29 百度在线网络技术（北京）有限公司 For detecting the method and apparatus of facial image
CN108563559A (en) * 2018-03-12 2018-09-21 平安普惠企业管理有限公司 A kind of test method of identifying code, device, terminal device and storage medium
CN108921782A (en) * 2018-05-17 2018-11-30 腾讯科技（深圳）有限公司 A kind of image processing method, device and storage medium
CN109409161A (en) * 2018-10-22 2019-03-01 腾讯科技（深圳）有限公司 Graphic code recognition methods, device, terminal and storage medium
CN109753838A (en) * 2018-12-12 2019-05-14 深圳市三宝创新智能有限公司 Two-dimensional code identification method, device, computer equipment and storage medium
CN109784186A (en) * 2018-12-18 2019-05-21 深圳云天励飞技术有限公司 A kind of pedestrian recognition methods, device, electronic equipment and computer readable storage medium again
CN109815765A (en) * 2019-01-21 2019-05-28 东南大学 A kind of method and device for extracting the business license information containing two dimensional code
CN109978792A (en) * 2019-03-28 2019-07-05 厦门美图之家科技有限公司 A method of generating image enhancement model

Also Published As

Publication number Publication date
CN112241640B (en) 2023-06-30

Similar Documents

Publication Publication Date Title
US9405960B2 (en) Face hallucination using convolutional neural networks
US11882357B2 (en) Image display method and device
CN109410207B (en) NCC (non-return control) feature-based unmanned aerial vehicle line inspection image transmission line detection method
JP4468442B2 (en) Imaging system performance measurement
WO2018105028A1 (en) Inspection device and inspection method
CN108965742B (en) Special-shaped screen display method and device, electronic equipment and computer readable storage medium
JP2018084982A (en) Image processing apparatus, image processing method, and program
JP6688277B2 (en) Program, learning processing method, learning model, data structure, learning device, and object recognition device
CN111429533B (en) Camera lens distortion parameter estimation device and method
US20160275682A1 (en) Machine vision image sensor calibration
US11875485B2 (en) Compensating for geometric distortion of images in constrained processing environments
WO2014070489A1 (en) Recursive conditional means image denoising
US20220398698A1 (en) Image processing model generation method, processing method, storage medium, and terminal
US20230127009A1 (en) Joint objects image signal processing in temporal domain
JP2021179833A (en) Information processor, method for processing information, and program
CN113379609B (en) Image processing method, storage medium and terminal equipment
US10373299B1 (en) Compensating for geometric distortion of images in constrained processing environments
CN110555877B (en) Image processing method, device and equipment and readable medium
US20120038785A1 (en) Method for producing high resolution image
JP7443030B2 (en) Learning method, program, learning device, and method for manufacturing learned weights
CN112241640B (en) Graphic code determining method and device and industrial camera
Wang et al. The influence of chromatic aberration on demosaicking
WO2013011797A1 (en) Degradation restoration system, degradation restoration method and program
CN112241670B (en) Image processing method and device
CN113379610A (en) Training method of image processing model, image processing method, medium, and terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant