CN110766689A - Method and device for detecting article image defects based on convolutional neural network - Google Patents

Method and device for detecting article image defects based on convolutional neural network Download PDF

Info

Publication number
CN110766689A
CN110766689A CN201911075959.0A CN201911075959A CN110766689A CN 110766689 A CN110766689 A CN 110766689A CN 201911075959 A CN201911075959 A CN 201911075959A CN 110766689 A CN110766689 A CN 110766689A
Authority
CN
China
Prior art keywords
image
threshold segmentation
fixed
size
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911075959.0A
Other languages
Chinese (zh)
Inventor
周洪峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Microproducts To Mdt Infotech Ltd
Original Assignee
Shenzhen Microproducts To Mdt Infotech Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Microproducts To Mdt Infotech Ltd filed Critical Shenzhen Microproducts To Mdt Infotech Ltd
Priority to CN201911075959.0A priority Critical patent/CN110766689A/en
Publication of CN110766689A publication Critical patent/CN110766689A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a method and a device for detecting image defects of an article based on a convolutional neural network, wherein the method comprises the following steps: obtaining pixels of an article image, cutting the pixels into fixed-size images, extracting pixel values of the fixed-size images and converting the pixel values into data matrixes of image pixels; performing gray level normalization, filtering processing and threshold segmentation processing on the fixed-size image according to the data matrix to obtain preprocessed image data; training according to the preprocessed image data and the article image defect type thereof to obtain a convolutional neural network for article image defect detection; and acquiring an article image to be detected to obtain prediction data of the article image to be detected, and substituting the prediction data of the article image to be detected into the convolutional neural network to obtain a defect detection result of the article image to be detected. The invention can realize accurate, efficient and low-cost detection of the image defects of the articles.

Description

Method and device for detecting article image defects based on convolutional neural network
Technical Field
The present application relates to the field of article image defect detection technologies, and in particular, to a method and an apparatus for detecting an article image defect based on a convolutional neural network.
Background
The products produced by production and processing may have bad defects, the detection and the selection of the bad defect products from the products are particularly important, and the traditional manual detection of the defects of the products consumes labor and cannot ensure that the bad defect products are completely detected. Thus, although image defect detection by image recognition is performed on an article, the image defect detection method has many types of article defects, has various image features of the same type of defects, and makes it difficult to accurately detect various defects of defective images of the article by the conventional image defect detection method, and has low detection efficiency.
The method for detecting the image defects of the article has the advantages that one of key points is the setting of the image features of the article, the image features are extracted and compared to obtain the image defects of the article after the image of the article is obtained and processed, in the process, the accuracy of the setting of the image features of the article and the accuracy of image recognition are important factors influencing the accuracy of the detection of the image defects of the article, the characteristic factors need to be set by professionals according to a large amount of experience, the cost of the image detection of the article is improved, and the accuracy of the image detection of the article cannot be guaranteed.
The existing method for detecting the defects of the article image is to judge whether the article image has defects according to the comparison of a pixel threshold value and a pixel range outside a conventional pixel threshold value after the article image is processed, and further judge whether the article is a qualified product, wherein the judging method needs to set a threshold value for judgment in advance, the setting of a threshold value judging value has no standardized and normalized setting method, and the qualification rate of the product cannot be ensured if the set threshold value is too large and too many failed samples exist; if the threshold is set too small, a lot of qualified products will be discarded, increasing the cost of product inspection.
Therefore, how to provide a scheme for detecting the image defect of the article, which is accurate, efficient and low in cost, is a technical problem to be solved in the field.
Disclosure of Invention
The application aims to provide a method and a device for detecting the image defects of articles based on a convolutional neural network, and the method and the device can be used for solving the technical problems that in the prior art, the image defects of the articles are not high in accuracy, low in efficiency and high in cost, and the qualified articles are easy to run off.
In order to achieve the above object, the present application provides a method for detecting an image defect of an article based on a convolutional neural network, comprising:
obtaining pixels of an article image, cutting the article image into fixed-size images according to the corresponding relation between preset pixels and image sizes, extracting pixel values of the fixed-size images and converting the pixel values into data matrixes of image pixels;
carrying out gray scale normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave; performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data;
training according to the preprocessed image data and the article image defect type thereof to obtain a convolutional neural network for article image defect detection;
acquiring a preprocessing strategy of selecting corresponding pixel cutting, gray normalization, filtering and threshold segmentation for the image defect type of the to-be-detected article image; and carrying out pixel cutting, gray level normalization, filtering and threshold segmentation on the to-be-detected article image according to the preprocessing strategy to obtain prediction data of the to-be-detected article image, and substituting the prediction data of the to-be-detected article image into the convolutional neural network to obtain a defect detection result of the to-be-detected article image.
Optionally, performing gray scale normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image is as follows:
acquiring a pixel value of each pixel in the fixed-size image according to the data matrix;
carrying out gray scale normalization processing on the pixel values of the fixed-size image according to the following formula to obtain pixel values with enhanced contrast of the fixed-size image, wherein the formula is as follows:
Figure BDA0002262444720000021
wherein, IoutFor the converted gray values, IinFor the grey value of each pixel of the original image, IminIs the minimum pixel value, I, of the original imagemaxIs the maximum pixel value of the original image.
Optionally, the fixed size image data with normalized gray scale is filtered by a gaussian kernel function modulated by a sinusoidal plane wave, and the filtering is performed by:
and (3) obtaining a complex expression by using a Gaussian kernel function modulated by sinusoidal plane waves for the image data with the fixed size and with the gray level normalized:
Figure BDA0002262444720000031
wherein the content of the first and second substances,in order to obtain new pixel displacement image data, x and y are pixel coordinates, and x' is xcos θ + ysin θ; y' ═ xsin θ + ycos θ; wavelength lambda represents the wavelength parameter of the cosine function in the filter kernel function, direction theta represents the direction of the parallel strips in the filter kernel, and phase shiftRepresenting phase parameters of a cosine function in a filter kernel function, wherein the length-width ratio gamma is a space aspect ratio, determining the ellipticity of the shape of the filter function, and sigma represents the standard deviation of a Gaussian factor of the filter function;
optionally, in the filter function, a relation between a half-response spatial frequency bandwidth b and gaussian factors σ and a wavelength λ is as follows:
Figure BDA0002262444720000033
optionally, the threshold segmentation is performed on the filtered data of the fixed-size image to obtain preprocessed image data, and the preprocessing step includes:
obtaining image data of a tested defect image after filtration, and respectively carrying out maximum inter-class variance threshold segmentation, maximum entropy method global threshold segmentation and local dynamic threshold segmentation;
comparing the effects of the results of the maximum inter-class variance threshold segmentation, the maximum entropy global threshold segmentation and the local dynamic threshold segmentation with the defects of the actual image, and selecting the threshold segmentation closest to the actual situation as a preprocessing segmentation strategy;
and performing threshold segmentation on the filtered data of the image with the fixed size according to the preprocessing segmentation strategy to obtain preprocessed image data.
In another aspect, the present invention further provides an apparatus for detecting an image defect of an article based on a convolutional neural network, including: the system comprises an image pixel cutter, an image preprocessor, a convolutional neural network creator and an article image defect detector; wherein the content of the first and second substances,
the image pixel cutter is connected with the image preprocessor to obtain pixels of an article image, cuts the image into an image with a fixed size according to the corresponding relation between preset pixels and the image size, extracts pixel values of the image with the fixed size and converts the pixel values into a data matrix of image pixels;
the image preprocessor is connected with the image pixel cutter and the convolutional neural network creator and is used for carrying out gray level normalization processing on the fixed-size image according to the data matrix so as to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave; performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data;
the convolutional neural network creator is connected with the image preprocessor and the article image defect detector and is used for training according to the preprocessed image data and the article image defect types thereof to obtain a convolutional neural network for article image defect detection;
the article image defect detector is connected with the image pixel cutter, the image preprocessor and the convolutional neural network creator to obtain a preprocessing strategy of selecting corresponding pixel cutting, gray normalization, filtering and threshold segmentation for the image defect type of the article image to be detected; and carrying out pixel cutting, gray level normalization, filtering and threshold segmentation on the to-be-detected article image according to the preprocessing strategy to obtain prediction data of the to-be-detected article image, and substituting the prediction data of the to-be-detected article image into the convolutional neural network to obtain a defect detection result of the to-be-detected article image.
Optionally, wherein the image preprocessor comprises: a gray scale normalization processor and an enhanced image data processor; wherein the content of the first and second substances,
the gray normalization processor is connected with the image pixel cutter and the enhanced image data processor and is used for acquiring the pixel value of each pixel in the image with the fixed size according to the data matrix;
carrying out gray scale normalization processing on the pixel values of the fixed-size image according to the following formula to obtain pixel values with enhanced contrast of the fixed-size image, wherein the formula is as follows:
Figure BDA0002262444720000041
wherein, IoutFor the converted gray values, IinFor the grey value of each pixel of the original image, IminIs the minimum pixel value, I, of the original imagemaxThe maximum pixel value of the original image is taken;
the enhanced image data processor is connected with the gray level normalization processor and the convolutional neural network creator and is used for filtering the fixed-size image data with the gray level normalization by a Gaussian kernel function modulated by a sine plane wave; and performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data.
Optionally, wherein the image preprocessor comprises: a gray scale normalization processor and an image data filtering processor; wherein the content of the first and second substances,
the gray normalization processor is connected with the image pixel cutter and the image data filtering processor, and performs gray normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image;
the image data filtering processor is connected with the gray level normalization processor and the convolutional neural network creator, and obtains a complex expression by using a Gaussian kernel function modulated by sinusoidal plane waves for the image data with fixed size with gray level normalization:
Figure BDA0002262444720000051
wherein the content of the first and second substances,in order to obtain new pixel displacement image data, x and y are pixel coordinates, and x' is xcos θ + ysin θ; y' ═ xsin θ + ycos θ; wavelength lambda represents the wavelength parameter of the cosine function in the filter kernel function, direction theta represents the direction of the parallel strips in the filter kernel, and phase shift
Figure BDA0002262444720000054
Representing phase parameters of a cosine function in a filter kernel function, wherein the length-width ratio gamma is a space aspect ratio, determining the ellipticity of the shape of the filter function, and sigma represents the standard deviation of a Gaussian factor of the filter function;
filtering the new pixel displacement image data to obtain the data of the fixed-size image after filtering, wherein the formula is as follows:
Figure BDA0002262444720000053
wherein, S (x, y) is data after fixed-size image filtering;
and performing threshold segmentation on the filtered data of the fixed-size image to obtain preprocessed image data.
Optionally, wherein, in the image data filtering processor, a relation between a half-response spatial frequency bandwidth b and gaussian factors σ and a wavelength λ is as follows:
Figure BDA0002262444720000061
optionally, wherein the image preprocessor comprises: an enhancement and filtering processor and a threshold segmentation processor; wherein the content of the first and second substances,
the enhancement and filtering processor is connected with the image pixel cutter and the threshold segmentation processor, and performs gray level normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave;
the threshold segmentation processor is connected with the enhancing and filtering processor and the convolutional neural network creator, and comprises: the device comprises a pre-threshold segmentation processing unit, a threshold segmentation strategy processing unit and a threshold segmentation processing unit; the pre-threshold segmentation processing unit is connected with the enhancement and filtering processor and the threshold segmentation strategy processing unit, and is used for obtaining image data of the tested defect image after being filtered and respectively carrying out maximum inter-class variance threshold segmentation, maximum entropy global threshold segmentation and local dynamic threshold segmentation;
the threshold segmentation strategy processing unit is connected with the pre-threshold segmentation processing unit and the threshold segmentation processing unit, compares the effects of the results of the maximum inter-class variance threshold segmentation, the maximum entropy global threshold segmentation and the local dynamic threshold segmentation with the defects of the actual image, and selects the threshold segmentation closest to the actual situation as the pre-processing segmentation strategy;
and the threshold segmentation processing unit is connected with the threshold segmentation strategy processing unit and the convolutional neural network creator, and performs threshold segmentation on the filtered data of the image with the fixed size according to the preprocessing segmentation strategy to obtain preprocessed image data.
The method and the device for detecting the image defects of the article based on the convolutional neural network have the following beneficial effects that:
(1) according to the method and the device for detecting the image defects of the article based on the convolutional neural network, the image defects of the article are trained through preprocessing, the convolutional neural network is trained and established, the image data of the obstructed condition is predicted according to the existing image data, the trained convolutional neural network model is utilized to predict the future image defect condition of the article, and accurate, efficient and low-cost image defect detection of the article can be realized.
(2) According to the method and the device for detecting the image defects of the article based on the convolutional neural network, the image defects are identified by adopting the article image data processing and classification prediction and the convolutional neural network, so that more accurate qualified probability can be given according to historical image data, errors in image defect judgment are reduced, the original mode of threshold division by one cutting is changed, the accuracy of product inspection is greatly improved, and the cost loss caused by unnecessary product inspection errors is greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a schematic flow chart illustrating a method for detecting an image defect of an article based on a convolutional neural network according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a schematic flow chart of the method for detecting the image defect of an article based on the convolutional neural network according to an embodiment of the present invention;
fig. 3 is a schematic flow chart illustrating the process of detecting the defect of the object image by the method for detecting the defect of the object image by the user based on the convolutional neural network according to the embodiment of the present invention;
FIG. 4 is a flowchart illustrating a second method for detecting defects in an image of an article based on a convolutional neural network according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a third method for detecting an image defect of an article based on a convolutional neural network according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a time-domain convolution kernel image corresponding to different wavelengths of the same convolution kernel according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating a time domain convolution kernel image corresponding to the same convolution kernel in different parallel line directions according to an embodiment of the present invention;
FIG. 8 is a flowchart illustrating a fourth method for detecting defects in an image of an article based on a convolutional neural network according to an embodiment of the present invention;
FIG. 9 is a schematic structural diagram of an apparatus for detecting an image defect of an article based on a convolutional neural network according to an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of a second apparatus for detecting an image defect of an article based on a convolutional neural network according to an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of a third apparatus for detecting an image defect of an article based on a convolutional neural network according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of a fourth apparatus for detecting an image defect of an article based on a convolutional neural network according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
As shown in fig. 1 to 3, fig. 1 is a schematic flowchart of a method for detecting an image defect of an article based on a convolutional neural network in this embodiment; FIG. 2 is a schematic flow chart illustrating a schematic flow chart of the method for detecting the image defect of an article based on the convolutional neural network according to the present embodiment; fig. 3 is a schematic flow chart illustrating the method for detecting the defect of the object image by the user side based on the convolutional neural network according to the embodiment. In the embodiment, the convolutional neural network is used for identifying the image defects, so that more accurate qualified probability can be given according to historical image data, and image defect judgment errors are reduced. Specifically, the method for detecting the image defect of the article based on the convolutional neural network comprises the following steps:
step 101, obtaining pixels of an article image, cutting the article image into an image with a fixed size according to a corresponding relation between preset pixels and the image size, extracting pixel values of the image with the fixed size, and converting the pixel values into a data matrix of image pixels.
The method comprises the steps of obtaining pixels of an article Image, reading RGB values (Image pixel values) of the Image, converting the Image into numerical matrix data through an Image method in a PIL package of python, converting the RGB value of each pixel of the Image into a three-dimensional or two-dimensional list through the method, and converting one Image into a data matrix of Image pixel points.
102, performing gray level normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave; and performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data.
Different image preprocessing methods can be set according to different image defects, for example, the following metal surface defects are mainly provided, which are respectively: and setting a corresponding image preprocessing strategy aiming at the metal surface by surface spots, surface rolling-in oxide skin and surface scratches. The main steps of image preprocessing are as follows: contrast enhancement, filtering processing, threshold segmentation and morphology processing.
And 103, training according to the preprocessed image data and the article image defect type thereof to obtain a convolutional neural network for article image defect detection.
104, acquiring an image defect type of an image of an article to be detected, and selecting a corresponding preprocessing strategy of pixel cutting, gray level normalization, filtering and threshold segmentation; and carrying out pixel cutting, gray level normalization, filtering and threshold segmentation on the to-be-detected object image according to a preprocessing strategy to obtain prediction data of the to-be-detected object image, and substituting the prediction data of the to-be-detected object image into a convolutional neural network to obtain a defect detection result of the to-be-detected object image.
In some optional embodiments, as shown in fig. 4, which is a schematic flow chart of a second method for detecting an image defect of an article based on a convolutional neural network in this embodiment, different from that in fig. 1, a gray-scale normalization process is performed on a fixed-size image according to a data matrix to enhance a contrast of the fixed-size image, where the method includes:
step 401, obtaining a pixel value of each pixel in the fixed-size image according to the data matrix.
Step 402, performing gray level normalization processing on the pixel value of the fixed-size image according to the following formula to obtain the pixel value with enhanced contrast of the fixed-size image, wherein the formula is as follows:
Figure BDA0002262444720000091
wherein, IoutFor the converted gray values, IinFor the grey value of each pixel of the original image, IminIs the minimum pixel value, I, of the original imagemaxIs the maximum pixel value of the original image.
The gray normalization can enable the gray value of an image pixel to be distributed between 0 and 255, and interference on subsequent processing due to insufficient image contrast (unbalanced image pixel brightness distribution) is avoided.
In some optional embodiments, as shown in fig. 5, which is a schematic flow chart of a third method for detecting an image defect of an article based on a convolutional neural network in this embodiment, different from fig. 1, the fixed-size image data with normalized gray scale is subjected to a filtering process by a gaussian kernel function modulated by a sinusoidal plane wave, and the method includes:
step 501, obtaining a complex expression by using a gaussian kernel function modulated by a sinusoidal plane wave for the fixed-size image data with the normalized gray level:
Figure BDA0002262444720000101
wherein the content of the first and second substances,
Figure BDA0002262444720000102
in order to obtain new pixel displacement image data, x and y are pixel coordinates, and x' is xcos θ + ysin θ; y' ═ xsin θ + ycos θ; wavelength lambda represents the wavelength parameter of the cosine function in the filter kernel function, direction theta represents the direction of the parallel strips in the filter kernel, and phase shift
Figure BDA0002262444720000106
Representing phase parameters of a cosine function in a filter kernel function, wherein the length-width ratio gamma is a space aspect ratio, determining the ellipticity of the shape of the filter function, and sigma represents the standard deviation of a Gaussian factor of the filter function;
step 502, filtering according to the new pixel displacement image data to obtain data after filtering of the image with fixed size, wherein the formula is as follows:
Figure BDA0002262444720000103
wherein S (x, y) is data after fixed size image filtering.
The filtered image can be divided into a real part and an imaginary part, wherein the real part can carry out smooth filtering on the image, the imaginary part can be used for edge detection, and the filtering convolution kernel can simultaneously carry out filtering smoothing operation and texture edge sharpening, so that the effect of killing two birds with one stone can be obtained.
Optionally, in the filter function, the relationship between the half-response spatial frequency bandwidth b and the gaussian factor σ and the wavelength λ is as follows:
Figure BDA0002262444720000104
conventional filtering algorithms, such as mean filtering, gaussian filtering, etc., often cannot produce good effects in specific application fields, and a suitable special filtering algorithm is required for the specific application fields. The filter commonly used in the field of texture extraction is a Gabor filter, which is mainly based on the wavelet analysis theory and is obtained by performing scale transformation and rotation transformation on a mother wavelet multiplied by a Gaussian function and a complex exponential function. It has been found that Gabor filters are well suited for texture expression and separation, and in the spatial domain, a two-dimensional Gabor filter is a gaussian kernel modulated by a sinusoidal plane wave.
The value of the wavelength λ is specified in pixels, and is usually 2 or more, but not more than one fifth of the input image size; the effective value of the direction theta is a real number from 0 to 360 degrees; phase shift
Figure BDA0002262444720000105
The value range of (a) is-180 degrees to 180 degrees, wherein the equations corresponding to 0 degrees and 180 degrees are symmetrical with the origin, and the equations of-90 degrees and 90 degrees are respectively in central symmetry with the origin; the aspect ratio γ determines the ellipticity of the shape of the Gabor function, which is circular when γ is 1; when gamma is<1, the shape is elongated in the direction parallel to the stripe direction, and usually this value is 0.5.
Optionally, the significance of each parameter in the Gabor filter kernel is as follows:
(1) λ is the wavelength of the modulation sine function, which determines the proportion of the effective non-zero area of the convolution kernel to the whole kernel, and can extract different frequency component information of the image, generally > 2. As shown in fig. 6, a schematic diagram of a time domain convolution kernel image corresponding to different λ of the same convolution kernel is shown.
(2) Theta represents the parallel line direction of the Gabor filter, and the parameter determines the filtering direction selection characteristic of the Gabor filter, namely, the sharpening and smoothing effects on different directions are different. As shown in fig. 7, a schematic diagram of a time domain convolution kernel image showing the same convolution kernel but different θ corresponds to each other is shown. The two parameters, wavelength λ and parallel line direction θ, will determine the orientation and frequency selective characteristics of the Gabor filter. Other parameters of the filtering process are: phase offset, spatial scaling factor and bandwidth.
The Gabor filter has the following advantages:
(1) the method is insensitive to illumination change and can provide good adaptability to the illumination change, so that different metal detection hardware systems have different background light intensities due to light source difference, and the method has a good filtering effect and strong robustness. (2) Compared with the traditional Fourier transform, the Gabor wavelet transform has good time-frequency localization characteristics. That is, the direction, the base band width, and the center frequency of the Gabor filter can be adjusted very easily, and the resolving power of the signal in the time domain and the frequency domain can be satisfied at the same time. (3) Gabor filtering has a multi-resolution characteristic, i.e., zoom capability. The method adopts a multi-channel filtering technology, applies a group of Gabor wavelets with different frequency domain characteristics to image transformation, and each channel can obtain certain local characteristics of an input image, so that the image can be analyzed on different thickness granularities according to requirements. Considering that different metal defects have large differences in size (smaller roll-in scale, larger scratches), they can extract different defects by changing the wavelength of the modulating sinusoid, corresponding to different frequency components in the image. (4) The Gabor filter can filter different directions by changing, so that whether the texture has directional anisotropy or not can be judged, and the types of defects (scratches have directional anisotropy, and spots and rolled-in scale have directional isotropy) can be judged to a certain extent according to the texture. (5) The Gabor filtering can enhance image features (sharpening of imaginary parts) such as edges, peaks, valleys and ridge outlines and the like while filtering and denoising, which is equivalent to enhancing metal surface defect information, and the wavelet characteristics of the Gabor filtering indicate that the Gabor filtering result is a powerful tool for describing local gray level distribution of an image, so that the Gabor filtering can be used for extracting texture information of the image. In summary, for processing the metal surface defect texture, the Gabor filtering is a good choice, the filtering characteristic of the Gabor filtering is matched with the surface defect characteristic, and the Gabor operator can be used for spatial filtering.
In some optional embodiments, as shown in fig. 8, which is a schematic flow chart of a fourth method for detecting an image defect of an article based on a convolutional neural network in this embodiment, different from that in fig. 1, threshold segmentation is performed on data of a filtered fixed-size image to obtain preprocessed image data, where:
and 801, obtaining the image data after the tested defect image is filtered, and respectively performing maximum inter-class variance threshold segmentation, maximum entropy global threshold segmentation and local dynamic threshold segmentation.
And step 802, comparing the effects of the results of the maximum inter-class variance threshold segmentation, the maximum entropy global threshold segmentation and the local dynamic threshold segmentation with the defects of the actual image, and selecting the threshold segmentation closest to the actual situation as a preprocessing segmentation strategy.
And 803, performing threshold segmentation on the filtered data of the fixed-size image according to a preprocessing segmentation strategy to obtain preprocessed image data.
Thresholding segmentation (maximum inter-class variance threshold segmentation) is the most common traditional image segmentation method, and becomes the most basic and widely applied segmentation technology in image segmentation because of simple implementation, small calculation amount and stable performance. In order to extract surface defects better, a proper threshold segmentation algorithm is crucial, and common ways for threshold segmentation are a global threshold method, a local threshold method and a multi-threshold segmentation method. Considering the defects to be extracted, a single threshold segmentation algorithm is used. The algorithm can contend for different thresholds for different images, and a dynamic threshold-changing algorithm can be used. Considering the maximum inter-class variance (OTSU) method and the maximum entropy method in global thresholding, a Niblack algorithm can be used for local thresholding, and the effects of the three can be actually tested and compared by using a defect image, so that the best effect can be selected.
The maximum inter-class variance method (OTSU) divides an image into a background part and an object part according to the gray characteristic of the image. The larger the inter-class variance between the background and the object is, the larger the difference between the two parts constituting the image is, and the smaller the difference between the two parts is caused when part of the object is mistaken for the background or part of the background is mistaken for the object. Thus, a partition with the largest inter-class variance means the least probability of false positives. The main algorithm is as follows:
the threshold value is gradually increased from 0 to 255 (1 is added each time), and the image is divided into a background (0) and a foreground (1) according to the threshold value.
For each threshold, the following calculation is performed:
calculating the proportion of each background pixel point and each foreground pixel point in the total pixel points;
calculating the gray average value of the background pixel points and the foreground pixel points;
calculating the gray average value of all pixel points;
calculating the binary image at that timeThe between-class variance: sigma2＝w1(μ-μ1)2+w0(μ-μ0)2。
w0 is the proportion of background pixel points in the whole image
u0 average gray scale w0
w1 is the ratio of foreground pixel points to the whole image
u1 average gray scale w1
u is the average gray scale of the whole image
And selecting a threshold value which enables the inter-class variance to be maximum, wherein the optimal threshold value is obtained immediately at the moment, and the value can be found through continuous iteration and traversal.
The maximum entropy method (maximum entropy global threshold segmentation) is another global threshold segmentation algorithm based on image statistical information, the principle is based on the information entropy in statistics, and the larger the uncertainty (the larger the information amount) of a system is, the larger the entropy is. Therefore, the maximum entropy method is used for ensuring the entropy between the image classes to be maximum so as to ensure that the binary image contains as much foreground information (defects) as possible. Considering the KSW algorithm, the general idea is still to traverse the threshold:
the threshold value is gradually increased from 0 to 255 (1 is added each time), and the image is divided into a background (0) and a foreground (1) according to the threshold value.
For each threshold, the following calculation is performed:
calculating the probability distribution of each gray value in the background and the foreground;
computing entropy in the background and foreground;
wherein is the cumulative distribution in the background;
Pn,1-Pnrespectively representing the cumulative probability of the background and foreground pixels of the n threshold segmentation, wherein the sum of the cumulative probability of the background and foreground pixels is 1; pi，PjRepresenting the probabilities of the background and foreground pixels of the i-threshold segmentation and the j-threshold segmentation, respectively; i denotes the maximum of all thresholds for the foreground pixel as the foreground pixel value (1-255) t; h1(T),H2(T) represents the entropy of the background and foreground pixels, respectively; and selecting a threshold value which enables the entropy between the classes to be maximum, wherein the optimal threshold value is obtained by iteration continuously, and the value can be found.
The basic idea of the Niblack method (local dynamic threshold segmentation), which is a simple and effective local dynamic threshold algorithm, is to calculate the mean and variance of the pixel points in the neighborhood of each point in the image in its neighborhood, and then binarize with the following threshold: t (x, y) ═ m (x, y) + k × (x, y); for each pixel, (x, y) is the threshold of the point, m is the mean of the pixels in the neighborhood of the point, and s is the standard deviation of the pixels in the r × r field of the point; k is a correction coefficient. The method has the advantages of being capable of well processing aiming at single pixel, and has the disadvantages of low processing speed, no consideration of boundary problem and the need of presetting adjustment parameters.
Morphological treatment: for the metal surface defect after binarization, some gaps exist in some areas (for example, the scratch is shallow in some places) which should be communicated after binarization, and closing operation (namely, expansion and corrosion operation are performed first) is required to close the small slit and ensure that the overall shape is unchanged, so as to avoid that the same scratch is detected as a plurality of separate independent scratches due to the fact that a part of the scratch is broken in the middle. Refine the operation, because some metal surface mar broad, it has certain width in addition to lead to the mar after the operation of closing, this will lead to after carry out the edge to the defect and draw when, the distance between (about) the edge is great about the mar, can be two independent mar by the false detection when follow-up is detected, edge distance and two mar intervals are close about leading to promptly, lead to can't judge that 2 total gross scratches of mar still have 3 mar (1 gross scratch, 2 fine scratches), still 4 mar (4 fine scratches). It is also possible to interpret two separate fine scratches as one coarse scratch. Therefore, the scratch needs to be refined, so that the scratch is narrowed, and meanwhile, the fine scratch is lost due to excessive refining action.
In some application embodiments, the following method may be used for model training and tuning of the convolutional neural network:
structurally:
input layer: the image size is 227 × 227 × 3, where 3 denotes that the number of channels (R, G, B) of the input image is 3.
Convolutional layer: the filter size is 11 × 11, the number of filters is 96, and the convolution step Stride is 4. (the filter size is only listed for width and height, the number of channels of the filter matrix is the same as the number of channels of the input picture, not listed here)
Pooling layer: max pooling, filter size 3 × 3, step size Stride 2.
Convolutional layer: the filter size is 5 × 5, the number of filters is 256, the step size Stride is 1, and padding uses sameconvolation, i.e., the convolutional layer output image and the input image are kept constant in width and height.
Pooling layer: max pooling, filter size 3 × 3, step size Stride 2.
Convolutional layer: the size of the filter is 3 × 3, the number of filters is 384, the step size Stride is 1, and the padding uses sameconvolation.
Convolutional layer: the size of the filter is 3 × 3, the number of filters is 384, the step size Stride is 1, and the padding uses sameconvolation.
Convolutional layer: the size of the filter is 3 × 3, the number of filters is 256, the step size Stride is 1, and the padding uses sameconvolation.
Pooling layer: max pooling, filter size 3 × 3, step length s ═ 2; after the pooling operation is finished, the output matrix flatten with the size of 6 × 6 × 256 is formed into a 9216-dimensional vector.
Full connectivity layer: the number of neurons is 4096.
Full connectivity layer: the number of neurons is 4096.
Full connection layer, output layer: softmax activates the function, with a neuron number of 1000, representing 1000 categories.
About 60million parameters;
a total of 12 layers of neural network structures;
using ReLU as the activation function: f. of(x)＝max(0,x)
Calculation principle of convolutional layer dimensions:
input matrix format: four dimensions, in order: number of samples, image height, image width, number of image channels.
Output matrix format: the order and meaning of the dimensions of the output matrix are the same, but the dimensions of the last three dimensions (image height, image width, number of image channels) are changed.
Weight matrix (convolution kernel) format: again four dimensions, but the meaning of the dimensions is different from both above: convolution kernel height, convolution kernel width, number of input channels, number of output channels (number of convolution kernels).
The interdependence relationship among the input matrix, the weight matrix, and the output matrix.
The number of input channels (in depth) of the convolution kernel is determined by the number of channels of the input matrix.
The number of channels (out depth) of the output matrix is determined by the number of output channels of the convolution kernel.
The two dimensions of height and width (width) of the output matrix are determined by the input matrix, convolution kernel and scanning mode. The calculation formula is as follows:
the pooling layer calculation formula is: n ═ W-F +2P)/S +1
Inputting: the data dimension is W x W; filter size F × F; step length S; the number of pixels P of padding;
and (3) outputting: n × N matrix data.
softmax equation:
Figure BDA0002262444720000162
inputting: t represents the number of categories, such as 7 categories, then T is 7; aj represents the jth value in this vector; and ak in the denominator represents k values in the vector, so there will be a summation symbol (where summation is k from 1 to T, T is equal to T in the above figure, i.e. meaning the number of categories, and the range of j is also 1 to T), and the output will beAnd (3) discharging: sjThe probability that the sample belongs to each class.
And predicting the image defects based on the model, predicting future images by using the trained convolutional neural network model, wherein the prediction results are from image records after the production of new products is finished, such as previous training by adopting historical data, so that a prediction result of whether the current image is defective or not is obtained according to the current image data.
In some alternative embodiments, as shown in fig. 9, a schematic structural diagram of an apparatus 900 for detecting an image defect of an article based on a convolutional neural network in this embodiment is used to implement the above method for detecting an image defect of an article based on a convolutional neural network. The device includes: image pixel slicer 901, image preprocessor 902, convolutional neural network creator 903, and article image defect detector 904.
The image pixel cutter 901 is connected with the image preprocessor 902 to obtain pixels of an article image, cut the image into fixed-size images according to the corresponding relation between preset pixels and image sizes, extract pixel values of the fixed-size images and convert the pixel values into a data matrix of image pixels.
The image preprocessor 902 is connected with the image pixel cutter 903 and the convolutional neural network creator 904, and performs gray normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave; and performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data.
And the convolutional neural network creator 903 is connected with the image preprocessor 902 and the article image defect detector 904, and trains according to the preprocessed image data and the article image defect type thereof to obtain a convolutional neural network for article image defect detection.
An article image defect detector 904, connected with the image pixel cutter 901, the image preprocessor 902 and the convolutional neural network creator 903, for obtaining a preprocessing strategy of selecting corresponding pixel cutting, gray normalization, filtering and threshold segmentation for the image defect type of the article image to be detected; and carrying out pixel cutting, gray level normalization, filtering and threshold segmentation on the to-be-detected object image according to a preprocessing strategy to obtain prediction data of the to-be-detected object image, and substituting the prediction data of the to-be-detected object image into a convolutional neural network to obtain a defect detection result of the to-be-detected object image.
In some alternative embodiments, as shown in fig. 10, which is a schematic structural diagram of an apparatus 1000 for detecting an image defect of an article based on a convolutional neural network according to the second embodiment, different from fig. 9, an image preprocessor 902 includes: a grayscale normalization processor 1001 and an enhanced image data processor 1002.
The grayscale normalization processor 1001 is connected to the image pixel slicer 901 and the enhanced image data processor 1002, and obtains a pixel value of each pixel in the fixed-size image according to the data matrix.
Carrying out gray level normalization processing on the pixel value of the image with the fixed size according to the following formula to obtain the pixel value with the enhanced contrast of the image with the fixed size, wherein the formula is as follows:
Figure BDA0002262444720000181
wherein, IoutFor the converted gray values, IinFor the grey value of each pixel of the original image, IminIs the minimum pixel value, I, of the original imagemaxThe maximum pixel value of the original image is taken;
an enhanced image data processor 1002, connected to the grayscale normalization processor 1001 and the convolutional neural network creator 903, for filtering the grayscale-normalized fixed-size image data by a gaussian kernel function modulated by a sinusoidal plane wave; and performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data.
In some alternative embodiments, as shown in fig. 11, which is a schematic structural diagram of an apparatus 1100 for detecting an image defect of an article based on a convolutional neural network in this embodiment, different from fig. 9, an image preprocessor 902 includes: a grayscale normalization processor 1101 and an image data filtering processor 1102.
The grayscale normalization processor 1101 is connected to the image pixel slicer 901 and the image data filter processor 1102, and performs grayscale normalization on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image.
The image data filtering processor 1102 is connected to the grayscale normalization processor 1101 and the convolutional neural network creator 903, and obtains a complex expression by using a gaussian kernel function modulated by a sinusoidal plane wave for the grayscale-normalized fixed-size image data:
Figure BDA0002262444720000182
wherein the content of the first and second substances,
Figure BDA0002262444720000183
in order to obtain new pixel displacement image data, x and y are pixel coordinates, and x' is xcos θ + ysin θ; y' ═ xsin θ + ycos θ; wavelength lambda represents the wavelength parameter of the cosine function in the filter kernel function, direction theta represents the direction of the parallel strips in the filter kernel, and phase shift
Figure BDA0002262444720000184
Representing phase parameters of a cosine function in a filter kernel function, wherein the length-width ratio gamma is a space aspect ratio, determining the ellipticity of the shape of the filter function, and sigma represents the standard deviation of a Gaussian factor of the filter function;
and carrying out filtering processing according to the new pixel displacement image data to obtain data after filtering of the image with the fixed size, wherein the formula is as follows:
wherein, S (x, y) is data after fixed-size image filtering;
and performing threshold segmentation on the filtered data of the fixed-size image to obtain preprocessed image data.
Optionally, in the image data filtering processor, the relationship between the half-response spatial frequency bandwidth b and the gaussian factor σ and the wavelength λ is as follows:
Figure BDA0002262444720000192
in some optional embodiments, as shown in fig. 12, which is a schematic structural diagram of a fourth apparatus 1200 for detecting an image defect of an article based on a convolutional neural network in this embodiment, different from fig. 9, an image preprocessor 902 includes: an enhancement and filtering processor 1201 and a threshold segmentation processor 1202.
The enhancing and filtering processor 1201 is connected with the image pixel cutter 901 and the threshold segmentation processor 1202, and performs gray level normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; and filtering the fixed-size image data with the normalized gray scale by a Gaussian kernel function modulated by a sinusoidal plane wave.
The threshold segmentation processor 1202, connected to the enhancement and filtering processor 1201 and the convolutional neural network creator 903, includes: a pre-threshold segmentation processing unit 1221, a threshold segmentation policy processing unit 1222, and a threshold segmentation processing unit 1223; the pre-threshold segmentation processing unit 1221 is connected to the enhancement and filtering processor 1201 and the threshold segmentation policy processing unit 1222, and obtains filtered image data of the tested defect image, and performs maximum inter-class variance threshold segmentation, maximum entropy global threshold segmentation, and local dynamic threshold segmentation, respectively.
And a threshold segmentation policy processing unit 1222, connected to the pre-threshold segmentation processing unit 1221 and the threshold segmentation processing unit 1223, for comparing the effects of the maximum inter-class variance threshold segmentation, the maximum entropy global threshold segmentation, and the local dynamic threshold segmentation with the defects of the actual image, and selecting the threshold segmentation closest to the actual situation as the pre-processing segmentation policy.
And a threshold segmentation processing unit 1223, connected to the threshold segmentation policy processing unit 1222 and the convolutional neural network creator 903, for performing threshold segmentation on the filtered data of the fixed-size image according to a pre-processing segmentation policy to obtain pre-processed image data.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A method for detecting image defects of an article based on a convolutional neural network is characterized by comprising the following steps:
obtaining pixels of an article image, cutting the article image into fixed-size images according to the corresponding relation between preset pixels and image sizes, extracting pixel values of the fixed-size images and converting the pixel values into data matrixes of image pixels;
carrying out gray scale normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave; performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data;
training according to the preprocessed image data and the article image defect type thereof to obtain a convolutional neural network for article image defect detection;
acquiring a preprocessing strategy of selecting corresponding pixel cutting, gray normalization, filtering and threshold segmentation for the image defect type of the to-be-detected article image; and carrying out pixel cutting, gray level normalization, filtering and threshold segmentation on the to-be-detected article image according to the preprocessing strategy to obtain prediction data of the to-be-detected article image, and substituting the prediction data of the to-be-detected article image into the convolutional neural network to obtain a defect detection result of the to-be-detected article image.
2. The method for detecting defects of an article image based on a convolutional neural network as claimed in claim 1, wherein the gray-scale normalization processing is performed on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image, and the method comprises the following steps:
acquiring a pixel value of each pixel in the fixed-size image according to the data matrix;
carrying out gray scale normalization processing on the pixel values of the fixed-size image according to the following formula to obtain pixel values with enhanced contrast of the fixed-size image, wherein the formula is as follows:
wherein, IoutFor the converted gray values, IinFor the grey value of each pixel of the original image, IminIs the minimum pixel value, I, of the original imagemaxIs the maximum pixel value of the original image.
3. The method for detecting defects in images of articles based on convolutional neural network as claimed in claim 1, wherein the fixed size image data with normalized gray scale is filtered by gaussian kernel function modulated by sinusoidal plane wave as:
and (3) obtaining a complex expression by using a Gaussian kernel function modulated by sinusoidal plane waves for the image data with the fixed size and with the gray level normalized:
Figure FDA0002262444710000021
wherein the content of the first and second substances,in order to obtain new pixel displacement image data, x and y are pixel coordinates, and x' is xcos θ + ysin θ; y' ═ xsin θ + ycos θ; the wavelength lambda represents the wavelength parameter of the cosine function in the filter kernel function, and the direction theta represents the filteringDirection of parallel strips in the nucleus, phase shiftThe phase parameter of the cosine function in the filter kernel function is represented, the aspect ratio gamma is a spatial aspect ratio, the ellipticity of the shape of the filter function is determined, and the sigma represents the standard deviation of the gaussian factor of the filter function.
4. The method of claim 3, wherein the half-response spatial frequency bandwidth b in the filter function is related to the Gaussian factor σ and the wavelength λ by the following relationship:
5. the method for detecting defects in images of articles based on convolutional neural network as claimed in claim 1, wherein the filtered data of the fixed size image is threshold-segmented to obtain pre-processed image data, which is:
obtaining image data of a tested defect image after filtration, and respectively carrying out maximum inter-class variance threshold segmentation, maximum entropy method global threshold segmentation and local dynamic threshold segmentation;
comparing the effects of the results of the maximum inter-class variance threshold segmentation, the maximum entropy global threshold segmentation and the local dynamic threshold segmentation with the defects of the actual image, and selecting the threshold segmentation closest to the actual situation as a preprocessing segmentation strategy;
and performing threshold segmentation on the filtered data of the image with the fixed size according to the preprocessing segmentation strategy to obtain preprocessed image data.
6. An apparatus for detecting image defects of an object based on a convolutional neural network, comprising: the system comprises an image pixel cutter, an image preprocessor, a convolutional neural network creator and an article image defect detector; wherein the content of the first and second substances,
the image pixel cutter is connected with the image preprocessor to obtain pixels of an article image, cuts the image into an image with a fixed size according to the corresponding relation between preset pixels and the image size, extracts pixel values of the image with the fixed size and converts the pixel values into a data matrix of image pixels;
the image preprocessor is connected with the image pixel cutter and the convolutional neural network creator and is used for carrying out gray level normalization processing on the fixed-size image according to the data matrix so as to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave; performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data;
the convolutional neural network creator is connected with the image preprocessor and the article image defect detector and is used for training according to the preprocessed image data and the article image defect types thereof to obtain a convolutional neural network for article image defect detection;
the article image defect detector is connected with the image pixel cutter, the image preprocessor and the convolutional neural network creator to obtain a preprocessing strategy of selecting corresponding pixel cutting, gray normalization, filtering and threshold segmentation for the image defect type of the article image to be detected; and carrying out pixel cutting, gray level normalization, filtering and threshold segmentation on the to-be-detected article image according to the preprocessing strategy to obtain prediction data of the to-be-detected article image, and substituting the prediction data of the to-be-detected article image into the convolutional neural network to obtain a defect detection result of the to-be-detected article image.
7. The apparatus of claim 6, wherein the image preprocessor comprises: a gray scale normalization processor and an enhanced image data processor; wherein the content of the first and second substances,
the gray normalization processor is connected with the image pixel cutter and the enhanced image data processor and is used for acquiring the pixel value of each pixel in the image with the fixed size according to the data matrix;
carrying out gray scale normalization processing on the pixel values of the fixed-size image according to the following formula to obtain pixel values with enhanced contrast of the fixed-size image, wherein the formula is as follows:
wherein, IoutFor the converted gray values, IinFor the grey value of each pixel of the original image, IminIs the minimum pixel value, I, of the original imagemaxThe maximum pixel value of the original image is taken;
the enhanced image data processor is connected with the gray level normalization processor and the convolutional neural network creator and is used for filtering the fixed-size image data with the gray level normalization by a Gaussian kernel function modulated by a sine plane wave; and performing threshold segmentation on the filtered data of the image with the fixed size to obtain preprocessed image data.
8. The apparatus of claim 6, wherein the image preprocessor comprises: a gray scale normalization processor and an image data filtering processor; wherein the content of the first and second substances,
the gray normalization processor is connected with the image pixel cutter and the image data filtering processor, and performs gray normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image;
the image data filtering processor is connected with the gray level normalization processor and the convolutional neural network creator, and obtains a complex expression by using a Gaussian kernel function modulated by sinusoidal plane waves for the image data with fixed size with gray level normalization:
Figure FDA0002262444710000042
wherein the content of the first and second substances,
Figure FDA0002262444710000043
in order to obtain new pixel displacement image data, x and y are pixel coordinates, and x' is xcos θ + ysin θ; y' ═ xsin θ + ycos θ; wavelength lambda represents the wavelength parameter of the cosine function in the filter kernel function, direction theta represents the direction of the parallel strips in the filter kernel, and phase shift
Figure FDA0002262444710000044
Representing phase parameters of a cosine function in a filter kernel function, wherein the length-width ratio gamma is a space aspect ratio, determining the ellipticity of the shape of the filter function, and sigma represents the standard deviation of a Gaussian factor of the filter function;
filtering the new pixel displacement image data to obtain the data of the fixed-size image after filtering, wherein the formula is as follows:
Figure FDA0002262444710000045
wherein, S (x, y) is data after fixed-size image filtering;
and performing threshold segmentation on the filtered data of the fixed-size image to obtain preprocessed image data.
9. The apparatus of claim 8, wherein the image data filter processor is configured to correlate the half-response spatial frequency bandwidth b with the gaussian factor σ and the wavelength λ as follows:
Figure FDA0002262444710000051
10. the apparatus of claim 6, wherein the image preprocessor comprises: an enhancement and filtering processor and a threshold segmentation processor; wherein the content of the first and second substances,
the enhancement and filtering processor is connected with the image pixel cutter and the threshold segmentation processor, and performs gray level normalization processing on the fixed-size image according to the data matrix to enhance the contrast of the fixed-size image; filtering the fixed-size image data with the normalized gray level through a Gaussian kernel function modulated by a sine plane wave;
the threshold segmentation processor is connected with the enhancing and filtering processor and the convolutional neural network creator, and comprises: the device comprises a pre-threshold segmentation processing unit, a threshold segmentation strategy processing unit and a threshold segmentation processing unit; the pre-threshold segmentation processing unit is connected with the enhancement and filtering processor and the threshold segmentation strategy processing unit, and is used for obtaining image data of the tested defect image after being filtered and respectively carrying out maximum inter-class variance threshold segmentation, maximum entropy global threshold segmentation and local dynamic threshold segmentation;
the threshold segmentation strategy processing unit is connected with the pre-threshold segmentation processing unit and the threshold segmentation processing unit, compares the effects of the results of the maximum inter-class variance threshold segmentation, the maximum entropy global threshold segmentation and the local dynamic threshold segmentation with the defects of the actual image, and selects the threshold segmentation closest to the actual situation as the pre-processing segmentation strategy;
and the threshold segmentation processing unit is connected with the threshold segmentation strategy processing unit and the convolutional neural network creator, and performs threshold segmentation on the filtered data of the image with the fixed size according to the preprocessing segmentation strategy to obtain preprocessed image data.
CN201911075959.0A 2019-11-06 2019-11-06 Method and device for detecting article image defects based on convolutional neural network Pending CN110766689A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911075959.0A CN110766689A (en) 2019-11-06 2019-11-06 Method and device for detecting article image defects based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911075959.0A CN110766689A (en) 2019-11-06 2019-11-06 Method and device for detecting article image defects based on convolutional neural network

Publications (1)

Publication Number Publication Date
CN110766689A true CN110766689A (en) 2020-02-07

Family

ID=69336011

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911075959.0A Pending CN110766689A (en) 2019-11-06 2019-11-06 Method and device for detecting article image defects based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN110766689A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111583272A (en) * 2020-04-17 2020-08-25 西安工程大学 Improved Niblack infrared image segmentation method combined with maximum entropy
CN112308114A (en) * 2020-09-24 2021-02-02 赣州好朋友科技有限公司 Method and device for sorting scheelite and readable storage medium
CN112419291A (en) * 2020-11-30 2021-02-26 佛山职业技术学院 Training method of bottle blank defect detection model, storage medium and terminal equipment
CN112710632A (en) * 2021-03-23 2021-04-27 四川京炜交通工程技术有限公司 Method and system for detecting high and low refractive indexes of glass beads
CN112907519A (en) * 2021-01-29 2021-06-04 广州信邦智能装备股份有限公司 Metal curved surface defect analysis system and method based on deep learning
CN114494924A (en) * 2022-02-17 2022-05-13 江苏云舟通信科技有限公司 Visual target information identification platform
CN114565607A (en) * 2022-04-01 2022-05-31 南通沐沐兴晨纺织品有限公司 Fabric defect image segmentation method based on neural network
CN114792314A (en) * 2022-06-21 2022-07-26 南通永卓金属制品有限公司 Laser beam-based metal mesh defect detection method and artificial intelligence system
CN114881949A (en) * 2022-04-26 2022-08-09 成都唐源电气股份有限公司 Tunnel surface defect identification method
CN115174807A (en) * 2022-06-28 2022-10-11 上海艾为电子技术股份有限公司 Anti-shake detection method and device, terminal equipment and readable storage medium
CN115660971A (en) * 2022-10-08 2023-01-31 镕铭微电子(济南)有限公司 Method for realizing USM sharpening based on deep learning hardware accelerator
CN116309578A (en) * 2023-05-19 2023-06-23 山东硅科新材料有限公司 Plastic wear resistance image auxiliary detection method using silane coupling agent
CN117325012A (en) * 2023-10-25 2024-01-02 江阴市精奇数控有限公司 Crack defect management device for grinding bearing

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107833220A (en) * 2017-11-28 2018-03-23 河海大学常州校区 Fabric defect detection method based on depth convolutional neural networks and vision significance
CN109101976A (en) * 2018-07-10 2018-12-28 温州大学 A kind of detection method of arc extinguishing grid pieces surface defect
CN110349126A (en) * 2019-06-20 2019-10-18 武汉科技大学 A kind of Surface Defects in Steel Plate detection method based on convolutional neural networks tape label

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107833220A (en) * 2017-11-28 2018-03-23 河海大学常州校区 Fabric defect detection method based on depth convolutional neural networks and vision significance
CN109101976A (en) * 2018-07-10 2018-12-28 温州大学 A kind of detection method of arc extinguishing grid pieces surface defect
CN110349126A (en) * 2019-06-20 2019-10-18 武汉科技大学 A kind of Surface Defects in Steel Plate detection method based on convolutional neural networks tape label

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
吴进仪: "基于机器视觉的衬布缺陷检测系统研究", 《中国优秀博硕士论文全文数据库（硕士） 工程科技I辑》 *
王一丁 等: "《数字图像处理》", 31 August 2015 *

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111583272A (en) * 2020-04-17 2020-08-25 西安工程大学 Improved Niblack infrared image segmentation method combined with maximum entropy
CN112308114A (en) * 2020-09-24 2021-02-02 赣州好朋友科技有限公司 Method and device for sorting scheelite and readable storage medium
CN112419291A (en) * 2020-11-30 2021-02-26 佛山职业技术学院 Training method of bottle blank defect detection model, storage medium and terminal equipment
CN112419291B (en) * 2020-11-30 2024-06-04 佛山职业技术学院 Training method of bottle embryo defect detection model, storage medium and terminal equipment
CN112907519A (en) * 2021-01-29 2021-06-04 广州信邦智能装备股份有限公司 Metal curved surface defect analysis system and method based on deep learning
CN112710632A (en) * 2021-03-23 2021-04-27 四川京炜交通工程技术有限公司 Method and system for detecting high and low refractive indexes of glass beads
CN114494924A (en) * 2022-02-17 2022-05-13 江苏云舟通信科技有限公司 Visual target information identification platform
CN114565607B (en) * 2022-04-01 2024-06-04 汕头市鼎泰丰实业有限公司 Fabric defect image segmentation method based on neural network
CN114565607A (en) * 2022-04-01 2022-05-31 南通沐沐兴晨纺织品有限公司 Fabric defect image segmentation method based on neural network
CN114881949A (en) * 2022-04-26 2022-08-09 成都唐源电气股份有限公司 Tunnel surface defect identification method
CN114792314A (en) * 2022-06-21 2022-07-26 南通永卓金属制品有限公司 Laser beam-based metal mesh defect detection method and artificial intelligence system
CN115174807A (en) * 2022-06-28 2022-10-11 上海艾为电子技术股份有限公司 Anti-shake detection method and device, terminal equipment and readable storage medium
CN115660971A (en) * 2022-10-08 2023-01-31 镕铭微电子(济南)有限公司 Method for realizing USM sharpening based on deep learning hardware accelerator
CN115660971B (en) * 2022-10-08 2024-02-23 镕铭微电子(济南)有限公司 Method for realizing USM sharpening based on deep learning hardware accelerator
CN116309578A (en) * 2023-05-19 2023-06-23 山东硅科新材料有限公司 Plastic wear resistance image auxiliary detection method using silane coupling agent
CN116309578B (en) * 2023-05-19 2023-08-04 山东硅科新材料有限公司 Plastic wear resistance image auxiliary detection method using silane coupling agent
CN117325012A (en) * 2023-10-25 2024-01-02 江阴市精奇数控有限公司 Crack defect management device for grinding bearing
CN117325012B (en) * 2023-10-25 2024-04-12 江阴市精奇数控有限公司 Crack defect management device for grinding bearing

Similar Documents

Publication Publication Date Title
CN110766689A (en) Method and device for detecting article image defects based on convolutional neural network
Fan et al. Road crack detection using deep convolutional neural network and adaptive thresholding
CN111310558B (en) Intelligent pavement disease extraction method based on deep learning and image processing method
CN109753914B (en) License plate character recognition method based on deep learning
CN115861135B (en) Image enhancement and recognition method applied to panoramic detection of box body
Zaitoun et al. Survey on image segmentation techniques
CN116168026B (en) Water quality detection method and system based on computer vision
CN116664559B (en) Machine vision-based memory bank damage rapid detection method
CN111079596A (en) System and method for identifying typical marine artificial target of high-resolution remote sensing image
CN109919960B (en) Image continuous edge detection method based on multi-scale Gabor filter
Dixit et al. Image texture analysis-survey
CN114882040B (en) Sewage treatment detection method based on template matching
CN113221881B (en) Multi-level smart phone screen defect detection method
Jian et al. Imbalanced defect classification for mobile phone screen glass using multifractal features and a new sampling method
CN116152115B (en) Garbage image denoising processing method based on computer vision
Liang et al. Automatic defect detection of texture surface with an efficient texture removal network
CN114764801A (en) Weak and small ship target fusion detection method and device based on multi-vision significant features
CN108694415B (en) Image feature extraction method and device and water source image classification method and device
Wen et al. Complex surface ROI detection for steel plate fusing the gray image and 3D depth information
Wu et al. Research on crack detection algorithm of asphalt pavement
Baareh et al. Performance evaluation of edge detection using Sobel, Homogeneity and Prewitt algorithms
US10115195B2 (en) Method and apparatus for processing block to be processed of urine sediment image
Scharfenberger et al. Image saliency detection via multi-scale statistical non-redundancy modeling
Dong A pixel-wise framework based on convolutional neural network for surface defect detection
CN112686222B (en) Method and system for detecting ship target by satellite-borne visible light detector

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200207