CN116309586A - Defect detection method, device, equipment and medium based on convolutional neural network - Google Patents

Defect detection method, device, equipment and medium based on convolutional neural network Download PDF

Info

Publication number
CN116309586A
CN116309586A CN202310572870.5A CN202310572870A CN116309586A CN 116309586 A CN116309586 A CN 116309586A CN 202310572870 A CN202310572870 A CN 202310572870A CN 116309586 A CN116309586 A CN 116309586A
Authority
CN
China
Prior art keywords
target image
neural network
convolutional neural
flaw
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310572870.5A
Other languages
Chinese (zh)
Inventor
葛铭
沈井学
魏江
洪鼎岳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Baizijian Technology Co ltd
Original Assignee
Hangzhou Baizijian Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Baizijian Technology Co ltd filed Critical Hangzhou Baizijian Technology Co ltd
Priority to CN202310572870.5A priority Critical patent/CN116309586A/en
Publication of CN116309586A publication Critical patent/CN116309586A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956—Inspecting patterns on the surface of objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30124—Fabrics; Textile; Paper
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/06—Recognition of objects for industrial automation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biochemistry (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Evolutionary Computation (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention discloses a flaw detection method, device, equipment and medium based on a convolutional neural network. The method comprises the following steps: acquiring a target image of a cloth to be detected; inputting the target image into a convolutional neural network model obtained by training in advance to obtain thermodynamic diagrams, sizes or offset corresponding to the target image; determining a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset; wherein the convolutional neural network model comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module. According to the scheme provided by the embodiment of the invention, the flaw detection can be rapidly and accurately carried out on the cloth, and a large amount of human resources are saved.

Description

Defect detection method, device, equipment and medium based on convolutional neural network
Technical Field
The embodiment of the invention relates to the technical field of artificial intelligence, in particular to a flaw detection method, device, equipment and medium based on a convolutional neural network.
Background
In the textile industry, the annual production of cloth is hundreds of millions, and when flaws occur in the cloth, the unit price loss of the cloth is 45-50%, so how to accurately detect flaws in the cloth is the focus of each manufacturer.
At present, most manufacturers detect flaws on cloth manually, but due to subjective reasons of people, a large number of flaws are missed to be detected, and the working efficiency is low.
How to detect flaws on cloth quickly and accurately is a key issue in the industry.
Disclosure of Invention
The embodiment of the invention provides a flaw detection method, device, equipment and medium based on a convolutional neural network, so as to rapidly and accurately detect flaws of cloth.
According to an aspect of the embodiment of the invention, there is provided a cloth flaw detection method based on a convolutional neural network, including:
acquiring a target image of a cloth to be detected;
inputting the target image into a convolutional neural network model obtained by training in advance to obtain thermodynamic diagrams, sizes or offset corresponding to the target image;
determining a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset;
wherein the convolutional neural network model comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
Optionally, the acquiring the target image of the cloth to be detected includes:
acquiring an image of the cloth to be detected in real time through a pre-installed camera;
and adjusting the size of the image of the cloth to be detected to obtain the target image.
Optionally, the convolutional neural network model is trained by:
acquiring a training data set; wherein the training data set comprises a plurality of cloth images containing flaws and a plurality of cloth images not containing flaws;
inputting the training data set into a target convolutional neural network for iterative training to obtain the convolutional neural network model;
wherein the target convolutional neural network comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
Optionally, the inputting the target image into a convolutional neural network model obtained by training in advance to obtain a thermodynamic diagram, a size or an offset corresponding to the target image includes:
determining, by the thermodynamic diagram creation module, a thermodynamic diagram corresponding to the target image;
and determining the size or the offset corresponding to the target image according to the thermodynamic diagram through the intermediate module.
Optionally, the determining, according to at least one of the thermodynamic diagram, the size or the offset, a flaw area of a flaw included in the target image includes:
and determining a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset by the output determining module.
Optionally, the determining, by the output determining module, the flaw area of the flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset includes:
determining thermodynamic losses according to the thermodynamic diagrams, and determining flaw center points according to the thermodynamic losses;
determining a size loss from the size;
determining an offset loss from the offset;
and superposing the size loss or the offset loss at the flaw center point to determine a flaw area.
Optionally, the thermodynamic diagram building module uses an hourglass CNN as a backbone network.
According to another aspect of the embodiment of the present invention, there is provided a cloth defect detecting device based on a convolutional neural network, including:
the target image acquisition module is used for acquiring a target image of the cloth to be detected;
the input module is used for inputting the target image into a convolutional neural network model which is obtained through training in advance, and obtaining thermodynamic diagrams, sizes or offset corresponding to the target image;
the flaw area determining module determines a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset;
wherein the convolutional neural network model comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
According to another aspect of an embodiment of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor, so that the at least one processor can execute the cloth defect detection method based on the convolutional neural network according to any embodiment of the present invention.
According to another aspect of the embodiments of the present invention, there is provided a computer readable storage medium storing computer instructions for implementing the fabric defect detection method based on a convolutional neural network according to any one of the embodiments of the present invention when executed by a processor.
According to the technical scheme, the target image of the cloth to be detected is obtained; inputting the target image into a convolutional neural network model obtained by training in advance to obtain thermodynamic diagrams, sizes or offset corresponding to the target image; according to at least one of the thermodynamic diagram, the size or the offset, the flaw area of the flaw contained in the target image is determined, so that flaw detection can be rapidly and accurately performed on the cloth, and a large amount of human resources are saved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention, nor is it intended to be used to limit the scope of the embodiments of the invention. Other features of embodiments of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a cloth defect detection method based on a convolutional neural network according to a first embodiment of the present invention;
FIG. 2 is a graph showing a cloth image and a defective area detection result according to a first embodiment of the present invention;
fig. 3 is a schematic structural diagram of a fabric defect detecting device based on a convolutional neural network according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device for implementing the cloth flaw detection method based on the convolutional neural network according to an embodiment of the present invention.
Detailed Description
In order to make the embodiments of the present invention better understood by those skilled in the art, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the embodiments of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the embodiments of the present invention and the above-described drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a cloth flaw detection method based on a convolutional neural network according to an embodiment of the present invention, where the method may be performed by a cloth flaw detection device based on a convolutional neural network, and the cloth flaw detection device based on a convolutional neural network may be implemented in hardware and/or software, and the cloth flaw detection device based on a convolutional neural network may be configured in an electronic device such as a computer, a server or a tablet computer. Specifically, referring to fig. 1, the method specifically includes the following steps:
and 110, acquiring a target image of the cloth to be detected.
The fabric to be detected may be all produced and delivered, or may be a fabric being produced by a loom, which is not limited in this embodiment.
In an alternative implementation of the present embodiment, the image of the cloth to be detected may be acquired in real time by a camera mounted at a fixed position (for example, mounted at the top of a loom), and the acquired image is determined as the target image.
Optionally, in this embodiment, acquiring the target image of the fabric to be detected may include: acquiring an image of the cloth to be detected in real time through a pre-installed camera; and adjusting the size of the image of the cloth to be detected to obtain the target image.
The pre-installed camera may be a color camera or a black-and-white camera installed at the top of the loom or the top of the cloth defect detecting mechanism, which is not limited in this embodiment.
In an alternative implementation manner of the embodiment, the image of the cloth to be detected can be acquired in real time through a camera installed at the top end of the loom; further, the acquired image may be resized, for example, to 256×256 or 512×512, to obtain the target image according to the embodiment.
The number of target images in the present embodiment may be one or a plurality of target images, and is not limited in the present embodiment.
And 120, inputting the target image into a convolutional neural network model which is trained in advance, and obtaining a thermodynamic diagram, a size or an offset corresponding to the target image.
In an optional implementation manner of this embodiment, after the target image is acquired, the target image may be further input into a convolutional neural network model that is trained in advance, so as to obtain a thermodynamic diagram, a size, an offset, or the like corresponding to the target image.
The convolutional neural network model may be a CNN network model, which may include: thermodynamic diagram establishing module, intermediate module and output determining module.
In this embodiment, the convolutional neural network model may be trained as follows: acquiring a training data set; wherein the training data set comprises a plurality of cloth images containing flaws and a plurality of cloth images not containing flaws; inputting the training data set into a target convolutional neural network for iterative training to obtain the convolutional neural network model; wherein the target convolutional neural network comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
Optionally, in this embodiment, the thermodynamic diagram building module uses an hourglass CNN as a backbone network.
In an optional implementation manner of this embodiment, inputting the target image into a convolutional neural network model obtained by training in advance, to obtain a thermodynamic diagram, a size or an offset corresponding to the target image may include: determining, by the thermodynamic diagram creation module, a thermodynamic diagram corresponding to the target image; and determining the size or the offset corresponding to the target image according to the thermodynamic diagram through the intermediate module.
Optionally, in this embodiment, after the target image is input into the convolutional neural network model, a thermodynamic diagram corresponding to the target image may be determined by a thermodynamic diagram establishment module in the convolutional neural network model; further, the size or offset corresponding to the target image may be determined by an intermediate module in the convolutional neural network model.
The size or offset corresponding to the target image in the present embodiment does not refer to the size or offset of the target image itself, but refers to the size or offset of a flaw existing in the target image.
In one example of this embodiment, a hoursside-like CNN may be used as a backbone network (backbone) to establish a desired thermodynamic diagram (hetmap). Wherein, the intensity (intensity peak) thereof represents key points (keypoints); these keypoints are divided into four corners (markers) or center points (markers) of the object for how to generate the required bounding boxes, which can then help predict the size (dimension) and location (location) of the object. In addition, in this embodiment, there are other CNN branches to predict the dimension (e.g., wide or high) of the containing box.
In this embodiment, after the backbone network (thermodynamic diagram building block), three branch networks may be connected, each for predicting thermodynamic diagram (hetmap) outputs (with c channels), dimension (dimension) outputs (with 2 channels), offset (with 2 channels). All last positions have a total of c+4 channels. The thermodynamic diagram branch is matched with the final key point prediction result of the regression mechanism, and the thermodynamic diagram is provided with c channels corresponding to c categories. Finally, selecting the thermodynamic diagram value of each channel to be the highest; which corresponds to the original location, is used as the center point (points) of the category.
It should be noted that in this embodiment, the size branches are used to predict width and height, the offset branches are used to compensate for discrete errors caused by the convolution process used in downsampling, and the two branches are implemented by a regression mechanism, and are not related to class prediction, so that all classes can share the information. The size branch and offset branch referred to in this embodiment correspond to the intermediate module referred to in this embodiment.
Step 130, determining a flaw area of the flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset.
In an optional implementation manner of this embodiment, after obtaining the thermodynamic diagram, the size or the offset corresponding to the target image, the flaw area of the flaw included in the target image may be further determined according to at least one of the thermodynamic diagram, the size or the offset. For example, in the present embodiment, a flaw area of a flaw included in a target image may be determined from a thermodynamic diagram; the flaw area of the flaw included in the target image may be determined based on the thermodynamic diagram and the size; the flaw area of the flaw included in the target image may also be determined based on the thermodynamic diagram, the size, and the offset.
Optionally, in this embodiment, determining the flaw area of the flaw included in the target image according to at least one of the thermodynamic diagram, the size, or the offset may include: and determining a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset by the output determining module.
In an optional implementation manner of this embodiment, the determining, by the output determining module, a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size, or the offset may include: determining thermodynamic losses according to the thermodynamic diagrams, and determining flaw center points according to the thermodynamic losses; determining a size loss from the size; determining an offset loss from the offset; and superposing the size loss or the offset loss at the flaw center point to determine a flaw area.
In one example of this embodiment, the correct location (location) and its predicted box (bound box) can be evaluated by thermodynamic diagram prediction; the distance from the center point (center) to the four sides can be predicted at the same time.
It will be appreciated that, due to textile defects, the appearance is that there is no proportion or criterion, so conventionally common target detection by Anchor-based, whether by a one-stage or two-stage mechanism, cannot avoid the situation that its aspect ratio is not fixed, because they have the following problems: (1) The definition of Anchor limits the performance of the detection algorithm to some extent; (2) Post-processing operations such as NMS reduce the speed of the overall detection algorithm; (3) Anchor is permanently unset and variable.
In order to solve the above problem, the convolutional neural network model in this embodiment adopts an Anchor-free target detection algorithm, which has the following advantages compared with other single-stage or double-stage target detection algorithms: the method can remove low-efficiency complex Anchor operation, and further improves the performance of a detection algorithm; the filtering operation can be directly performed on the thermodynamic diagram, so that time-consuming NMS post-processing operation is removed, and the running speed of the whole algorithm is further improved.
In this embodiment, the backbone network of the convolutional neural network is designed by adopting a residual network architecture concept such as ResNet, so that a plurality of networks can be stacked (stack) and finally a multi-scale (feature maps) is obtained. Illustratively, assuming a total of c textile-type flaws to be inspected, the thermodynamic diagram has c channels; each channel corresponds to a category, the brightest area on the thermodynamic diagram represents the position of the center point of the target, and the center point offset branch is used for compensating pixel errors caused by mapping points on the thermodynamic diagram after the mapping to the original diagram, namely the displacement of the center point in the framing frame, and meanwhile, the flaw size can be estimated.
The convolutional neural network model involved in this embodiment is mainly divided into three basic phases: preprocessing, extracting characteristic values and classifying. The preprocessing stage mainly divides the shot image into N x N square grids, and then median filtering or Gaussian filtering is carried out to filter noise. In the characteristic value extraction stage, the required characteristic value can be extracted through a plurality of convolution layers, thermodynamic diagrams and pooling layers; a multi-pooling mechanism can be used for feature value fusion in the classification stage. Thereby achieving the purpose of classification.
According to the technical scheme, a target image of the cloth to be detected is obtained; inputting the target image into a convolutional neural network model obtained by training in advance to obtain thermodynamic diagrams, sizes or offset corresponding to the target image; according to at least one of the thermodynamic diagram, the size or the offset, the flaw area of the flaw contained in the target image is determined, so that flaw detection can be rapidly and accurately performed on the cloth, and a large amount of human resources are saved.
In order to better understand the cloth flaw detection method based on convolutional neural network in the present embodiment, fig. 2 is a cloth image and flaw area detection result thereof in the present embodiment, and the size is 8192×2000; in the process, it is first cut into a plurality of small images, for example, into a plurality of cloth images of 256×256 or a plurality of images of 512×512, which are not limited in this embodiment.
Further, each small image may be sequentially input into the convolutional neural network model related to the embodiment, to obtain a thermodynamic diagram, a size or an offset corresponding to each small image; further, the flaw area of the flaw included in each small image may be determined from the thermodynamic diagram, the size, or the offset corresponding to each small image; finally, splicing the small images to obtain a flaw area of the cloth image; the box in fig. 2 is a defective area, and small black dots in the defective area are holes in the cloth.
Example two
Fig. 3 is a schematic structural diagram of a fabric defect detecting device based on a convolutional neural network according to a second embodiment of the present invention. As shown in fig. 4, the apparatus includes: a target image acquisition module 310, an input module 320, and a defective region determination module 330.
A target image acquisition module 310, configured to acquire a target image of a fabric to be detected;
the input module 320 is configured to input the target image into a convolutional neural network model that is trained in advance, to obtain a thermodynamic diagram, a size, or an offset corresponding to the target image;
a flaw area determining module 330, configured to determine a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size, or the offset;
wherein the convolutional neural network model comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
According to the scheme of the embodiment, a target image of the cloth to be detected is acquired through a target image acquisition module; inputting the target image into a convolutional neural network model which is obtained through training in advance through an input module, and obtaining thermodynamic diagrams, sizes or offset corresponding to the target image; and determining the flaw area of the flaw point contained in the target image by the flaw area determining module according to at least one of the thermodynamic diagram, the size or the offset, so that flaw detection can be performed on the cloth rapidly and accurately, and a large amount of manpower resources are saved.
In an optional implementation manner of this embodiment, the target image acquisition module 310 is specifically configured to acquire, through a pre-installed camera, an image of the fabric to be detected in real time;
and adjusting the size of the image of the cloth to be detected to obtain the target image.
In an alternative implementation of this embodiment, the convolutional neural network model is trained by:
acquiring a training data set; wherein the training data set comprises a plurality of cloth images containing flaws and a plurality of cloth images not containing flaws;
inputting the training data set into a target convolutional neural network for iterative training to obtain the convolutional neural network model;
wherein the target convolutional neural network comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
In an alternative implementation manner of this embodiment, the input module 320 is specifically configured to determine, by using the thermodynamic diagram creation module, a thermodynamic diagram corresponding to the target image;
and determining the size or the offset corresponding to the target image according to the thermodynamic diagram through the intermediate module.
In an optional implementation manner of this embodiment, the defect area determining module 330 is specifically configured to determine, by using the output determining module, a defect area of a defect included in the target image according to at least one of the thermodynamic diagram, the size, or the offset.
In an alternative implementation manner of this embodiment, the defect area determining module 330 is further specifically configured to determine a thermodynamic diagram loss according to the thermodynamic diagram, and determine a defect center point according to the thermodynamic diagram loss;
determining a size loss from the size;
determining an offset loss from the offset;
and superposing the size loss or the offset loss at the flaw center point to determine a flaw area.
In an optional implementation manner of this embodiment, the thermodynamic diagram building module uses an hourglass CNN as a backbone network.
The cloth flaw detection device based on the convolutional neural network provided by the embodiment of the invention can execute the cloth flaw detection method based on the convolutional neural network provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example III
Fig. 4 shows a schematic diagram of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the embodiments of the invention described and/or claimed herein.
As shown in fig. 4, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a cloth flaw detection method based on a convolutional neural network.
In some embodiments, the cloth flaw detection method based on convolutional neural network may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the cloth flaw detection method based on convolutional neural network described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the cloth flaw detection method based on a convolutional neural network in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of embodiments of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the embodiments of the present invention may be performed in parallel, sequentially or in a different order, so long as the desired result of the technical solution of the embodiments of the present invention can be achieved, which is not limited herein.
The above detailed description should not be construed as limiting the scope of the embodiments of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the embodiments of the present invention should be included in the scope of the embodiments of the present invention.

Claims (10)

1. The cloth flaw detection method based on the convolutional neural network is characterized by comprising the following steps of:
acquiring a target image of a cloth to be detected;
inputting the target image into a convolutional neural network model obtained by training in advance to obtain thermodynamic diagrams, sizes or offset corresponding to the target image;
determining a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset;
wherein the convolutional neural network model comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
2. The method according to claim 1, wherein the acquiring the target image of the cloth to be inspected comprises:
acquiring an image of the cloth to be detected in real time through a pre-installed camera;
and adjusting the size of the image of the cloth to be detected to obtain the target image.
3. The method of claim 1, wherein the convolutional neural network model is trained by:
acquiring a training data set; wherein the training data set comprises a plurality of cloth images containing flaws and a plurality of cloth images not containing flaws;
inputting the training data set into a target convolutional neural network for iterative training to obtain the convolutional neural network model;
wherein the target convolutional neural network comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
4. The method of claim 1, wherein inputting the target image into a pre-trained convolutional neural network model to obtain a thermodynamic diagram, size, or offset corresponding to the target image comprises:
determining, by the thermodynamic diagram creation module, a thermodynamic diagram corresponding to the target image;
and determining the size or the offset corresponding to the target image according to the thermodynamic diagram through the intermediate module.
5. The method of claim 4, wherein determining a flaw area of a flaw included in the target image based on at least one of the thermodynamic diagram, the size, or the offset comprises:
and determining a flaw area of a flaw included in the target image according to at least one of the thermodynamic diagram, the size or the offset by the output determining module.
6. The method of claim 5, wherein determining, by the output determination module, a flaw area of a flaw included in a target image based on at least one of the thermodynamic diagram, the size, or the offset, comprises:
determining thermodynamic losses according to the thermodynamic diagrams, and determining flaw center points according to the thermodynamic losses;
determining a size loss from the size;
determining an offset loss from the offset;
and superposing the size loss or the offset loss at the flaw center point to determine a flaw area.
7. The method of claim 1, wherein the thermodynamic diagram creation module uses an hourglass CNN as a backbone network.
8. Cloth flaw detection device based on convolutional neural network, characterized by comprising:
the target image acquisition module is used for acquiring a target image of the cloth to be detected;
the input module is used for inputting the target image into a convolutional neural network model which is obtained through training in advance, and obtaining thermodynamic diagrams, sizes or offset corresponding to the target image;
a flaw area determining module, configured to determine a flaw area of a flaw included in a target image according to at least one of the thermodynamic diagram, the size, or the offset;
wherein the convolutional neural network model comprises: the device comprises a thermodynamic diagram establishing module, an intermediate module and an output determining module.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the convolutional neural network-based cloth flaw detection method of any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to implement the convolutional neural network-based fabric flaw detection method of any one of claims 1-7 when executed.
CN202310572870.5A 2023-05-22 2023-05-22 Defect detection method, device, equipment and medium based on convolutional neural network Pending CN116309586A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310572870.5A CN116309586A (en) 2023-05-22 2023-05-22 Defect detection method, device, equipment and medium based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310572870.5A CN116309586A (en) 2023-05-22 2023-05-22 Defect detection method, device, equipment and medium based on convolutional neural network

Publications (1)

Publication Number Publication Date
CN116309586A true CN116309586A (en) 2023-06-23

Family

ID=86781975

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310572870.5A Pending CN116309586A (en) 2023-05-22 2023-05-22 Defect detection method, device, equipment and medium based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN116309586A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270687A (en) * 2020-10-16 2021-01-26 鲸斛(上海)智能科技有限公司 Cloth flaw identification model training method and cloth flaw detection method
CN113255837A (en) * 2021-06-29 2021-08-13 南昌工程学院 Improved CenterNet network-based target detection method in industrial environment
CN113643228A (en) * 2021-05-26 2021-11-12 四川大学 Nuclear power station equipment surface defect detection method based on improved CenterNet network
CN113781391A (en) * 2021-08-02 2021-12-10 南京中科创达软件科技有限公司 Image defect detection method and related equipment
CN114782311A (en) * 2022-03-14 2022-07-22 华南理工大学 Improved multi-scale defect target detection method and system based on CenterNet
CN115410030A (en) * 2022-07-25 2022-11-29 杭州电子科技大学 Target detection method, target detection device, computer equipment and storage medium
CN116029979A (en) * 2022-11-17 2023-04-28 杭州脉尖智能科技有限公司 Cloth flaw visual detection method based on improved Yolov4

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270687A (en) * 2020-10-16 2021-01-26 鲸斛(上海)智能科技有限公司 Cloth flaw identification model training method and cloth flaw detection method
CN113643228A (en) * 2021-05-26 2021-11-12 四川大学 Nuclear power station equipment surface defect detection method based on improved CenterNet network
CN113255837A (en) * 2021-06-29 2021-08-13 南昌工程学院 Improved CenterNet network-based target detection method in industrial environment
CN113781391A (en) * 2021-08-02 2021-12-10 南京中科创达软件科技有限公司 Image defect detection method and related equipment
CN114782311A (en) * 2022-03-14 2022-07-22 华南理工大学 Improved multi-scale defect target detection method and system based on CenterNet
CN115410030A (en) * 2022-07-25 2022-11-29 杭州电子科技大学 Target detection method, target detection device, computer equipment and storage medium
CN116029979A (en) * 2022-11-17 2023-04-28 杭州脉尖智能科技有限公司 Cloth flaw visual detection method based on improved Yolov4

Similar Documents

Publication Publication Date Title
US20220036533A1 (en) Image defect detection method and apparatus, electronic device, storage medium and product
CN112949767B (en) Sample image increment, image detection model training and image detection method
CN112597837B (en) Image detection method, apparatus, device, storage medium, and computer program product
CN113436100B (en) Method, apparatus, device, medium, and article for repairing video
CN115908988B (en) Defect detection model generation method, device, equipment and storage medium
WO2024002187A1 (en) Defect detection method, defect detection device, and storage medium
CN113378969B (en) Fusion method, device, equipment and medium of target detection results
CN115471476A (en) Method, device, equipment and medium for detecting component defects
CN116046790B (en) Defect detection method, device, system, electronic equipment and storage medium
CN116559177A (en) Defect detection method, device, equipment and storage medium
CN113963167B (en) Method, device and computer program product applied to target detection
CN115905016A (en) BIOS Setup search function test method and device, electronic equipment and storage medium
CN116309586A (en) Defect detection method, device, equipment and medium based on convolutional neural network
CN114845099A (en) Method and device for automatically debugging screen image quality
CN115019057A (en) Image feature extraction model determining method and device and image identification method and device
CN117112816B (en) Sorting method, device, equipment and storage medium for security inspection images
CN117115568B (en) Data screening method, device, equipment and storage medium
CN113643257B (en) Image noise detection method, device, equipment, storage medium and program product
CN116309587A (en) Cloth flaw detection method and device, electronic equipment and storage medium
CN117274361A (en) Material surface area measurement method and device, electronic equipment and medium
CN117078708A (en) Training method, device, equipment, medium and product for image detection and model
CN118053027A (en) Defect identification method and device, electronic equipment and storage medium
CN117557777A (en) Sample image determining method and device, electronic equipment and storage medium
CN117611529A (en) Mobile computer shell detection method, device, equipment and storage medium
CN116433926A (en) Film first frame determining method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20230623

RJ01 Rejection of invention patent application after publication