CN116977269A - Image detection method, apparatus, device, readable storage medium, and program product - Google Patents
Image detection method, apparatus, device, readable storage medium, and program product Download PDFInfo
- Publication number
- CN116977269A CN116977269A CN202310431195.4A CN202310431195A CN116977269A CN 116977269 A CN116977269 A CN 116977269A CN 202310431195 A CN202310431195 A CN 202310431195A CN 116977269 A CN116977269 A CN 116977269A
- Authority
- CN
- China
- Prior art keywords
- image
- model
- virtual training
- difference data
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 191
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 449
- 238000005259 measurement Methods 0.000 claims abstract description 96
- 238000012549 training Methods 0.000 claims abstract description 81
- 238000000034 method Methods 0.000 claims abstract description 71
- 238000013442 quality metrics Methods 0.000 claims abstract description 24
- 238000000605 extraction Methods 0.000 claims description 50
- 230000002159 abnormal effect Effects 0.000 claims description 44
- 230000008569 process Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 9
- 230000005856 abnormality Effects 0.000 claims description 7
- 238000007689 inspection Methods 0.000 abstract description 16
- 238000005516 engineering process Methods 0.000 abstract description 11
- 206010063385 Intellectualisation Diseases 0.000 abstract description 9
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 description 19
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000009776 industrial production Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 125000003275 alpha amino acid group Chemical group 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application provides an image detection method, an image detection device, image detection equipment, an image detection readable storage medium and a program product, which can be applied to the fields or scenes such as artificial intelligence technology, industrial quality inspection and the like, wherein the method comprises the following steps: processing the image to be detected by using a target image processing model obtained through training according to the normal sample image and the interpolation sample image of the normal sample image to obtain a reconstructed image of the image to be detected; determining reconstruction quality measurement data according to the image to be detected and the reconstruction image; performing virtual training on the target image processing model by utilizing the image to be detected and the interpolation image of the image to be detected, and determining gradient measurement data according to related data related to the virtual training; image detection results are determined from the reconstructed quality metric data and the gradient metric data. The embodiment of the application can realize the automation and the intellectualization of the image detection, thereby improving the image detection efficiency and reducing the image detection cost.
Description
Technical Field
The present application relates to the field of artificial intelligence, and in particular, to an image detection method, an image detection apparatus, a computer device, a computer readable storage medium, and a computer program product.
Background
Industrial quality inspection refers to the process of quality inspection of industrial products in a manufacturing process to find out different kinds of defects present in the products. The industrial quality inspection can timely and quickly reflect the problems existing in the current production and manufacturing process, and is beneficial to improving the production efficiency of industrial products.
The manual quality inspection refers to a method for detecting quality of an industrial product to be detected or an image to be detected which is shot by an industrial product by a professional quality inspection person. Because the manual quality inspection is highly dependent on the professional technology and experience of professional quality inspection personnel, the detection efficiency of the manual quality inspection method is lower, and the detection cost is higher.
Disclosure of Invention
The embodiment of the application provides an image detection method, an image detection device, image detection equipment, a readable storage medium and a program product, which can realize the automation and the intellectualization of image detection, thereby effectively improving the detection efficiency of image detection and reducing the detection cost.
In one aspect, an embodiment of the present application provides an image detection method, including:
performing feature extraction processing on an image to be detected by using a feature extraction network of a target image processing model to obtain image features of the image to be detected;
Performing image reconstruction processing according to the image characteristics by using an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected; the target image processing model is obtained by training a normal sample image and an interpolation sample image of the normal sample image;
determining reconstruction quality metric data according to the image to be detected and the reconstruction image;
obtaining an interpolation image of the image to be detected, virtually training the target image processing model by using the image to be detected and the interpolation image, and determining gradient measurement data of the target image processing model according to related data related to virtual training;
and determining an image detection result of the image to be detected according to the reconstruction quality measurement data and the gradient measurement data, wherein the image detection result is used for indicating whether the image to be detected is a normal image or an abnormal image.
In one aspect, an embodiment of the present application provides an image detection apparatus, including:
the processing unit is used for carrying out feature extraction processing on the image to be detected by utilizing a feature extraction network of the target image processing model to obtain image features of the image to be detected;
The processing unit is further used for performing image reconstruction processing according to the image characteristics by utilizing an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected; the target image processing model is obtained by training a normal sample image and an interpolation sample image of the normal sample image;
a computing unit for determining reconstruction quality metric data from the image to be detected and the reconstructed image;
the acquisition unit is used for acquiring an interpolation image of the image to be detected, virtually training the target image processing model by utilizing the image to be detected and the interpolation image, and determining gradient measurement data of the target image processing model according to related data related to virtual training;
the computing unit is further configured to determine an image detection result of the image to be detected according to the reconstruction quality metric data and the gradient metric data, where the image detection result is used to indicate that the image to be detected is a normal image or an abnormal image.
In one aspect, an embodiment of the present application provides a computer device, including: the image detection method comprises a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are mutually connected, executable program codes are stored in the memory, and the processor is used for calling the executable program codes to realize the image detection method provided by the embodiment of the application.
Correspondingly, the embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores instructions which, when run on a computer, cause the computer to realize the image detection method provided by the embodiment of the application.
Accordingly, embodiments of the present application also provide a computer program product comprising a computer program or computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer program or the computer instructions from the computer readable storage medium, and the processor executes the computer program or the computer instructions, so that the computer device implements the image detection method provided by the embodiment of the application.
According to the method, a target image processing model obtained through training according to a normal sample image and an interpolation sample image is utilized to process an image to be detected, a reconstructed image of the image to be detected is obtained, and reconstruction quality measurement data is determined according to the image to be detected and the reconstructed image; performing virtual training on the target image processing model by utilizing the image to be detected and the interpolation image of the image to be detected, and determining gradient measurement data of the target image processing model according to related data related to the virtual training; and determining an image detection result indicating that the image to be detected is a normal image or an abnormal image according to the reconstruction quality measurement data and the gradient measurement data. The image detection method provided by the embodiment of the application can accurately determine whether the image to be detected is a normal image according to the related data (including the reconstructed image of the image to be detected, the interpolation image and the related data related to the virtual training) related to the image processing process to be detected by the image processing model, thereby realizing automation and intellectualization of image detection, effectively improving the image detection efficiency and reducing the image detection cost; the image processing model is obtained only by training the normal sample image, and the model training efficiency of the image processing model is higher because the product yield of the industrial production line is higher, the acquisition paths of the normal sample image are more, and the acquisition speed is higher; because the image processing model is obtained according to the training of the normal sample image, the reconstruction quality measurement data and the gradient measurement data obtained by processing the normal image by using the image processing model are usually smaller, and the reconstruction quality measurement data and the gradient measurement data obtained by processing the abnormal image by using the image processing model are usually larger, the reconstruction quality measurement data and the gradient measurement data obtained by processing the image to be detected can be used for determining whether the image to be detected is the normal image according to the image processing model; by combining the two-dimensional data of the reconstructed quality measurement data and the gradient measurement data, whether the image to be detected is a normal image or not can be determined more accurately.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a system architecture of an image detection system according to an embodiment of the present application;
fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a virtual training method according to an embodiment of the present application;
FIG. 4 illustrates a data determination method provided by an embodiment of the present application;
FIG. 5 illustrates a method for determining gradient metric data provided by an embodiment of the present application;
fig. 6 illustrates an image detection method provided by an embodiment of the present application;
FIG. 7 is a schematic flow chart of a model training method according to an embodiment of the present application;
FIG. 8A illustrates another model training method provided by an embodiment of the present application;
FIG. 8B illustrates another model training method provided by an embodiment of the present application;
Fig. 9 is a schematic structural diagram of an image detection device according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be noted that the descriptions of "first," "second," and the like in the embodiments of the present application are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a technical feature defining "first", "second" may include at least one such feature, either explicitly or implicitly.
The traditional industrial quality inspection is generally to perform manual visual inspection on an image to be inspected by a quality inspection worker, and the quality inspection worker judges whether the image to be inspected is an abnormal image according to professional technology and experience, however, the quality inspection method has low inspection efficiency and high inspection cost. In some cases, the image to be detected may be detected using an anomaly detection model. The abnormal detection model is obtained by training an abnormal sample image and a normal sample image containing manual labeling labels, and the abnormal sample image is usually difficult to obtain because the yield of products of factory generated lines is high, and a great deal of cost is required for manually labeling the abnormal sample image. Based on the above, the application provides an image detection method, which is characterized in that a target image processing model obtained by training an interpolation sample image according to a normal sample image and a normal sample image is utilized to process an image to be detected, so as to obtain a reconstructed image of the image to be detected; determining reconstruction quality measurement data according to the image to be detected and the reconstruction image; performing virtual training on the target image processing model by utilizing the image to be detected and the interpolation image of the image to be detected, and determining gradient measurement data of the target image processing model according to related data related to the virtual training; and determining an image detection result for indicating that the image to be detected is a normal image or an abnormal image according to the reconstruction quality measurement data and the gradient measurement data. Compared with the method of manual quality inspection, the image detection method provided by the application can accurately determine whether the image to be detected is a normal image or not through the related data obtained by processing the image to be detected by the image processing model, thereby realizing automation and intellectualization of image detection, effectively improving the detection efficiency and accuracy of image detection and saving the labor cost; in addition, the image processing model is obtained only by training the normal sample image, and the normal sample image is easy to obtain due to the high product yield of the industrial production line, so that the model training efficiency of the image processing model obtained by training the normal sample image is high.
The image detection method provided by the application can be applied to the technical field of artificial intelligence. Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In addition, the Computer Vision (CV) technology in the artificial intelligence technology refers to that a camera and a Computer are used to replace human eyes to perform machine Vision such as recognition, following and measurement on a target, and further perform image processing, so that the target is an image more suitable for human eyes to observe or transmit to an instrument for detection. In the application, the image to be detected can be obtained by shooting an object to be detected (such as an industrial product) by using machine vision equipment such as a camera, and the image to be detected is input into a target image processing model for processing, so that an image detection result is obtained, and a computer can conveniently perform further operation according to whether the image to be detected is a normal image or not.
The image detection method provided by the application can also be applied to cloud technology scenes. Cloud technology (close technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. For example: according to the image detection method provided by the application, the image to be detected can be obtained through cloud data or cloud application, and the image to be detected is processed by utilizing the target image processing model, so that an image detection result of the image to be detected is obtained. Also for example: the image processing model may be trained from the normal sample image and the interpolated sample image of the normal sample image using a cloud server.
In a feasible embodiment, the image detection method provided by the embodiment of the application can be based on a blockchain technology, wherein the blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The system comprises a series of blocks (blocks) which are mutually connected according to the sequence of the generated time, the new blocks are not removed once being added into a Block chain, and record data submitted by nodes in the Block chain system are recorded in the blocks. Optionally, data (e.g., images to be detected, reconstructed quality metric data, etc.) generated by performing the image detection method may be stored in a blockchain network in the form of blocks, or a plurality of normal sample images may be obtained from the blockchain network for use in other traffic scenarios. In addition, the device performing the image detection method may be a node device in a blockchain network.
The architecture of the image detection system provided by the embodiment of the application will be described with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a schematic diagram of a system architecture of an image detection system according to an embodiment of the application. As shown in fig. 1, the image detection system includes a feature extraction network and an image reconstruction network, and an image detection device 101, a sensing device 102, and a database 103, wherein the image detection device 101 includes the image detection device 101 and the sensing device 102, and the image detection device 101 and the database 103 may be connected through a network, for example, a local area network, a wide area network, a mobile internet, and the like. The image detection device 101 may acquire an image to be detected from the sensing device 102 and the database 103; the sensing device 102 may generate an image to be detected and send the image to the image detection device 101; the database 103 may store images to be detected and normal sample images. Wherein,,
the image detection device 101 may be a server, which may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms, but is not limited thereto. The present application is not limited with respect to the number of servers.
The sensing device 102 may be a device for generating an image to be detected, and the sensing device 102 may be a camera, an infrared camera, an ultraviolet camera, or the like, or may be a handheld device (such as a smart phone, a tablet computer), a computing device (such as a personal computer (personal computer, PC), a vehicle-mounted terminal, an intelligent voice interaction device, a wearable device, or other intelligent apparatuses, which include a camera function, but is not limited thereto.
Database 103 may be a built-in or self-contained database of image detection device 101; the database 103 may also be a peripheral database connected to the image detection device 101, for example, a cloud database (i.e., a database deployed in the cloud), and specifically may be deployed based on any one of a private cloud, a public cloud, a hybrid cloud, an edge cloud, and so on, so that the cloud database has different focused functions.
The working principle of the image detection system shown in fig. 1 will be described in detail as follows:
the image detection device 101 may acquire an image to be detected from the sensing device 102 or the database 103, where the image detection device 101 includes a target image processing model, and the target image processing model is obtained by training a normal sample image and an interpolated sample image of the normal sample image; performing feature extraction processing on the image to be detected by using a feature extraction network of the target image processing model to obtain image features of the image to be detected; performing image reconstruction processing according to the image characteristics by using an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected; determining reconstruction quality measurement data according to the image to be detected and the reconstruction image; the image detection device 101 may acquire an interpolation image of an image to be detected, virtually train the target image processing model by using the image to be detected and the interpolation image, and determine gradient metric data of the target image processing model according to related data related to the virtual training; and determining an image detection result for indicating that the image to be detected is a normal image or an abnormal image according to the reconstruction quality metric data and the gradient metric data. The image detection method provided by the embodiment of the application can accurately determine whether the image to be detected is a normal image through the related data obtained by processing the image to be detected by the image processing model, thereby realizing the automation and the intellectualization of the image detection, effectively improving the efficiency of the image detection and reducing the cost consumed by the image detection.
It will be understood that the architecture diagram of the image detection system described in the embodiment of the present application is for more clearly describing the image detection method of the embodiment of the present application, and does not constitute a limitation of the image detection method provided in the embodiment of the present application. For example, the image detection method provided by the embodiment of the present application may be performed by a server or a server cluster other than the image detection apparatus 101, in addition to the image detection apparatus 101. For example, the image detection method provided by the embodiment of the present application may also be performed by the sensing device 102 having the computing capability and the model processing capability separately. Those of ordinary skill in the art will appreciate that the number of sensing devices and image detection devices in fig. 1 is merely illustrative. Any number of sensing devices and image detection devices may be configured as desired for a service implementation. Moreover, with the evolution of the system architecture and the appearance of new service scenes, the image detection method provided by the embodiment of the application is also applicable to similar technical problems.
Referring to fig. 2, fig. 2 is a flowchart of an image detection method according to an embodiment of the application. The image detection method may be implemented by the image detection apparatus 101 described above, or may be implemented by another apparatus. The flow of the image detection method provided in the embodiment of the application includes but is not limited to:
S201, performing feature extraction processing on an image to be detected by using a feature extraction network of a target image processing model to obtain image features of the image to be detected.
In the embodiment of the application, the image to be detected may be a surface image shot for an object to be detected, and the object to be detected may be an industrial product needing quality detection, for example: the image to be detected can also be an image which needs quality detection, such as equipment parts, mechanical finished products and the like. If the image to be detected is a surface image shot for the object to be detected, the image detection result corresponding to the image to be processed can be used for indicating that the image to be detected is a normal image or an abnormal image, and can also be used for indicating that the image to be detected in the image to be detected is a normal object or an abnormal object; if the image to be detected is an image requiring quality detection, an image detection result corresponding to the image to be processed can be used for indicating that the image to be detected is a normal image or an abnormal image. The feature extraction network of the target image processing model is utilized to perform feature extraction processing on the image to be processed, so that the image feature of the image to be detected (the image feature can be called as a feature vector or hidden variable of the image to be processed) can be obtained. The target image processing model is trained using a normal sample image and an interpolated sample image of the normal sample image. (the training flow of the target image processing model is the same as the model training flow described in steps S701 to S706 described below), the target image processing model further includes an image reconstruction network.
In one embodiment, the specific structure of the target image processing model may be determined from the structure of the convolutional neural network (Convolutional Neural Network, CNN) model. Convolutional neural networks are a type of feedforward neural network that includes convolutional computation and has a deep structure, and are one of representative algorithms for deep learning. The feature extraction network of the target image processing model may include a plurality of convolution layers, a pooling layer, and a nonlinear activation function layer.
S202, performing image reconstruction processing according to the image characteristics by using an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected.
In the embodiment of the application, the image reconstruction network of the target image processing model can perform image reconstruction processing according to the image characteristics to obtain a reconstructed image of the image to be detected. Since the reconstructed image is reconstructed according to the image characteristics of the image to be detected, the similarity between the reconstructed image and the image to be detected is generally high, and the difference between the reconstructed image and the image to be detected can reflect the reconstruction effect of the target image processing model.
In one embodiment, the image reconstruction network of the target image processing model may include a plurality of deconvolution layers, nonlinear activation function layers.
S203, determining reconstruction quality measurement data according to the image to be detected and the reconstruction image.
In the embodiment of the application, the reconstruction quality measurement data is determined according to the image to be detected and the reconstruction image, and the reconstruction quality measurement data reflects the difference degree between the image to be detected and the reconstruction image.
In an embodiment, the reconstruction quality metric data of the image to be detected may be determined by calculating the difference between each pixel in the image to be detected and the corresponding pixel in the reconstructed image, and the specific calculation manner may be as shown in the following formula (1):
in the formula (1), D recons Representing reconstruction quality metric data of an image to be detected, H representing the number of pixels in the vertical direction of the image to be processed, W representing the number of pixels in the horizontal direction of the image to be processedThe number, h, represents pixel position data in the current vertical direction (h is a positive integer), w represents pixel position data in the current horizontal direction (w is a positive integer), x represents an image to be processed, R (x) represents a reconstructed image of the image to be processed, and then expression (1) represents that the reconstructed quality metric data of the image to be detected is equal to an average value of differences between the value of each pixel point in the image to be processed and the value of the corresponding pixel point in the reconstructed image.
S204, obtaining an interpolation image of the image to be detected.
In the embodiment of the application, the interpolation image is obtained by carrying out interpolation processing on the image to be detected, and the size of the interpolation image is the same as that of the image to be detected. The specific calculation method of the interpolation image can be as follows:
x′＝interplate(x)(2)
in the formula (2), x represents an image to be detected, x 'represents an interpolation image of the image to be detected, and the formula (2) represents interpolation processing of the image to be detected x to obtain an interpolation image x'.
In an embodiment, an interpolation method such as a nearest neighbor interpolation method may be used to perform interpolation processing on an image to be detected, so as to obtain an interpolated image. Nearest-neighbor interpolation, also known as zero-order interpolation, is to make the gray value of the transformed image pixel equal to the gray value of the input pixel nearest to it, and is applicable to scaling of images.
S205, performing virtual training on the target image processing model by using the image to be detected and the interpolation image, and determining gradient measurement data of the target image processing model according to related data related to the virtual training.
In the embodiment of the application, the virtual training refers to performing virtual adjustment on the model parameters of the target image processing model in the image detection process of the target image processing model, and the model parameters after virtual adjustment are only temporarily used in the current detection process. Virtual training does not actually change the model parameters of the target image processing model. For example: in the 1 st image detection process, virtually training a target image processing model (the model parameter is a1 at the moment) according to an image to be processed (x 1) and an interpolation image of the image to be processed, virtually adjusting the model parameter in the target image processing model to a2, and determining gradient measurement data of the target image processing model according to related data (such as virtually adjusted model parameter a 2) related to virtual training; in the actual detection process of the 2 nd time, a target image processing model with a model parameter of a1 is still used, the target image processing model (with the model parameter of a1 at the moment) is virtually trained by utilizing the image to be processed (x 2) and an interpolation image of the image to be processed, the model parameter in the target image processing model is virtually adjusted to a3, and gradient measurement data of the target image processing model are determined according to related data (such as the virtually adjusted model parameter a 3) related to the virtual training. The gradient measurement data can reflect the model parameter adjustment gradient (namely the magnitude of the model parameter adjustment amplitude) of the image processing model in the virtual training process, and because the target image processing model is obtained by training according to a large number of normal sample images and the characteristic difference among a plurality of normal images is usually smaller, when the image to be detected is the normal image, the model parameter adjustment amplitude of the model is smaller, namely the gradient measurement data of the image processing model is smaller when the image to be detected and the interpolation image corresponding to the image to be detected are used for carrying out virtual training on the target image processing model; when the image to be detected is an abnormal image, the characteristic difference between the abnormal image and the normal image is usually larger, so that when the image to be detected and the interpolation image corresponding to the image to be detected are utilized to virtually train the target image processing model, the model parameter adjustment amplitude of the model is larger, namely the gradient measurement data of the image processing model is larger. Therefore, the gradient metric data may be one of the bases for judging whether the image to be detected is a normal image or an abnormal image.
In an embodiment, multiple virtual training may be performed on the target image processing model by using the image to be detected and the interpolation image, and gradient metric data of the target image processing model may be determined according to related data related to the virtual training, where the implementation manner may be as follows: in the I virtual training stage, the image processing model after virtual training in the I-1 virtual training stage is used for respectively processing the image to be detected and the interpolation image to obtain the reference image characteristics and the reconstruction reference image of the image to be detected, the interpolation image characteristics and the reconstruction interpolation image of the interpolation image; determining model difference data and target model parameter gradients of an I-1 virtual training stage according to the image to be detected, the reference image characteristics, the reconstructed reference image, the interpolation image characteristics, the reconstructed interpolation image and model parameters of an image processing model after virtual training of the I-1 virtual training stage; performing virtual training again on the image processing model after virtual training in the I-1 virtual training stage by using the model difference data in the I virtual training stage to obtain the image processing model after virtual training in the I virtual training stage; determining gradient measurement data of a target image processing model according to the target model parameter gradients of each virtual training stage; wherein I is any positive integer less than or equal to T, T is the total number of virtual training stages, and T is an integer greater than 1; when I is 1, the image processing model after virtual training in the I-1 virtual training stage is a target image processing model. The multiple virtual training process may be as shown in fig. 3. Assuming that the total number of virtual training stages T is 2, and the model parameter of the target image processing model is theta 1; when the I is 1, the image processing model after virtual training in the I-1 virtual training stage is a target image processing model, and the target image processing model is utilized to process the image x1 to be detected and the interpolation image x1 'to obtain the reference image characteristic and the reconstruction reference image of the image x1 to be detected and the interpolation image characteristic and the reconstruction interpolation image of the interpolation image x 1'; determining model difference data and target model parameter gradients in a 1 st virtual training stage according to an image to be detected, a reference image characteristic, a reconstructed reference image, an interpolation image characteristic, a reconstructed interpolation image and model parameters theta 1 of a target image processing model, and performing virtual training on the target image processing model by using the model difference data in the 1 st virtual training stage (assuming that a result of the virtual training is that the model parameters theta 1 of the target image processing model are adjusted to theta 2), so as to obtain an image processing model after the 1 st virtual training stage (at this time, the model parameters of the image processing model are theta 2); when I is 2, respectively processing the image x1 to be detected and the interpolation image x1 'thereof by using an image processing model (model parameter is theta 2) after virtual training in the 1 st virtual training stage to obtain reference image characteristics and reconstruction reference images of the image x1 to be detected and interpolation image characteristics and reconstruction interpolation images of the interpolation image x 1'; according to the image to be detected, the reference image characteristic, the reconstructed reference image, the interpolation image characteristic, the reconstructed interpolation image and the model parameter theta 2 of the image processing model after virtual training in the 1 st virtual training stage, determining model difference data and target model parameter gradients in the 2 nd virtual training stage, and virtually training the image processing model after virtual training in the 1 st virtual training stage by utilizing the model difference data in the 2 nd virtual training stage (the model parameter theta 2 of the image processing model is supposed to be adjusted to be theta 3 as a result of virtual training), so as to obtain the image processing model after virtual training in the 2 nd virtual training stage (at the moment, the model parameter of the image processing model is theta 3).
In an embodiment, a back propagation method may be used to implement virtual training on the image processing model after virtual training in the I-1 virtual training stage by using the model difference data in the I virtual training stage, to obtain the image processing model after virtual training in the I virtual training stage.
In an embodiment, according to the to-be-detected image, the reference image feature, the reconstructed reference image, the interpolation image feature, the reconstructed interpolation image, and the model parameters of the image processing model after virtual training in the I-1 th virtual training stage, the implementation manner of determining the model difference data and the target model parameter gradient in the I-1 th virtual training stage may be: determining first feature difference data according to the reference image features and the interpolation image features; determining first reconstruction difference data according to the difference between the image to be detected and the reconstruction reference image and the difference between the interpolation image and the reconstruction interpolation image; and determining model difference data and target model parameter gradients in the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and model parameters of the image processing model after virtual training in the I-1 virtual training stage. Assuming that the image to be detected is x, the interpolation image of the image to be detected is x', I is 1, and the image processing model after virtual training in the I-1 virtual training stage is the target image processing model, the feature extraction network of the target image processing model can be utilized to perform feature extraction processing on the image to be processed, so as to obtain the reference image feature of the image to be processed, wherein the reference image feature is shown in the following formula (3):
z(x)＝ encoder (；θ e )(3)
In the formula (3), z (x) represents the reference image characteristic of the image to be processed, x represents the image to be processed, and θ e Network parameters of a feature extraction network representing a target image processing model. Performing feature extraction processing on the interpolation image by using a feature extraction network of the target image processing model to obtain interpolation image features of the interpolation image, wherein the interpolation image features are represented by the following formula (4):
z′(x′)＝f encoder (x′；θ e ) (4)
in the formula (4), z ' (x ') represents an interpolation image characteristic of the interpolation image, x ' represents the interpolation image, θ e Network parameters of a feature extraction network representing a target image processing model. Since the interpolation image is determined based on the image to be detected, the reference image feature of the image to be detected and the interpolation image feature of the interpolation image have similar distribution, and the first feature difference data may be determined based on the reference image feature and the interpolation image feature by a calculation method shown in the following formula (5).
L dist ＝|z(x)-z′(x′)| (5)
In the formula (5), L dist Representing first feature difference data, z (x) represents a reference image feature, and z '(x') represents an interpolated image feature. Performing image reconstruction processing according to the reference image characteristics by using an image reconstruction network of the target image processing model to obtain a reconstructed reference image; the image reconstruction network using the target image processing model performs image reconstruction processing according to the interpolation image characteristics to obtain a method for reconstructing an interpolation image, which can be represented by the following formulas (6) and (7):
R(z(x))＝f decoder (z(x)；θ d ) (6)
R′(z′(x′))＝f decoder (z′(x′)；θ d ) (7)
In the formula (6), R (z (x)) represents a reconstructed reference image of the image to be detected, z (x) represents a reference image feature, θ d Network parameters of an image reconstruction network representing a target image processing model. In the formula (7), R ' (z ' (x ') represents a reconstructed interpolation image of the interpolation image, z ' (x ') represents an interpolation image feature, θ d Network parameters of an image reconstruction network representing a target image processing model. The method for determining the first reconstruction difference data according to the difference between the image to be detected and the reconstruction reference image, the difference between the interpolation image and the reconstruction interpolation image can be as shown in the following formula (8):
L recons ＝|x-R(z(x))|+|x′-R′(z′(x′))|)/2 (8)
in the formula (8), L recons Representing first reconstructed difference data, x representing the image to be detected, R (z (x)) representing the reconstructed reference image, x 'representing the interpolated image, R' (z '(x')) representing the reconstructed interpolated image. As shown in fig. 4, the process of determining the first feature difference data and the first reconstructed difference data of the I-th virtual training stage is: inputting an image x to be processed into a feature extraction network of an image processing model after virtual training in an I-1 virtual training stage to perform feature extraction processing to obtain a reference image feature z (x); inputting an interpolation image x ' corresponding to the image to be processed into a feature extraction network of an image processing model after virtual training in an I-1 virtual training stage to perform feature extraction processing, so as to obtain an interpolation image feature z ' (x '); determining first feature difference data according to the reference image features and the interpolation image features; inputting the reference image characteristic z (x) into an image reconstruction network of an image processing model virtually trained in the I-1 virtual training stage to perform image reconstruction processing to obtain a reconstructed reference image R (z (x)); inputting the interpolation image characteristic z ' (x ') into an image reconstruction network of an image processing model virtually trained in the I-1 virtual training stage to perform image reconstruction processing to obtain a reconstructed interpolation image R ' (z ' (x ')); and determining first reconstruction difference data according to the image to be detected, the reconstruction reference image, the interpolation image and the reconstruction interpolation image.
In one embodiment, the first feature difference data may be determined from the reference image features and the interpolated image features using a calculation of the mean absolute error (Mean Absolute Error, MAE), or may be determined using a calculation of the mean square error (Mean Square Error, MSE).
In one embodiment, the method for calculating the difference between the image to be detected and the reconstructed reference image and the difference between the interpolated image and the reconstructed interpolated image in equation (8) may be to calculate the sum of the differences corresponding to all the pixels in the two images, and then divide the sum by the number of pixels in one image. The specific implementation formula may be as in formula (1) above.
In an embodiment, determining the implementation manner of the model difference data and the target model parameter gradient of the I-th virtual training stage according to the first feature difference data, the first reconstruction difference data, and the model parameters of the image processing model after the virtual training of the I-1-th virtual training stage may be: determining an intermediate model parameter gradient of the first virtual training stage according to the first characteristic difference data, the first reconstruction difference data and model parameters of the image processing model after virtual training in the first-1 virtual training stage; when I is 1, determining gradient difference data of the I virtual training stage according to the intermediate model parameter gradient of the I virtual training stage; when I is greater than 1, acquiring recorded intermediate model parameter gradients of each virtual training stage from the 1 st virtual training stage to the I-1 st virtual training stage, and determining gradient difference data of the I th virtual training stage according to the intermediate model parameter gradients of each virtual training stage from the 1 st virtual training stage to the I-1 st virtual training stage and the intermediate model parameter gradients of the I th virtual training stage; determining model difference data of an I virtual training stage according to the first characteristic difference data, the first reconstruction difference data and gradient difference data of the I virtual training stage; and determining the target model parameter gradient of the I virtual training stage according to the model difference data of the I virtual training stage and the model parameters of the image processing model after virtual training of the I-1 virtual training stage.
In an embodiment, determining the implementation manner of the intermediate model parameter gradient of the I-th virtual training stage according to the first feature difference data, the first reconstruction difference data, and the model parameters of the image processing model after virtual training in the I-1-th virtual training stage may be: and determining a first virtual training parameter according to the first characteristic difference data and the first reconstruction difference data, and determining an intermediate model parameter gradient of the I-th virtual training stage according to the first virtual training parameter and the model parameter of the image processing model after virtual training in the I-1-th virtual training stage. The calculation method of the first virtual training parameter may be as follows formula (9):
L vstage1 ＝ dist +1* recons (9)
in the formula (9), L vstage1 Representing a first virtual training parameter, L dist Representing first characteristic difference data, L recons And beta 1 is a first balance parameter for balancing the proportion of the first characteristic difference data and the first reconstruction difference data in the first virtual training parameter. The calculation method of the intermediate model parameter gradient in the I-th virtual training stage can be shown as the following formula (10):
in the formula (10), the amino acid sequence of the compound,intermediate model parameter gradient, L, representing the I-th virtual training phase vstage1 Representing the first virtual training parameter, θ d Model parameters representing the image processing model after virtual training in the I-1 th virtual training stage. And (3) performing bias derivation on the first virtual training parameters and the model parameters of the image processing model after virtual training in the I-1 virtual training stage to obtain the intermediate model parameter gradient in the I virtual training stage.
In one embodiment, model parameters θ of the image processing model after virtual training in the I-1 virtual training stage d Network model parameters of the network may be reconstructed for images in the image processing model.
In one embodiment, when I is 1, the virtual I is determined based on the intermediate model parameter gradient of the virtual I training stageThe gradient difference data of the training stage can be: when I is 1, determining the intermediate model parameter gradient of the I-th virtual training phase as gradient difference data of the I-th virtual training phase, assuming that: the intermediate model parameter gradient of the 1 st virtual training stage is L vstage1 The gradient difference data of the 1 st virtual training stage is L reg ＝ vstage1 The method comprises the steps of carrying out a first treatment on the surface of the When I is greater than 1, acquiring recorded intermediate model parameter gradients of each of the 1 st virtual training stage to the I-1 st virtual training stage, and determining gradient difference data of the I-th virtual training stage according to the intermediate model parameter gradients of each of the 1 st virtual training stage to the I-1 st virtual training stage and the intermediate model parameter gradients of the I-th virtual training stage, wherein the calculation method for determining the gradient difference data of the I-th virtual training stage can be represented by the following formula (11):
In the formula (11), L reg Gradient difference data representing an I-th virtual training stage, sim (a, B) representing determining a similarity distance between a and B using a similarity calculation function, K representing a kth layer (K is a positive integer of K or less) of an image reconstruction network in an image processing model,representing intermediate model parameter gradients in a kth layer of the image reconstruction network during an ith virtual training phase; />Representing the average intermediate model parameter gradient in the kth layer of the image reconstruction network from the 1 st virtual training stage to the I-1 st virtual training stage. Equation (11) represents the difference between the mean value of the intermediate model parameter gradient of the 1 st virtual training stage and the intermediate model parameter gradient of the 1 st to I-1 st virtual training stages. The calculation method for determining the similarity distance between A and B by using sim (A, B) function can be cosine similarity calculation method shown in the following formula (12)：
In the formula (12), A, B denotes two objects of a distance to be calculated, and n denotes a dimension that the object has.
In an embodiment, a calculation method for determining model difference data of an I-th virtual training stage according to the first feature difference data, the first reconstruction difference data and gradient difference data of the I-th virtual training stage may be as shown in the following formula (13):
L vstage2 ＝L dist +β*L recons +γ*L reg (13)
L vstage2 Model difference data representing the I-th virtual training stage, L dist Representing first characteristic difference data, L recons Representing the first reconstructed difference data, L reg Gradient difference data representing the I-th virtual training phase. β2 and γ2 are balance parameters for adjusting the specific weights of the first feature difference data, the first reconstruction difference data, and the gradient difference data of the I-th virtual training stage in the model difference data.
In one embodiment, the implementation manner of determining the target model parameter gradient of the ith virtual training stage according to the model difference data of the ith virtual training stage and the model parameters of the image processing model after virtual training in the (I-1) th virtual training stage may be as shown in the following formula (14):
formula (14) represents model parameter bias processing of the model difference data of the I virtual training stage and the image processing model after virtual training of the I-1 virtual training stage to obtain a target model parameter gradient of the I virtual training stage, wherein D I A gradient of the target model parameters representing the I-th virtual training phase,model difference data, θ, representing the I-th virtual training phase d Model parameters representing the image processing model after virtual training in the I-1 th virtual training stage.
In an embodiment, the implementation manner of determining the gradient metric data of the target image processing model according to the target model parameter gradient of each virtual training stage may be: when the total number T of virtual training stages is equal to 1, determining the gradient of the parameters of the target model in the 1 st virtual training stage as gradient measurement data of the target image processing model; and when the total number T of virtual training stages is greater than 1, determining gradient measurement data of the target image processing model according to the target model parameter gradients of each virtual training stage from the 1 st virtual training stage to the T-1 st virtual training stage and the target model parameter gradient of the T virtual training stage. When T is greater than 1, the method of determining gradient metric data of the target image processing model may be as follows formula (15):
equation (15) represents the average value corresponding to the target model parameter gradient of each virtual training stage from the 1 st virtual training stage to the T-1 st virtual training stage, calculates the relative distance with the target model parameter gradient of the T virtual training stage, and obtains gradient metric data of the target image processing model, wherein T represents the total stage data of the virtual training, T represents the current virtual training stage number, K represents the network layer number of the image reconstruction network of the image processing model, sim (A, B) represents the similarity distance between A and B determined by using a similarity calculation function, Representing the average value of the model parameter gradients of the k-th layer of the image reconstruction network for each of the 1 st through T-1 st virtual training phases,representing the target model parameter gradient of the T virtual training phase.
In one embodiment, the sim (a, B) similarity calculation function in the above formula (15) may be a cosine calculation function (the specific calculation manner is shown in the above formula (12)), or may be another distance measurement algorithm.
Referring to fig. 5, a method of determining gradient metric data for a target image processing model is shown. Assuming that the model parameter of the target image processing model is a1 (the network parameter of the image reconstruction network of the target image processing model is θ1), the total virtual training stage number T is 20, the image to be detected is x, the interpolation image of the image to be detected is x', the method for determining the gradient measurement data of the target image processing model is as follows: in the 1 st virtual training stage, inputting an image to be detected and an interpolation image into a target image processing model for feature extraction processing, and determining first feature difference data of the 1 st virtual training stage according to a processing result; performing image reconstruction processing by using a target image processing model, determining a first reconstruction difference number of a1 st virtual training stage according to a processing result, determining an intermediate model parameter gradient of the 1 st virtual training stage according to the first characteristic difference data, the first reconstruction difference data and a network parameter theta 1 of an image reconstruction network of the target image processing model, and determining the intermediate model parameter gradient as gradient difference data (Lreg 1) of the 1 st virtual training stage; determining model difference data (Lvstage 2-1) of the 1 st virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the gradient difference data of the 1 st virtual training stage; determining a target model parameter gradient (D1) of the 1 st virtual training stage according to a network parameter theta 1 of an image reconstruction network of the target image processing model and model difference data (Lvstage 2-1) of the 1 st virtual training stage; performing model parameter virtual adjustment on the target image processing model by using model difference data (Lvstage 2-1) of the 1 st virtual training stage to obtain an image processing model (called a first virtual training model for short) after virtual training of the 1 st virtual training stage, wherein the model parameter of the image processing model is a2 (wherein, the network parameter of an image reconstruction network is theta 2); in the 2 nd virtual training stage, inputting the image to be detected and the interpolation image into a first virtual training model for feature extraction processing, and determining first feature difference data of the 2 nd virtual training stage according to a processing result; performing image reconstruction processing by using a first virtual training model, determining first reconstruction difference data of a2 nd virtual training stage according to a processing result, determining an intermediate model parameter gradient of the 2 nd virtual training stage according to the first characteristic difference data, the first reconstruction difference data and a network parameter theta 2 of an image reconstruction network of the first virtual training model, and determining gradient difference data (Lreg 2) of the 2 nd virtual training stage according to an intermediate model parameter gradient Lreg1 of the 1 st virtual training stage (the intermediate model parameter gradient of the 1 st virtual training stage is equal to the gradient difference data of the 1 st virtual training stage) and an intermediate model parameter gradient of the 2 nd virtual training stage; determining model difference data (Lvstage 2-2) of the 2 nd virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the gradient difference data of the 2 nd virtual training stage; determining a target model parameter gradient (D2) of the 2 nd virtual training stage according to a network parameter theta 1 of an image reconstruction network of the target image processing model, model difference data (Lvstage 2-1) of the 1 st virtual training stage, a network parameter theta 2 of the image reconstruction network of the first virtual training model, and model difference data (Lvstage 2-2) of the 2 nd virtual training stage; and performing model parameter virtual adjustment on the first virtual sequence model by using model difference data (Lvstage 2-2) of the 2 nd virtual training stage to obtain an image processing model (called a second virtual training model for short) after virtual training of the 2 nd virtual training stage, wherein the model parameter of the image processing model is a3 (wherein the network parameter of the image reconstruction network is theta 3). The method for determining the target model parameter gradient and the model difference data in the 3 rd virtual training stage to the 20 th virtual training stage is similar to that of the 2 nd virtual training model (the target model parameter gradient in the 20 th virtual training stage is D20). Gradient metric data (drags) of the target image processing model is determined from 20 target model parameter gradients in the 1 st virtual training stage through the 20 th virtual training stage.
S206, determining an image detection result of the image to be detected according to the reconstruction quality measurement data and the gradient measurement data, wherein the image detection result is used for indicating whether the image to be detected is a normal image or an abnormal image.
In the embodiment of the application, the image detection result is used for indicating that the image to be detected is a normal image or an abnormal image, and when the image to be detected is an image shot for an object to be detected (such as an industrial product, etc.), the image detection result can also be used for indicating that the object to be detected is a normal object or an abnormal object.
In one embodiment, the anomaly determination score may be determined from the reconstructed quality metric data and the gradient metric data; if the anomaly determination score is smaller than the score threshold, determining that the image detection result of the image to be detected is a first image detection result, wherein the first image detection result indicates that the image to be detected is a normal image; if the anomaly determination score is greater than or equal to the score threshold, determining that the image detection result of the image to be detected is a second image detection result, wherein the second image detection result indicates that the image to be detected is an anomaly image. The image detection method provided by the embodiment of the application can determine the image detection result of the image to be detected according to the reconstruction quality measurement data and the gradient measurement data of the image to be detected, and the characteristics of multiple dimensions of the image to be detected are synthesized to judge whether the image to be detected is a normal image, so that the accuracy of the image detection method is effectively improved.
In one embodiment, the implementation of determining the anomaly decision score based on the reconstructed quality metric data and the gradient metric data may be represented by the following equation (16):
score＝D recons + grad (16)
in formula (16), score represents an abnormality determination score, D recons Representing reconstructed quality metric data, D grad Representing gradient metric data.
Referring to fig. 6, a schematic diagram of an image detection method according to an embodiment of the application is shown. The image detection method provided by the embodiment of the application comprises an image processing model, wherein the image processing model comprises a feature extraction network and an image reconstruction network. Inputting the image to be detected into a feature extraction network of an image processing model to obtain image features (also called hidden variables), and carrying out reconstruction processing on the image features by using an image reconstruction network of the image processing model to obtain a reconstructed image; and determining gradient measurement data and reconstruction quality measurement data of the image to be detected according to the reconstruction image, and determining an image detection result for indicating whether the image to be detected is a normal image or an abnormal image according to the gradient measurement data and the reconstruction quality measurement data of the image to be detected.
Based on the embodiment, the application has the following beneficial effects: the image detection method provided by the application can accurately determine whether the image to be detected is a normal image according to the related data (such as the reconstruction quality measurement data, the gradient measurement data and the like of the image to be detected) in the image processing process of the image to be detected by the image processing model, thereby realizing the automation and the intellectualization of the image detection, effectively improving the detection efficiency and the detection accuracy of the image detection and reducing the cost of the image detection; unlike the abnormal detection model obtained by training the abnormal sample image and the normal sample image containing the labeling information, the image detection model is obtained by training the normal sample image only, and the model training efficiency of the image processing model is higher because the product yield of the industrial production line is higher, the acquisition paths of the normal sample image are more, the acquisition speed is higher, the acquisition paths of the abnormal sample image are less, and the acquisition speed is slower; because the image processing model is obtained according to the normal sample image training, the reconstruction quality measurement data and the gradient measurement data obtained by processing the normal image by the image processing model are usually smaller, and the reconstruction quality measurement data and the gradient measurement data obtained by processing the abnormal image by the image processing model are usually larger, so that whether the image to be detected is the normal image can be determined according to the reconstruction quality measurement data and the gradient measurement data obtained by processing the image to be detected by the image processing model; in addition, by combining the two-dimensional data of the reconstructed quality measurement data and the gradient measurement data, whether the image to be detected is a normal image or not can be determined more accurately.
The main body for executing the steps in the above method embodiment may be configured by hardware, software, or a combination of hardware and software.
Fig. 7 is a schematic flow chart of a model training method of an image processing model according to an embodiment of the application. The model training method may be implemented by the image detection apparatus 101 described above, or may be implemented by another model training apparatus. The process of the model training method provided in the embodiment of the application includes but is not limited to:
s701, a first sample image set and a second sample image set are acquired, wherein the first sample image set includes a plurality of normal sample images and interpolation sample images thereof, and the second sample image set includes a plurality of normal reference images and interpolation reference images thereof.
In the embodiment of the application, the normal sample image and the normal reference image can be surface images shot for a sample object, wherein the sample object is a normal industrial product such as a device part, a mechanical finished product and the like, and the normal sample image and the normal reference image can also be normal images. The interpolation sample image is obtained by carrying out interpolation processing on the normal sample image; and (3) interpolating the normal reference image when interpolating the reference image.
In an embodiment, an interpolation method such as a nearest neighbor interpolation method may be used to perform interpolation processing on an image to be detected, so as to obtain an interpolated image. Nearest-neighbor interpolation, which may also be referred to as zero-order interpolation, is to make the gray value of the transformed image pixel equal to the gray value of the input pixel nearest to it, and is applicable to scaling of images.
In an embodiment, the first set of sample images may be the same as the second set of sample images or may be different from the second set of sample images.
S702, respectively carrying out feature extraction processing on a normal sample image and an interpolation sample image in a first sample image set by using an initial image processing model to obtain second feature difference data; and respectively carrying out image reconstruction processing on the normal sample image and the interpolation sample image by using the initial image processing model to obtain second reconstructed difference data.
In an embodiment of the application, the initial image processing model includes a feature extraction network and an image reconstruction network. Performing feature extraction processing on the normal sample images in the first sample image set by using a feature extraction network of the initial image processing model to obtain image features of the normal sample images; performing feature extraction processing on the interpolation sample images in the first sample image set by using a feature extraction network of the initial image processing model to obtain image features of the interpolation sample images; and determining second characteristic difference data according to the image characteristics of the normal sample image and the image characteristics of the interpolation sample image. Assuming that the normal sample image is x2 and the interpolated sample image is x2', the method for determining the second feature difference data may be:
In the formulas (17) (18) (19), z (x 2) represents the image characteristics of the normal sample image, z '(x 2') represents the image characteristics of the interpolated sample image,network parameters of a feature extraction network representing an initial anomaly detection model,/->Representing second difference characteristic data. Performing image reconstruction processing on the image features of the normal sample image by using an image reconstruction network of the initial image processing model to obtain a reconstructed image of the normal sample image; image reconstruction processing is carried out on the image characteristics of the interpolation sample image by utilizing an image reconstruction network of the initial image processing model to obtain the interpolation sample imageReconstructing an image; and determining second reconstruction difference data according to the reconstruction image of the normal sample image and the reconstruction image of the interpolation sample image. Assuming that the normal sample image is x2, the interpolated sample image is x2', the image feature of the normal sample image is z (x 2), and the image feature of the interpolated sample image is z ' (x 2 '), the method for determining the second reconstructed difference data may be implemented by the following formula:
in the formulas (20) (21) (22), R (z (x 2)) represents a reconstructed image of a normal sample image, R ' (z ' (x 2 ')) represents a reconstructed image of an interpolated sample image,network parameters of the image reconstruction network representing the initial image processing model,/- >Representing second reconstructed difference data. The second feature difference data and the second reconstructed difference data are determined as feature difference data and reconstructed difference data for the first stage model training.
In an embodiment, the second feature difference data and the second reconstructed difference data corresponding to each of the plurality of normal sample images may be calculated, an average value of the second feature difference data corresponding to the plurality of normal sample images may be determined as the feature difference data for the first stage model training, and an average value of the second reconstructed difference data corresponding to the plurality of normal sample images may be determined as the reconstructed difference data for the first stage model training.
In an embodiment, the second feature difference data may be determined from the reference image features and the interpolated image features using a calculation of the mean absolute error (Mean Absolute Error, MAE), or may be determined using a calculation of the mean square error (Mean Square Error, MSE).
In an embodiment, the method for determining the second reconstructed difference data may be: and calculating the sum of the difference values corresponding to all the pixel points in the two images, and dividing the sum by the number of the pixel points in one image. The specific implementation formula may be as in formula (1) above.
S703, determining model difference data of the initial image processing model according to the second characteristic difference data and the second reconstruction difference data, and performing model parameter adjustment on the initial image processing model by using the model difference data to obtain an intermediate image processing model.
In the embodiment of the present application, an implementation manner of determining model difference data of an initial image processing model according to the second feature difference data and the second reconstruction difference data may be as shown in the following formula (23):
in the formula (23), the amino acid sequence of the compound,representing second characteristic difference data,/for example>Representing the second reconstructed difference data, L stage-1 And model difference data representing the initial image processing model, β3 being used to balance the proportion of the second feature difference data and the second reconstructed difference data in the model difference data.
In one embodiment, model parameters of the initial image processing model may be updated using a method of model difference data back propagation of the initial image processing model.
In an embodiment, a plurality of second feature difference data and second reconstructed difference data may be determined by using a plurality of normal sample images, and model difference data of a plurality of initial image processing models may be obtained, the initial image processing models may be adjusted multiple times according to the plurality of initial image processing models, and when a difference between the model difference data of the initial image processing models in any two model adjustments is smaller than a set threshold, convergence of the initial image processing models may be determined, and the initial image processing models may be determined as intermediate image processing models.
S704, respectively carrying out feature extraction processing on the normal reference image and the interpolation reference image in the second sample image set by using the intermediate image processing model to obtain third feature difference data; and respectively carrying out image reconstruction processing on the normal reference image and the interpolation reference image by using the intermediate image processing model to obtain third reconstruction difference data.
In an embodiment of the application, the intermediate image processing model comprises a feature extraction network and an image reconstruction network. Performing feature extraction processing on the normal reference images in the second sample image set by using a feature extraction network of the intermediate image processing model to obtain image features of the normal reference images; performing feature extraction processing on the interpolation reference image in the second sample image set by using a feature extraction network of the intermediate image processing model to obtain image features of the interpolation reference image; and determining third characteristic difference data according to the image characteristics of the normal reference image and the image characteristics of the interpolation reference image. Performing image reconstruction processing on the image features of the normal reference image by using an image reconstruction network of the intermediate image processing model to obtain a reconstructed image of the normal reference image; performing image reconstruction processing on the image characteristics of the interpolation reference image by using an image reconstruction network of the intermediate image processing model to obtain a reconstructed image of the interpolation reference image; third reconstruction difference data is determined from the reconstructed image of the normal reference image and the reconstructed image of the interpolated reference image. The formula for determining the third characteristic difference data may be as in the above formulas (17) (18) (19), and the formula for determining the third reconstruction difference data may be as in the above formulas (20) (21) (22). The third feature difference data and the third reconstruction difference data are determined as feature difference data and reconstruction difference data for the second stage model training.
In an embodiment, the third feature difference data and the third reconstruction difference data corresponding to each of the plurality of normal reference images may be calculated, an average value of the third feature difference data corresponding to the plurality of normal reference images may be determined as the feature difference data for the second-stage model training, and an average value of the third reconstruction difference data corresponding to the plurality of normal reference images may be determined as the reconstruction difference data for the second-stage model training.
S705 determining gradient difference data of the intermediate image processing model according to the third feature difference data, the third reconstruction difference data, and model parameters of the intermediate image processing model.
In the embodiment of the application, the model parameters of the intermediate image processing model comprise network parameters of the feature extraction network and network parameters of the image reconstruction network. And determining gradient difference data of the intermediate image processing model according to the third characteristic difference data, the third reconstruction difference data and network parameters of an image reconstruction network of the intermediate image processing model.
In an embodiment, M is the total number of stages of the second stage model training (M is greater than or equal to 1), and the current training stage number J is a positive integer less than or equal to M, and the implementation manner of determining the gradient difference data of the intermediate image processing model may be: when J is 1, determining gradient difference data of an intermediate image processing model according to the third characteristic difference data, the third reconstruction difference data and model parameters of the intermediate image processing model in the J model training stage; and when J is greater than 1, determining gradient difference data of the intermediate image processing model according to the third characteristic difference data, the third reconstruction difference data, model parameters of the intermediate image processing model of the J model training stage and gradient difference data of the intermediate image processing models of the 1 st to J-1 st model training stages. The implementation method for determining gradient difference data of the intermediate image processing model can be represented by the following formulas (24) (25) (26):
In the formulas (24), (25) and (26),representing difference data according to a third feature +.>Represents third reconstruction difference data, β4 is a balance parameter for balancing the ratio of the third characteristic difference data to the third reconstruction difference data, L 1 Representing first phase model training parameters, +.>Network parameters of the image reconstruction network representing the intermediate image processing model, in formula (25), L reg Gradient difference data representing an intermediate image processing model when J is 1, in formula (26), L reg Gradient difference data representing the intermediate image processing model when J is greater than 1.
S706, determining model difference data of the intermediate image processing model according to the third characteristic difference data, the third reconstruction difference data and the gradient difference data of the intermediate image processing model, and performing model parameter adjustment on the intermediate image processing model by using the model difference data of the intermediate image processing model to obtain a target image processing model.
In the embodiment of the present application, the method for determining the model difference data of the intermediate image processing model according to the third feature difference data, the third reconstruction difference data, and the gradient difference data of the intermediate image processing model may be represented by the following formula (27):
In the formula (27), the third feature difference data, the third reconstruction difference data, and the gradient difference data of the intermediate image processing model are represented, and β5 and β6 are balance parameters for balancing the proportions of the third feature difference data, the third reconstruction difference data, and the gradient difference data of the intermediate image processing model in the model difference data.
In an embodiment, if the total number M of stages of the second-stage model training is greater than 1, the model difference data of the intermediate image processing models of the M training stages may be used to perform multiple model parameter adjustments on the intermediate image processing models. Model parameters in the intermediate image processing model may be updated in a back-propagation manner using model difference data of the intermediate image processing model. When the model difference data of the intermediate image processing model trained by any two adjacent second-stage models is smaller than a second preset threshold value, the convergence of the intermediate image processing model can be determined, and the intermediate image processing model is determined to be a target image processing model.
Referring to fig. 8A and 8B, a flow of model training is shown. The image processing model comprises a feature extraction network and an image reconstruction network, and the model training method mainly comprises two training stages: in a first training stage (as shown in fig. 8A), inputting the normal sample image and the interpolation sample image corresponding to the normal sample image into a feature extraction network of an initial image processing model for processing to obtain image features of the normal sample image and the interpolation sample image, and determining second feature difference data according to the image features; inputting the image characteristics of the normal sample image and the interpolation sample image into an image reconstruction network of an initial image processing model for processing to obtain reconstructed images of the normal sample image and the interpolation sample image, determining second reconstruction difference data according to the reconstructed images of the normal sample image and the interpolation sample image, and performing model parameter adjustment on the initial image processing model according to the second characteristic difference data and the second reconstruction characteristic data to obtain an intermediate image processing model; in the second training stage (as shown in fig. 8B), the normal sample image and the interpolated sample image are subjected to the aforementioned processing using the intermediate image processing model, so as to obtain third feature difference data and third reconstruction difference data; determining gradient difference data of the intermediate image processing model according to the third characteristic difference data, the third reconstruction difference data and the network parameters of the image reconstruction network of the intermediate image processing model; determining model difference data of the intermediate image processing model according to the third characteristic difference data, the third reconstruction difference data and the gradient difference data of the intermediate image processing model; and carrying out model parameter adjustment on the intermediate image processing model according to the model difference data to obtain a target image processing model.
Based on the embodiment, the application has the following beneficial effects: the image detection method provided by the application can accurately determine whether the image to be detected is a normal image according to the related data (such as the reconstruction quality measurement data, the gradient measurement data and the like of the image to be detected) in the image processing process of the image to be detected by the image processing model, thereby realizing the automation and the intellectualization of the image detection, effectively improving the detection efficiency and the detection accuracy of the image detection and reducing the cost of the image detection; unlike the abnormal detection model obtained by training the abnormal sample image and the normal sample image containing the labeling information, the image detection model is obtained by training the normal sample image only, and the model training efficiency of the image processing model is higher because the product yield of the industrial production line is higher, the acquisition paths of the normal sample image are more, the acquisition speed is higher, the acquisition paths of the abnormal sample image are less, and the acquisition speed is slower; because the image processing model is obtained according to the normal sample image training, the reconstruction quality measurement data and the gradient measurement data obtained by processing the normal image by the image processing model are usually smaller, and the reconstruction quality measurement data and the gradient measurement data obtained by processing the abnormal image by the image processing model are usually larger, so that whether the image to be detected is the normal image can be determined according to the reconstruction quality measurement data and the gradient measurement data obtained by processing the image to be detected by the image processing model; in addition, by combining the two-dimensional data of the reconstructed quality measurement data and the gradient measurement data, whether the image to be detected is a normal image or not can be determined more accurately.
Referring to fig. 9, fig. 9 is a schematic structural diagram of an image detection device according to an embodiment of the application. The device described in the embodiment of the application corresponds to the intelligent terminal, and comprises:
the processing unit 901 is configured to perform feature extraction processing on an image to be detected by using a feature extraction network of a target image processing model, so as to obtain image features of the image to be detected;
the processing unit 901 is further configured to perform image reconstruction processing according to the image feature by using an image reconstruction network of the target image processing model, so as to obtain a reconstructed image of the image to be detected; the target image processing model is obtained by training a normal sample image and an interpolation sample image of the normal sample image;
a computing unit 902, configured to determine reconstruction quality metric data according to the image to be detected and the reconstructed image;
an obtaining unit 903, configured to obtain an interpolated image of the image to be detected, virtually train the target image processing model using the image to be detected and the interpolated image, and determine gradient metric data of the target image processing model according to related data related to virtual training;
The computing unit 902 is further configured to determine an image detection result of the image to be detected according to the reconstructed quality metric data and the gradient metric data, where the image detection result is used to indicate that the image to be detected is a normal image or an abnormal image.
In an embodiment, the obtaining unit 903 is specifically configured to, when performing virtual training on the target image processing model using the image to be detected and the interpolation image, determine gradient metric data of the target image processing model according to relevant data related to the virtual training: in the I virtual training stage, the image processing model after virtual training in the I-1 virtual training stage is utilized to respectively process the image to be detected and the interpolation image, so as to obtain the reference image characteristics and the reconstruction reference image of the image to be detected, the interpolation image characteristics and the reconstruction interpolation image of the interpolation image; determining model difference data and target model parameter gradients of an I-1 virtual training stage according to the image to be detected, the reference image characteristics, the reconstructed reference image, the interpolation image characteristics, the reconstructed interpolation image and the model parameters of the image processing model after virtual training in the I-1 virtual training stage; performing virtual training again on the image processing model after virtual training in the I-1 virtual training stage by using the model difference data in the I virtual training stage to obtain an image processing model after virtual training in the I virtual training stage; determining gradient measurement data of the target image processing model according to the target model parameter gradients of each virtual training stage; wherein I is any positive integer less than or equal to T, T is the total number of virtual training stages, and T is an integer greater than 1; and when I is 1, virtually training the image processing model in the I-1 virtual training stage to be the target image processing model.
In an embodiment, the obtaining unit 903 is specifically configured to, when determining the model difference data and the target model parameter gradient of the I-th virtual training stage according to the to-be-detected image, the reference image feature, the reconstructed reference image, the interpolation image feature, the reconstructed interpolation image, and the model parameter of the image processing model after the virtual training in the I-1-th virtual training stage: determining first feature difference data according to the reference image features and the interpolation image features; determining first reconstruction difference data according to the difference between the image to be detected and the reconstruction reference image and the difference between the interpolation image and the reconstruction interpolation image; and determining model difference data and a target model parameter gradient of the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the model parameters of the image processing model after virtual training in the I-1 virtual training stage.
In an embodiment, the obtaining unit 903 is specifically configured to, when determining the model difference data and the target model parameter gradient of the I-th virtual training stage according to the first feature difference data, the first reconstructed difference data, and the model parameters of the image processing model after the I-1-th virtual training stage: determining an intermediate model parameter gradient of the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the model parameters of the image processing model after virtual training in the I-1 virtual training stage; when I is 1, determining gradient difference data of the I virtual training stage according to the intermediate model parameter gradient of the I virtual training stage; when I is greater than 1, acquiring recorded intermediate model parameter gradients of each virtual training stage from a 1 st virtual training stage to an I-1 st virtual training stage, and determining gradient difference data of the I st virtual training stage according to the intermediate model parameter gradients of each virtual training stage from the 1 st virtual training stage to the I-1 st virtual training stage and the intermediate model parameter gradients of the I st virtual training stage; determining model difference data of the ith virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the gradient difference data of the ith virtual training stage; and determining the target model parameter gradient of the I virtual training stage according to the model difference data of the I virtual training stage and the model parameters of the image processing model after virtual training in the I-1 virtual training stage.
In an embodiment, the target image processing model is determined according to an intermediate image processing model, and the intermediate image processing model is obtained by training an initial image processing model by using model difference data of the initial image processing model; model difference data of the initial image processing model is determined according to second feature difference data and second reconstruction difference data; the second characteristic difference data is determined according to image characteristics extracted from the normal sample image and the interpolation sample image respectively by using the initial image processing model; the interpolation sample image is obtained by carrying out interpolation processing on the normal sample image; the second reconstruction difference data is determined according to the normal sample image, the interpolation sample image, the sample reconstruction image of each of the normal sample image and the interpolation sample image, and the sample reconstruction image is obtained by performing image reconstruction according to the image characteristics of the corresponding sample image by using the initial image processing model.
In an embodiment, the target image processing model is obtained by training the intermediate image processing model by using model difference data of the intermediate image processing model; the model difference data is determined from third feature difference data, third reconstruction difference data, and gradient difference data of the intermediate image processing model; the third characteristic difference data and the third reconstruction difference data are respectively determined according to the processing results of the intermediate image processing model on the normal reference image and the interpolation reference image thereof; the gradient difference data is determined from the third feature difference data, the third reconstruction difference data, and model parameters of the intermediate image processing model.
In an embodiment, the calculating unit 902 is specifically configured to, when determining the image detection result of the image to be detected according to the reconstructed quality metric data and the gradient metric data: determining an anomaly decision score from the reconstructed quality metric data and the gradient metric data; if the abnormality judgment score is smaller than the score threshold, determining that the image detection result of the image to be detected is a first image detection result, wherein the first image detection result indicates that the image to be detected is a normal image; and if the abnormality judgment score is greater than or equal to the score threshold, determining that the image detection result of the image to be detected is a second image detection result, wherein the second image detection result indicates that the image to be detected is an abnormal image.
It may be understood that the functions of each functional unit of the image detection apparatus according to the embodiment of the present application may be specifically implemented according to the image detection method in the embodiment of the method, and the specific implementation process may refer to the related description in the embodiment of the method, which is not repeated herein.
Based on the embodiment, the application has the following beneficial effects: the image detection method provided by the application can accurately determine whether the image to be detected is a normal image according to the related data (such as the reconstruction quality measurement data, the gradient measurement data and the like of the image to be detected) in the image processing process of the image to be detected by the image processing model, thereby realizing the automation and the intellectualization of the image detection, effectively improving the detection efficiency and the detection accuracy of the image detection and reducing the cost of the image detection; unlike the abnormal detection model obtained by training the abnormal sample image and the normal sample image containing the labeling information, the image detection model is obtained by training the normal sample image only, and the model training efficiency of the image processing model is higher because the product yield of the industrial production line is higher, the acquisition paths of the normal sample image are more, the acquisition speed is higher, the acquisition paths of the abnormal sample image are less, and the acquisition speed is slower; because the image processing model is obtained according to the normal sample image training, the reconstruction quality measurement data and the gradient measurement data obtained by processing the normal image by the image processing model are usually smaller, and the reconstruction quality measurement data and the gradient measurement data obtained by processing the abnormal image by the image processing model are usually larger, so that whether the image to be detected is the normal image can be determined according to the reconstruction quality measurement data and the gradient measurement data obtained by processing the image to be detected by the image processing model; in addition, by combining the two-dimensional data of the reconstructed quality measurement data and the gradient measurement data, whether the image to be detected is a normal image or not can be determined more accurately.
Referring to fig. 10, fig. 10 is a schematic structural diagram of a computer device according to an embodiment of the application. The computer device described in the embodiment of the application comprises: processor 1001, communication interface 1002 and memory 1003. The processor 1001, the communication interface 1002, and the memory 1003 may be connected by a bus or other means, which is exemplified by the present embodiment.
Among them, the processor 1001 (or CPU (Central Processing Unit, central processing unit)) is a computing core and a control core of a computer device, which can parse various instructions in the computer device and process various data of the computer device, for example: the CPU can be used for analyzing a startup and shutdown instruction sent by a user to the computer equipment and controlling the computer equipment to perform startup and shutdown operation; and the following steps: the CPU may transmit various types of interaction data between internal structures of the computer device, and so on. The communication interface 1002 may optionally include a standard wired interface, a wireless interface (e.g., wi-Fi, mobile communication interface, etc.), controlled by the processor 1001 for transceiving data. Memory 1003 (Memory) is a Memory device in a computer device for storing programs and data. It will be appreciated that the memory 1003 herein may include either built-in memory of the computer device or extended memory supported by the computer device. Memory 1003 provides storage space that stores the operating system of the computer device, which may include, but is not limited to: android systems, iOS systems, windows Phone systems, etc., the application is not limited in this regard.
In an embodiment of the present application, the processor 1001 performs the following operations by executing executable program code in the memory 1003:
performing feature extraction processing on an image to be detected by using a feature extraction network of a target image processing model to obtain image features of the image to be detected;
performing image reconstruction processing according to the image characteristics by using an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected; the target image processing model is obtained by training a normal sample image and an interpolation sample image of the normal sample image;
determining reconstruction quality metric data according to the image to be detected and the reconstruction image;
obtaining an interpolation image of the image to be detected, virtually training the target image processing model by using the image to be detected and the interpolation image, and determining gradient measurement data of the target image processing model according to related data related to virtual training;
and determining an image detection result of the image to be detected according to the reconstruction quality measurement data and the gradient measurement data, wherein the image detection result is used for indicating whether the image to be detected is a normal image or an abnormal image.
In an embodiment, when the processor 1001 performs virtual training on the target image processing model using the image to be detected and the interpolation image, and determines gradient metric data of the target image processing model according to relevant data related to virtual training, the processor is specifically configured to: in the I virtual training stage, the image processing model after virtual training in the I-1 virtual training stage is utilized to respectively process the image to be detected and the interpolation image, so as to obtain the reference image characteristics and the reconstruction reference image of the image to be detected, the interpolation image characteristics and the reconstruction interpolation image of the interpolation image; determining model difference data and target model parameter gradients of an I-1 virtual training stage according to the image to be detected, the reference image characteristics, the reconstructed reference image, the interpolation image characteristics, the reconstructed interpolation image and the model parameters of the image processing model after virtual training in the I-1 virtual training stage; performing virtual training again on the image processing model after virtual training in the I-1 virtual training stage by using the model difference data in the I virtual training stage to obtain an image processing model after virtual training in the I virtual training stage; determining gradient measurement data of the target image processing model according to the target model parameter gradients of each virtual training stage; wherein I is any positive integer less than or equal to T, T is the total number of virtual training stages, and T is an integer greater than 1; and when I is 1, virtually training the image processing model in the I-1 virtual training stage to be the target image processing model.
In an embodiment, the processor 1001 is specifically configured to, when determining the model difference data and the target model parameter gradient of the I-th virtual training stage according to the to-be-detected image, the reference image feature, the reconstructed reference image, the interpolation image feature, the reconstructed interpolation image, and the model parameter of the image processing model after the virtual training of the I-1-th virtual training stage: determining first feature difference data according to the reference image features and the interpolation image features; determining first reconstruction difference data according to the difference between the image to be detected and the reconstruction reference image and the difference between the interpolation image and the reconstruction interpolation image; and determining model difference data and a target model parameter gradient of the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the model parameters of the image processing model after virtual training in the I-1 virtual training stage.
In an embodiment, the processor 1001 is specifically configured to, when determining the model difference data and the target model parameter gradient of the I-th virtual training stage according to the first feature difference data, the first reconstructed difference data, and the model parameters of the image processing model after the I-1-th virtual training stage: determining an intermediate model parameter gradient of the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the model parameters of the image processing model after virtual training in the I-1 virtual training stage; when I is 1, determining gradient difference data of the I virtual training stage according to the intermediate model parameter gradient of the I virtual training stage; when I is greater than 1, acquiring recorded intermediate model parameter gradients of each virtual training stage from a 1 st virtual training stage to an I-1 st virtual training stage, and determining gradient difference data of the I st virtual training stage according to the intermediate model parameter gradients of each virtual training stage from the 1 st virtual training stage to the I-1 st virtual training stage and the intermediate model parameter gradients of the I st virtual training stage; determining model difference data of the ith virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the gradient difference data of the ith virtual training stage; and determining the target model parameter gradient of the I virtual training stage according to the model difference data of the I virtual training stage and the model parameters of the image processing model after virtual training in the I-1 virtual training stage.
In an embodiment, the target image processing model is determined according to an intermediate image processing model, and the intermediate image processing model is obtained by training an initial image processing model by using model difference data of the initial image processing model; model difference data of the initial image processing model is determined according to second feature difference data and second reconstruction difference data; the second characteristic difference data is determined according to image characteristics extracted from the normal sample image and the interpolation sample image respectively by using the initial image processing model; the interpolation sample image is obtained by carrying out interpolation processing on the normal sample image; the second reconstruction difference data is determined according to the normal sample image, the interpolation sample image, the sample reconstruction image of each of the normal sample image and the interpolation sample image, and the sample reconstruction image is obtained by performing image reconstruction according to the image characteristics of the corresponding sample image by using the initial image processing model.
In an embodiment, the target image processing model is obtained by training the intermediate image processing model by using model difference data of the intermediate image processing model; the model difference data is determined from third feature difference data, third reconstruction difference data, and gradient difference data of the intermediate image processing model; the third characteristic difference data and the third reconstruction difference data are respectively determined according to the processing results of the intermediate image processing model on the normal reference image and the interpolation reference image thereof; the gradient difference data is determined from the third feature difference data, the third reconstruction difference data, and model parameters of the intermediate image processing model.
In an embodiment, the processor 1001 is specifically configured to, when determining an image detection result of the image to be detected according to the reconstructed quality metric data and the gradient metric data: determining an anomaly decision score from the reconstructed quality metric data and the gradient metric data; if the abnormality judgment score is smaller than the score threshold, determining that the image detection result of the image to be detected is a first image detection result, wherein the first image detection result indicates that the image to be detected is a normal image; and if the abnormality judgment score is greater than or equal to the score threshold, determining that the image detection result of the image to be detected is a second image detection result, wherein the second image detection result indicates that the image to be detected is an abnormal image.
In a specific implementation, the processor 1001, the communication interface 1002, and the memory 1003 described in the embodiment of the present application may execute an implementation manner of a computer device described in an image detection method provided in the embodiment of the present application, or may execute an implementation manner described in an image detection apparatus provided in the embodiment of the present application, which is not described herein again.
Based on the embodiment, the application has the following beneficial effects: the image detection method provided by the application can accurately determine whether the image to be detected is a normal image according to the related data (such as the reconstruction quality measurement data, the gradient measurement data and the like of the image to be detected) in the image processing process of the image to be detected by the image processing model, thereby realizing the automation and the intellectualization of the image detection, effectively improving the detection efficiency and the detection accuracy of the image detection and reducing the cost of the image detection; unlike the abnormal detection model obtained by training the abnormal sample image and the normal sample image containing the labeling information, the image detection model is obtained by training the normal sample image only, and the model training efficiency of the image processing model is higher because the product yield of the industrial production line is higher, the acquisition paths of the normal sample image are more, the acquisition speed is higher, the acquisition paths of the abnormal sample image are less, and the acquisition speed is slower; because the image processing model is obtained according to the normal sample image training, the reconstruction quality measurement data and the gradient measurement data obtained by processing the normal image by the image processing model are usually smaller, and the reconstruction quality measurement data and the gradient measurement data obtained by processing the abnormal image by the image processing model are usually larger, so that whether the image to be detected is the normal image can be determined according to the reconstruction quality measurement data and the gradient measurement data obtained by processing the image to be detected by the image processing model; in addition, by combining the two-dimensional data of the reconstructed quality measurement data and the gradient measurement data, whether the image to be detected is a normal image or not can be determined more accurately.
The embodiment of the application also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program runs on a computer, the computer is caused to execute the image detection method according to the embodiment of the application. The specific implementation manner may refer to the foregoing description, and will not be repeated here.
Embodiments of the present application also provide a computer program product comprising a computer program or computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer readable storage medium, and the processor executes the computer program or computer instructions to cause the computer device to perform the image detection method according to the embodiment of the present application. The specific implementation manner may refer to the foregoing description, and will not be repeated here.
It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the order of action described, as some steps may be performed in other order or simultaneously according to the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.
The above disclosure is illustrative only of some embodiments of the application and is not intended to limit the scope of the application, which is defined by the claims and their equivalents.
Claims (11)
1. An image detection method, the method comprising:
performing feature extraction processing on an image to be detected by using a feature extraction network of a target image processing model to obtain image features of the image to be detected;
performing image reconstruction processing according to the image characteristics by using an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected; the target image processing model is obtained by training a normal sample image and an interpolation sample image of the normal sample image;
Determining reconstruction quality metric data according to the image to be detected and the reconstruction image;
obtaining an interpolation image of the image to be detected, virtually training the target image processing model by using the image to be detected and the interpolation image, and determining gradient measurement data of the target image processing model according to related data related to virtual training;
and determining an image detection result of the image to be detected according to the reconstruction quality measurement data and the gradient measurement data, wherein the image detection result is used for indicating whether the image to be detected is a normal image or an abnormal image.
2. The method of claim 1, wherein virtually training the target image processing model using the image to be detected and the interpolation image and determining gradient metric data of the target image processing model from relevant data involved in virtual training comprises:
in the I virtual training stage, the image processing model after virtual training in the I-1 virtual training stage is utilized to respectively process the image to be detected and the interpolation image, so as to obtain the reference image characteristics and the reconstruction reference image of the image to be detected, the interpolation image characteristics and the reconstruction interpolation image of the interpolation image;
Determining model difference data and target model parameter gradients of an I-1 virtual training stage according to the image to be detected, the reference image characteristics, the reconstructed reference image, the interpolation image characteristics, the reconstructed interpolation image and the model parameters of the image processing model after virtual training in the I-1 virtual training stage; performing virtual training again on the image processing model after virtual training in the I-1 virtual training stage by using the model difference data in the I virtual training stage to obtain an image processing model after virtual training in the I virtual training stage;
determining gradient measurement data of the target image processing model according to the target model parameter gradients of each virtual training stage;
wherein I is any positive integer less than or equal to T, T is the total number of virtual training stages, and T is an integer greater than 1; and when I is 1, virtually training the image processing model in the I-1 virtual training stage to be the target image processing model.
3. The method of claim 2, wherein determining model difference data and target model parameter gradients for the I-th virtual training phase from model parameters of the image to be detected, the reference image features, the reconstructed reference image, the interpolated image features, the reconstructed interpolated image, the image processing model after virtual training for the I-1-th virtual training phase, comprises:
Determining first feature difference data according to the reference image features and the interpolation image features;
determining first reconstruction difference data according to the difference between the image to be detected and the reconstruction reference image and the difference between the interpolation image and the reconstruction interpolation image;
and determining model difference data and a target model parameter gradient of the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the model parameters of the image processing model after virtual training in the I-1 virtual training stage.
4. A method according to claim 3, wherein said determining model difference data and target model parameter gradients for the I-th virtual training stage based on the first feature difference data, the first reconstruction difference data, and model parameters of the image processing model after virtual training for the I-1-th virtual training stage comprises:
determining an intermediate model parameter gradient of the I-1 virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the model parameters of the image processing model after virtual training in the I-1 virtual training stage;
When I is 1, determining gradient difference data of the I virtual training stage according to the intermediate model parameter gradient of the I virtual training stage; when I is greater than 1, acquiring recorded intermediate model parameter gradients of each virtual training stage from a 1 st virtual training stage to an I-1 st virtual training stage, and determining gradient difference data of the I st virtual training stage according to the intermediate model parameter gradients of each virtual training stage from the 1 st virtual training stage to the I-1 st virtual training stage and the intermediate model parameter gradients of the I st virtual training stage;
determining model difference data of the ith virtual training stage according to the first characteristic difference data, the first reconstruction difference data and the gradient difference data of the ith virtual training stage;
and determining the target model parameter gradient of the I virtual training stage according to the model difference data of the I virtual training stage and the model parameters of the image processing model after virtual training in the I-1 virtual training stage.
5. The method of claim 1, wherein the target image processing model is determined from an intermediate image processing model, the intermediate image processing model being obtained by training an initial image processing model using model difference data of the initial image processing model; model difference data of the initial image processing model is determined according to second feature difference data and second reconstruction difference data; the second characteristic difference data is determined according to image characteristics extracted from the normal sample image and the interpolation sample image respectively by using the initial image processing model; the interpolation sample image is obtained by carrying out interpolation processing on the normal sample image; the second reconstruction difference data is determined according to the normal sample image, the interpolation sample image, the sample reconstruction image of each of the normal sample image and the interpolation sample image, and the sample reconstruction image is obtained by performing image reconstruction according to the image characteristics of the corresponding sample image by using the initial image processing model.
6. The method according to claim 5, wherein the target image processing model is obtained by training the intermediate image processing model with model difference data of the intermediate image processing model; the model difference data is determined from third feature difference data, third reconstruction difference data, and gradient difference data of the intermediate image processing model; the third characteristic difference data and the third reconstruction difference data are respectively determined according to the processing results of the intermediate image processing model on the normal reference image and the interpolation reference image thereof; the gradient difference data is determined from the third feature difference data, the third reconstruction difference data, and model parameters of the intermediate image processing model.
7. The method according to claim 1, wherein said determining an image detection result of the image to be detected from the reconstruction quality metric data and the gradient metric data comprises:
determining an anomaly decision score from the reconstructed quality metric data and the gradient metric data;
if the abnormality judgment score is smaller than the score threshold, determining that the image detection result of the image to be detected is a first image detection result, wherein the first image detection result indicates that the image to be detected is a normal image;
And if the abnormality judgment score is greater than or equal to the score threshold, determining that the image detection result of the image to be detected is a second image detection result, wherein the second image detection result indicates that the image to be detected is an abnormal image.
8. An image processing apparatus, the apparatus comprising:
the processing unit is used for carrying out feature extraction processing on the image to be detected by utilizing a feature extraction network of the target image processing model to obtain image features of the image to be detected;
the processing unit is further used for performing image reconstruction processing according to the image characteristics by utilizing an image reconstruction network of the target image processing model to obtain a reconstructed image of the image to be detected; the target image processing model is obtained by training a normal sample image and an interpolation sample image of the normal sample image;
a computing unit for determining reconstruction quality metric data from the image to be detected and the reconstructed image;
the acquisition unit is used for acquiring an interpolation image of the image to be detected, virtually training the target image processing model by utilizing the image to be detected and the interpolation image, and determining gradient measurement data of the target image processing model according to related data related to virtual training;
The computing unit is further configured to determine an image detection result of the image to be detected according to the reconstruction quality metric data and the gradient metric data, where the image detection result is used to indicate that the image to be detected is a normal image or an abnormal image.
9. A computer device, comprising: the image detection method according to any one of claims 1-7, comprising a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected to each other, wherein the memory stores executable program code, and the processor is configured to invoke the executable program code to implement the image detection method according to any one of claims 1-7.
10. A computer readable storage medium having stored therein computer instructions which, when run on a computer, cause the computer to implement the image detection method of any of claims 1-7.
11. A computer program product, characterized in that the computer program product comprises a computer program or computer instructions which, when executed by a processor, implements the image detection method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310431195.4A CN116977269A (en) | 2023-04-13 | 2023-04-13 | Image detection method, apparatus, device, readable storage medium, and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310431195.4A CN116977269A (en) | 2023-04-13 | 2023-04-13 | Image detection method, apparatus, device, readable storage medium, and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116977269A true CN116977269A (en) | 2023-10-31 |
Family
ID=88472048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310431195.4A Pending CN116977269A (en) | 2023-04-13 | 2023-04-13 | Image detection method, apparatus, device, readable storage medium, and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977269A (en) |
-
2023
- 2023-04-13 CN CN202310431195.4A patent/CN116977269A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11900628B2 (en) | Stereo matching method and apparatus, image processing apparatus, and training method therefor | |
| US11488308B2 (en) | Three-dimensional object detection method and system based on weighted channel features of a point cloud | |
| US11232286B2 (en) | Method and apparatus for generating face rotation image | |
| CN111179307A (en) | Visual target tracking method for full-volume integral and regression twin network structure | |
| CN111696196B (en) | Three-dimensional face model reconstruction method and device | |
| CN104008538B (en) | Based on single image super-resolution method | |
| KR20200031163A (en) | Neural network structure creation method and device, electronic device, storage medium | |
| CN106504276A (en) | The combinations matches cost algorithms of non local Stereo Matching Algorithm and parallax joint filling algorithm | |
| CN112712546A (en) | Target tracking method based on twin neural network | |
| CN111369550A (en) | Image registration and defect detection method, model, training method, device and equipment | |
| CN111127435A (en) | No-reference image quality evaluation method based on double-current convolutional neural network | |
| CN111914878A (en) | Feature point tracking training and tracking method and device, electronic equipment and storage medium | |
| CN112329726B (en) | Face recognition method and device | |
| CN111160225A (en) | Human body analysis method and device based on deep learning | |
| CN109685805A (en) | A kind of image partition method and device | |
| Lee | One-shot scale and angle estimation for fast visual object tracking | |
| CN114897884A (en) | No-reference screen content image quality evaluation method based on multi-scale edge feature fusion | |
| JP2016148588A (en) | Depth estimation model generation device and depth estimation device | |
| CN116977269A (en) | Image detection method, apparatus, device, readable storage medium, and program product | |
| CN114820755A (en) | Depth map estimation method and system | |
| CN113962928A (en) | Defect detection method, device and medium based on multi-scale feature distillation | |
| US11393069B2 (en) | Image processing apparatus, image processing method, and computer readable recording medium | |
| CN113128292A (en) | Image identification method, storage medium and terminal equipment | |
| CN113362462B (en) | Binocular stereoscopic vision parallax filtering method and device based on self-supervision learning | |
| TWI753382B (en) | Method for estimating three-dimensional human skeleton for a human body in an image, three-dimensional human skeleton estimator, and training method for the estimator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |