WO2022100366A1 - 工业缺陷识别方法、系统、计算设备及存储介质 - Google Patents

工业缺陷识别方法、系统、计算设备及存储介质 Download PDF

Info

Publication number
WO2022100366A1
WO2022100366A1 PCT/CN2021/124254 CN2021124254W WO2022100366A1 WO 2022100366 A1 WO2022100366 A1 WO 2022100366A1 CN 2021124254 W CN2021124254 W CN 2021124254W WO 2022100366 A1 WO2022100366 A1 WO 2022100366A1
Authority
WO
WIPO (PCT)
Prior art keywords
defect
industrial
target area
area
defects
Prior art date
Application number
PCT/CN2021/124254
Other languages
English (en)
French (fr)
Inventor
王秀东
金鑫
涂丹丹
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Priority to EP21890893.7A priority Critical patent/EP4231229A4/en
Publication of WO2022100366A1 publication Critical patent/WO2022100366A1/zh
Priority to US18/316,757 priority patent/US20230281784A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30136—Metal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30152—Solder
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the embodiments of the present application relate to the technical field of artificial intelligence (AI), and in particular, to an industrial defect identification method, system, and storage medium.
  • AI artificial intelligence
  • industrial products may have industrial defects, such as weld defects, during the production, manufacture and maintenance.
  • industrial defects such as weld defects
  • detection based on images of industrial products is one of the important means of industrial defect detection.
  • neural network models in the field of image recognition in recent years, there have been some methods for recognizing industrial defects in images through neural network models.
  • the current methods of industrial defect identification through neural network models are often only suitable for the identification of large-scale industrial defects, and the ability to identify small-scale industrial defects is limited, and the identification results often only include the identification of industrial defect types, and the identification dimension single.
  • the embodiments of the present application provide an industrial defect identification method, system, computing device and storage medium, which can improve the detection probability of small-sized industrial defects, improve the accuracy of defect location and size estimation, and enrich the identification dimensions of industrial defects.
  • the technical solution is as follows:
  • a method for identifying industrial defects comprising: acquiring an image to be identified, the image to be identified is an image reflecting an industrial product; and extracting at least one image from the image to be identified according to a target area detection model a target area, where the at least one target area is an area that may have industrial defects; according to the at least one target area and the defect detection model, obtain a defect rough selection area of the target area containing industrial defects in the at least one target area, and the types of industrial defects contained in the defect roughing area; determining the size and position of the industrial defects contained in the defect roughing area.
  • a target area is first extracted from the image to be recognized, and then a rough selection area of defects containing industrial defects is obtained from the target area.
  • a rough selection area of defects containing industrial defects is obtained from the target area.
  • the size ratio of industrial defects relative to the background is increased, and the detection probability of small-sized industrial defects can be improved.
  • further positioning and size estimation of industrial defects in the defect roughing area can effectively improve the accuracy of defect positioning and size estimation.
  • the embodiments of the present application not only identify the defect type, but also can locate and estimate the size of the industrial defect, which enriches the identification dimension of the industrial defect.
  • obtaining, according to the at least one target area and the defect detection model, a defect rough selection area of the target area containing industrial defects in the at least one target area, and the industrial defects contained in the defect rough selection area The implementation process of the type is as follows: taking the at least one target area as the input of the defect detection model, performing inference through the defect detection model, and obtaining the defect roughness of the target area containing industrial defects in the at least one target area. The position of the selected area in the target area, and the defect type of the industrial defect contained in the defect roughing area, wherein the defect detection model is an artificial intelligence AI model trained by using defect training samples; The position of the area in the target area, and the defect roughing area is extracted from the target area.
  • At least one target area is an area where industrial defects may occur.
  • the target area may or may not contain industrial defects.
  • the target area can be further detected by the defect detection model to determine whether the target area contains industrial defects. If it contains industrial defects, the defect detection model can output the industrial defects. The defect roughing area of the defect and the type of the industrial defect.
  • the target area extracted by the target area detection model is a large area where certain industrial defects may occur, and the defect rough selection area obtained by the defect detection model is based on the target area.
  • an area containing industrial defects is extracted that is smaller than the target area.
  • the proportion of industrial defects in the target area will be greater than that in the image to be identified, and the proportion in the rough selection area of defects will be greater than that in the target area. That is, by extracting the target area from the image to be recognized, and then extracting the defect roughing area from the target area, the proportion of industrial defects relative to the background is gradually increased, so that the detection probability of small-sized industrial defects can be improved.
  • the implementation process of determining the size and position of the industrial defects contained in the rough defect area may be: according to the gray level distribution in the rough defect area, determine the industrial defects contained in the rough defect area.
  • the position and size of the defect in the defect rough selection area according to the position of the defect rough selection area in the target area, the position of the target area in the to-be-recognized image, and the defect rough selection area
  • the position of the included industrial defects in the defect rough selection area is determined, and the position of the industrial defects included in the defect rough selection area in the to-be-identified image is determined.
  • the method further includes: generating a detection report, the detection report including the defects of the industrial defects detected in the to-be-identified image type, size, and location; provide the user with the test report.
  • the industrial defect is a weld seam defect
  • the target area is an area where the weld seam is located.
  • the method further includes: acquiring a plurality of industrial training samples, each industrial training sample corresponds to target area labeling information, and the target area labeling information includes the position of the multi-degree-of-freedom quadrilateral labeled in the corresponding industrial training sample. information and labeling category information used to indicate that the corresponding multi-degree-of-freedom quadrilateral is the target area; train the target area detection network through the multiple industrial training samples and the target area labeling information of each industrial training sample to obtain the target area Detection model.
  • the multi-degree-of-freedom quadrilateral is used to mark the target area that may contain industrial defects in the industrial training sample, and then the target area detection network is trained by the industrial training sample to obtain the target area detection model, so that , the target area detection model can more accurately detect various target areas that may contain defects distributed in different directions and shapes in the image.
  • the implementation process of training the target area detection network by using the multiple industrial training samples and the target area labeling information of each industrial training sample may be: according to the target area labeling information of the first industrial training sample, Determine a first circumscribed rectangle and a second circumscribed rectangle, where the first circumscribed rectangle refers to the minimum rotated circumscribed rectangle of the multi-degree-of-freedom quadrilateral marked in the first industrial training sample, and the second circumscribed rectangle refers to the A horizontal circumscribed rectangle of a circumscribed rectangle; the predicted value of the multi-degree-of-freedom quadrilateral in the first industrial training sample, the predicted value of the first circumscribed rectangle, the predicted value of the second circumscribed rectangle, and the Prediction value and prediction category information; prediction based on the target area labeling information of the first industrial training sample, the first circumscribed rectangle, the second circumscribed rectangle, and the multi-degree-of-freedom quadrilateral in the first industrial training sample value, the predicted value of the first circumscribed rectangle, the predicted value of the second circumscribed rectangle, and the predicted category information
  • the method further includes: acquiring a plurality of defect training samples; extracting a plurality of target areas from the plurality of defect training samples by using the target area detection model; The defect labeling information of the target area of the defect, the defect labeling information is used to indicate the defect type and location of the industrial defect in the corresponding target area; the defect detection network is detected by the target area containing the industrial defect and the defect labeling information of the corresponding target area. Perform training to obtain the defect detection model.
  • a target area detection model is obtained by first training, and then, the industrial defects included in the defect training samples are roughly located by the target area detection model to extract the target area, and then the defects marked in the target area are passed through.
  • the fine location and defect type of the defect detection network are trained to obtain a defect detection model.
  • the target area detection model uses a multi-degree-of-freedom quadrilateral as the detection frame, and the angle between each side of the multi-degree-of-freedom quadrilateral and each side of the to-be-recognized image is any value from 0 degrees to 180 degrees. .
  • a neural network detection model with a multi-degree-of-freedom quadrilateral as a detection frame is used, which can be suitable for different types of target areas, and the target area detection model has good versatility.
  • an industrial defect identification system in a second aspect, is provided, and the industrial defect identification system has the function of implementing the industrial defect identification method in the first aspect.
  • the industrial defect identification system includes at least one module, and the at least one module is used to implement the industrial defect identification system method provided in the first aspect above.
  • a computing device in a third aspect, includes a processor and a memory in its structure, and the memory is used to store a program that supports the computing device to execute the industrial defect identification method provided in the first aspect, and stores Data involved in implementing the industrial defect identification method provided by the first aspect.
  • the processor is configured to execute programs stored in the memory.
  • the computing device may also include a communication bus for establishing a connection between the processor and the memory.
  • a computer-readable storage medium is provided, and instructions are stored in the computer-readable storage medium, when the computer-readable storage medium is run on a computer, the computer executes the industrial defect identification method described in the first aspect.
  • a computer program product comprising instructions which, when run on a computer, cause the computer to execute the industrial defect identification method described in the first aspect above.
  • a target area is first extracted from an image to be recognized, and then a rough selection area of defects containing industrial defects is obtained from the target area.
  • a target area is first extracted from an image to be recognized, and then a rough selection area of defects containing industrial defects is obtained from the target area.
  • the industrial defects in the defect roughing area are further positioned and size estimated, which can effectively improve the accuracy of defect positioning and size estimation.
  • FIG. 1 is a schematic structural diagram of an industrial defect identification system provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of the deployment of an industrial defect identification system provided by an embodiment of the present application.
  • FIG. 3 is an application schematic diagram of an industrial defect identification system provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of the deployment of another industrial defect identification system provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
  • FIG. 6 is a flowchart of a training target area detection model and a defect detection model provided by an embodiment of the present application
  • FIG. 7 is a schematic diagram of a multi-degree-of-freedom quadrilateral marked in an industrial training sample provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a multi-degree-of-freedom quadrilateral marked in an industrial training sample, a generated first circumscribed rectangle and a second circumscribed rectangle provided by an embodiment of the present application;
  • FIG. 10 is a schematic structural diagram of a computer system provided by an embodiment of the present application.
  • industrial defects may occur in the process of production, manufacture and maintenance of industrial products.
  • welding is often used, and there may be weld defects in the welds left by the welding.
  • defects on a certain part of an automobile, defects such as cracks and scratches may exist on the part due to production or maintenance process problems.
  • Such defects in industrial products often affect the normal use of industrial products. Based on this, it is very necessary to detect industrial defects in industrial products.
  • detection based on images of industrial products is one of the important means of industrial defect detection. Among them, one of the main methods is to rely on humans to identify industrial defects from images of industrial products.
  • this method is not only detrimental to the health of workers, but also difficult to ensure the quality of identification.
  • some industrial defect recognition methods based on neural network models have emerged to replace manual recognition.
  • this method is often only suitable for the identification of large-sized defects, and the identification effect for small-sized defects is poor;
  • this method can often only identify whether there are industrial defects in the image, but cannot locate and estimate the size of industrial defects, and the identification dimension is relatively single.
  • the embodiments of the present application provide a method for identifying industrial defects in industrial product images based on an AI model, so as to enrich the identification dimensions of industrial defects and improve the identification effect of small-sized industrial defects.
  • the industrial defect identification method provided by the embodiment of the present application is executed by the industrial defect identification system 01 .
  • the industrial defect identification system 01 can be realized by a software system, can also be realized by a hardware device, and can also be realized by a combination of a software system and a hardware device.
  • the industrial defect identification system 01 when the industrial defect identification system 01 is a software system, referring to FIG. 1 , the industrial defect identification system 01 can logically divide the identification device 10 and the training device 11 .
  • the identification device 10 and the training device 11 can be logically divided into multiple modules, and each module has different functions.
  • the identification device 10 includes a target area detection module 101 , a coarse defect detection module 102 and a fine defect identification module 103 . It should be noted that, the embodiment of the present application only exemplarily divides the structure and functional modules of the identification device 10, but does not make any limitation on the specific division.
  • the target area detection module 101 is configured to acquire the to-be-recognized image, and extract at least one target area from the to-be-recognized image according to the target area detection model.
  • the image to be recognized is an image containing industrial products.
  • At least one target area refers to an area where industrial defects may exist.
  • the target area may be an area that contains a complete weld seam in the image to be identified.
  • the target area may be a polygonal area.
  • the target area is a multi-degree-of-freedom quadrilateral.
  • the so-called multi-degree-of-freedom quadrilateral means that the angle or parallel relationship between each side of the quadrilateral and the side of the image to be recognized is free, not limited to the vertical or parallel relationship, and the angle or parallel relationship between the four sides of the quadrilateral Relationship freedom, nor is it limited to vertical or parallel relationships. That is to say, the angle between each side of the multi-degree-of-freedom quadrilateral and each side of the image to be recognized may be any value from 0 degrees to 180 degrees.
  • the adjacent sides of the polygonal quadrilateral region may or may not be perpendicular, and the opposite sides may or may not be parallel.
  • the target area detection model is an AI model trained using industrial training samples.
  • the target area detection model is used to determine the area of interest, that is, the target area, where certain industrial defects may occur from the image to be recognized, and output the position of the target area in the image to be recognized.
  • the rough defect detection module 102 is configured to communicate with the target area detection module 101 and the defect fine identification module 103, receive at least one target area extracted by the target area detection module 101, and obtain at least one target area according to the at least one target area and the defect detection model.
  • the defect detection model is an AI model trained using defect training samples. And, the defect detection model is used to further detect the rough selection area of defects containing industrial defects from the target area, and output the position information of the rough selection area of defects in the target area.
  • the rough selection area of defects may refer to extracting a smaller area containing weld defects from the area containing the complete weld.
  • the defect detection model is also used to output the types of industrial defects contained in the defect roughing area.
  • at least one target area extracted by the aforementioned target area detection module 101 is an area where industrial defects may occur, that is to say, some target areas may not contain industrial defects. In this case , after the defect detection model detects these target areas, it will output a recognition result indicating that there is no defect in the target area.
  • the fine defect identification module 103 is configured to communicate with the rough defect detection module 102, receive the rough defect region extracted by the rough defect detection module 102, and determine the size and location of industrial defects contained in the rough defect region.
  • the fine defect identification module 103 is further configured to receive the position information of the rough defect selection area in the target area sent by the rough defect detection module 102, and communicate with the target area detection module 101 to receive the target area detection The location information of the target area in the to-be-recognized image sent by the module 101. Afterwards, the fine defect identification module 103 determines the size of the industrial defect included in the rough defect area and the position of the industrial defect in the rough defect area according to the gray level distribution in the rough defect area. According to the position information of the defect rough selection area in the target area, the position information of the target area including the defect rough selection area in the image to be identified, and the position of the industrial defect in the defect rough selection area, obtain the industrial defect to be identified. position in the image.
  • the identification device 10 may further include a report generation module 104, and the report generation module 104 is configured to communicate with the defect fine identification module 103, and receive the data sent by the defect fine identification module 103. and then generate a detection report according to the recognition result, and provide the detection report to the user.
  • the identification result includes the type, size and position of the industrial defect detected in the image to be identified.
  • the inspection report includes the above identification result.
  • the inspection report may also include other contents, for example, It may also include information such as whether the industrial product in the image to be recognized is qualified according to the recognition result, which is not limited in this embodiment of the present application.
  • the report generation module 104 is also configured to communicate with the rough defect detection module 102, receive the identification result output by the rough defect detection module 102 for indicating no defects in some target areas, and then generate a detection report according to the identification result. .
  • the target area detection model and the defect detection model are both trained AI models.
  • the target area detection model can be obtained by training the target area detection network by the training device 11, and the defect detection model can be obtained by training the defect detection network.
  • the training device 11 includes a target area detection model generation module 111 , a target area extraction module 112 , a defect detection model generation module 113 and a storage module 114 .
  • the target area detection model generation module 111 is configured to acquire multiple industrial training samples from the storage module 114, and train the target area detection network through the acquired multiple industrial training samples, thereby obtaining the target area detection model.
  • the plurality of industrial training samples are images each including a target area, and the target area refers to an area where industrial defects may occur.
  • each industrial training sample corresponds to target area labeling information
  • the target area identification information includes the position information of the multi-degree-of-freedom quadrilateral marked in the corresponding industrial training sample and the labeling category used to indicate that the corresponding multi-degree-of-freedom quadrilateral is the target area information.
  • the target area extraction and defect labeling module 112 is configured to acquire a plurality of defect training samples from the storage module 114, and extract a target area from each defect training sample through a trained target area detection model. Among them, multiple defect training samples are training samples containing target regions. After extracting the target area from each defect training sample, the target area extraction and defect labeling module 112 may also obtain defect labeling information obtained after the user labels the industrial defects in the target area, wherein the defect labeling information is used for Indicates the defect type and location of the industrial defect within the corresponding target area.
  • the defect detection model generation module 113 is configured to communicate with the target area extraction and defect labeling module 112, receive multiple target areas and the defect labeling information of each target area sent by the target area extraction and defect labeling module 112, and then pass the multiple target areas.
  • the defect detection network is trained by each target area and the defect annotation information of each target area, so as to obtain a defect detection model.
  • the training device 11 and the identification device 10 are two independent devices.
  • the target area detection model and the defect detection model obtained after being trained by the training device 11 will be deployed into the identification device 10 .
  • the training device 11 may also be part of the identification device 10 .
  • the industrial defect identification system 01 described above can be flexibly deployed.
  • the industrial defect identification system 01 is deployed in a cloud environment.
  • Cloud environment is an entity that uses basic resources to provide cloud services to users under the cloud computing model.
  • the cloud environment includes cloud data centers and cloud service platforms.
  • a cloud data center includes a large number of basic resources (including computing resources, storage resources, and network resources) owned by a cloud service provider, and the computing resources included in the cloud data center may be a large number of computing devices (eg, servers).
  • the industrial defect identification system 01 may be a software system deployed on a server or a virtual machine in a cloud data center, the software system may be used to identify industrial defects, and the software system may be distributed on multiple servers, Or distributed on multiple virtual machines, or distributed on virtual machines and servers. For example, as shown in FIG. 2, the industrial defect identification system 01 is deployed in a cloud environment.
  • the client 02 can send the to-be-recognized image to the industrial defect recognition system 01, and after receiving the to-be-recognized image, the industrial defect recognition system 01 can extract at least one target area from the to-be-recognized image according to the target area detection model.
  • a target area and defect detection model which obtains at least one target area containing industrial defects, a defect roughing area of the target area, and the types of industrial defects contained in the defect roughing area. After that, the size and location of industrial defects contained in the defect roughing area are determined. After that, the industrial defect identification system 01 feeds back the type, position and size of the industrial defect obtained by the above detection to the client 02 .
  • FIG. 3 is a schematic diagram of an application of the industrial defect identification system 01 in this application.
  • the industrial defect identification system 01 can be deployed in a cloud data center by a cloud service provider, and the cloud service provider will The functions provided by the industrial defect identification system 01 are abstracted into a cloud service, and the cloud service platform is for users to consult and purchase this cloud service. After purchasing the cloud service, the user can use the industrial defect identification service provided by the industrial defect identification system 01 in the cloud data center.
  • the industrial defect identification system 01 can also be deployed by the tenant in the computing resources of the cloud data center rented by the tenant. The tenant purchases the computing resource cloud service provided by the cloud service provider through the cloud service platform, and runs the industrial defect in the purchased computing resources.
  • the identification system 01 enables the industrial defect identification system 01 to identify industrial defects.
  • the industrial defect identification system 01 may also be a software system running on an edge computing device in an edge environment or composed of one or more edge computing devices in an edge environment.
  • the so-called edge environment refers to a set of devices including one or more edge computing devices in a certain application scenario, where the one or more edge computing devices can be computing devices in a data center or multiple data centers. computing equipment.
  • the industrial defect identification system 01 is a software system
  • the industrial defect identification system 01 may be deployed on multiple edge computing devices in a distributed manner, or may be centrally deployed on one edge computing device.
  • the industrial defect identification system 01 is distributed in edge computing devices 03 included in a data center of an enterprise, and a client 04 in the enterprise can send images to be identified to the enterprise.
  • the industrial defect identification system 01 after receiving the to-be-identified image, the industrial defect identification system 01 can identify the industrial defect contained in the to-be-identified image through the method provided in the embodiment of the present application, and feed back the identification result to the client 04 .
  • FIG. 5 is a schematic structural diagram of a computing device 500 provided by an embodiment of the present application.
  • the computing device 500 includes a processor 501 , a communication bus 502 , a memory 503 and at least one communication interface 504 .
  • the processor 501 may be a general-purpose central processing unit (Central Processing Unit, CPU), an application-specific integrated circuit (application-specific integrated circuit, ASIC), a graphics processing unit (graphics processing unit, GPU) or any combination thereof.
  • the processor 501 may include one or more chips, and the processor 501 may include an AI accelerator, such as a neural network processor (neural processing unit, NPU).
  • NPU neural network processor
  • Communication bus 502 may include pathways for communicating information between various components of computing device 500 (eg, processor 501, memory 503, communication interface 504).
  • the memory 503 may be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM)) or other types of static storage devices that can store information and instructions.
  • ROM read-only memory
  • RAM random access memory
  • dynamic storage device it can also be Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, CD-ROM storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by Any other medium accessed by the computer, but not limited to this.
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • CD-ROM Compact Disc Read-Only Memory
  • CD-ROM storage including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.
  • the memory 503 can exist independently and is connected to the processor 501 through the communication bus 502 .
  • the memory 503 may also be integrated with the processor 501 .
  • the memory 503 may store computer instructions, and when the computer instructions stored in the memory 503 are executed by the processor 501, the industrial defect identification method of the present application may be implemented.
  • the memory 503 may also store the data required by the processor in the process of executing the above method and the generated intermediate data and/or result data.
  • the communication interface 504 using any device such as a transceiver, is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN) and the like.
  • RAN Radio Access Network
  • WLAN Wireless Local Area Networks
  • the processor 501 may include one or more CPUs.
  • a computer device may include multiple processors.
  • processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor.
  • a processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).
  • the industrial defect identification system adopts a target area detection model and a defect detection model to identify industrial defects in the image to be recognized, wherein the target area detection model and the defect detection model are both trained AI models. Based on this, the following first introduces the training process of the target area detection model and the defect detection model.
  • FIG. 6 is a flowchart of a training target area detection model and a defect detection model provided by an embodiment of the present application.
  • the method can be applied to the training device in the aforementioned industrial defect identification system, see FIG. 6 , and the method includes the following steps:
  • Step 601 Acquire a plurality of industrial training samples, each of which corresponds to target region labeling information.
  • the training device collects a plurality of industrial training samples.
  • the industrial training sample may be an image collected by other collection devices and sent to the training device, for example, may be collected by a camera in an industrial environment and sent to the training device.
  • the industrial training samples can also be collected by the acquisition device and then sent to other devices for storage, and then sent to the training device by other devices.
  • the industrial training samples may also be images sent by the client to the industrial defect identification system for identification. That is, the training device can collect the images identified by the identification device in the industrial defect identification system, and use these images as industrial training samples.
  • the plurality of industrial training samples are images containing target regions.
  • the target area refers to an area where industrial defects may exist
  • the industrial defects refer to defects generated during the production, manufacture or maintenance of industrial products.
  • the industrial defects are weld defects, because the weld defects only appear in The area where the weld is located, so the target area is the weld area at this time.
  • the industrial defect only occurs on a certain part of the automobile, and then the target area is the area where the part is located, and details are not described herein again in this embodiment of the present application.
  • the training device After acquiring a plurality of industrial training samples, the training device sequentially displays each industrial training sample to the user, so that the user can mark the target area for each industrial training sample. Afterwards, the training device obtains the target area labeling information generated when the user labels each industrial training sample.
  • the user uses a quadrilateral labeling frame to label the target area in the industrial training sample, thereby obtaining a quadrilateral including the target area.
  • the quadrilateral frame is a multi-DOF quadrilateral frame.
  • the labeled quadrilateral is also a multi-DOF quadrilateral.
  • the so-called quadrilateral with multiple degrees of freedom means that the angle between the four sides of the quadrilateral and the four sides of the industrial training sample is any value between 0 and 180 degrees.
  • each side of the polygonal freeform and each side of the industrial training sample may be perpendicular or non-perpendicular, and may also be parallel or non-parallel.
  • the adjacent sides of the multi-DOF quadrilateral may be perpendicular or non-perpendicular, and the opposite sides may be parallel or non-parallel.
  • FIG. 7 is a multi-degree-of-freedom quadrilateral marked in an industrial training sample shown in an embodiment of the present application.
  • the edge Q 1 Q 2 of the multi-degree-of-freedom quadrilateral is neither perpendicular nor parallel to the edges of the industrial training sample, and the other three edges are neither perpendicular nor parallel to the edges of the industrial training sample parallel.
  • the sides Q 1 Q 2 and Q 3 Q 4 of the multi-degree-of-freedom quadrilateral are parallel, the sides Q 1 Q 4 and Q 2 Q 3 are not parallel, and each phase in the multi-degree-of-freedom quadrilateral is parallel.
  • the two adjacent sides are also not perpendicular to each other.
  • the user can use the quadrilateral frame to enclose the weld to obtain a quadrilateral surrounding the weld.
  • the welding seam is usually a long and irregular quadrilateral, and the welding seam is not horizontal or vertical in the industrial training samples, therefore, in the embodiment of the present application, the user can use multiple degrees of freedom In this way, the multi-degree-of-freedom quadrilateral annotation frame can better adapt to the shape of the weld and surround the weld more closely, so as to reduce the multi-freedom obtained by marking as much as possible.
  • the training device After the user marks the quadrilateral in the industrial training sample through the quadrilateral labeling frame, the training device automatically obtains the position information of the quadrilateral in the industrial training sample.
  • the position information may include coordinate information of the four vertices of the quadrilateral in the industrial training sample.
  • the training device may generate labeling category information of the quadrilateral, where the labeling category information is used to indicate that the quadrilateral is the target area.
  • the labeling category information may be the first identifier, for example, the first identifier is "1".
  • the training device may use the labeling category information and the position information of the quadrangle as the labeling information of the target area corresponding to the industrial training sample.
  • Step 602 Train a target area detection network by using a plurality of industrial training samples and target area labeling information of each industrial training sample to obtain a target area detection model.
  • the training device After acquiring the multiple industrial training samples and the target area labeling information of each industrial training sample, the training device trains the target area detection network through the multiple industrial training samples and the target area labeling information of each industrial training sample, thereby Get the target area detection model.
  • the process is described by taking one industrial training sample among the multiple industrial training samples, and the marked quadrilateral area is a multi-degree-of-freedom quadrilateral as an example.
  • the industrial training sample is hereinafter referred to as the first Industrial training samples.
  • the training device determines a first circumscribed rectangle and a second circumscribed rectangle according to the target area labeling information of the first industrial training sample, where the first circumscribed rectangle refers to the minimum rotation of the multi-degree-of-freedom quadrilateral marked in the first industrial training sample.
  • the second bounding rectangle refers to the horizontal bounding rectangle of the first bounding rectangle; the predicted value of the multi-degree-of-freedom quadrilateral in the first industrial training sample, the predicted value of the first bounding rectangle, and the second bounding rectangle are determined through the target area detection network.
  • the predicted value and predicted category information of the rectangle according to the target area labeling information of the first industrial training sample, the first circumscribed rectangle, the second circumscribed rectangle, the predicted value of the multi-degree-of-freedom quadrilateral in the first industrial training sample, the first circumscribed rectangle
  • the predicted value of , the predicted value of the second circumscribed rectangle and the predicted category information calculate the loss function value; update the parameters of the target area detection network according to the loss function value.
  • the training device first determines the minimum rotation circumscribed rectangle of the multi-degree-of-freedom quadrilateral marked in the first industrial training sample, that is, The first circumscribed rectangle is then determined, and the horizontal circumscribed rectangle of the first circumscribed rectangle, that is, the second circumscribed rectangle is determined.
  • the first circumscribed rectangle refers to a circumscribed rectangle whose four sides are not parallel to the four sides of the first industrial training sample and has the smallest area.
  • the second circumscribed rectangle refers to a circumscribed rectangle with the smallest area among the circumscribed rectangles of the first circumscribed rectangle, and two adjacent sides are respectively parallel to the two adjacent sides of the first industrial training sample.
  • the multi-DOF quadrilateral marked in the first industrial training sample is Q 1 Q 2 Q 3 Q 4
  • the first circumscribed rectangle is R 1 R 2 R 3 R 4
  • the second circumscribed rectangle is K 1K 2K 3K 4 .
  • the training device detects the first industrial training sample through the target area detection network, thereby obtaining the predicted value of the labeled multi-degree-of-freedom quadrilateral, the predicted value of the first circumscribed rectangle, the predicted value of the second circumscribed rectangle, and the predicted category information.
  • the predicted category information is used to indicate whether the predicted value of the labeled multi-degree-of-freedom quadrilateral is a foreground area or a background area. For example, if the predicted value of the labeled multi-degree-of-freedom quadrilateral is the foreground area, the predicted category information is the first identification; if the predicted value of the labeled multi-degree-of-freedom quadrilateral is the background area, the predicted category information is the second identification.
  • the training device After obtaining the predicted value of the labeled multi-degree-of-freedom quadrilateral, the predicted value of the first circumscribed rectangle, the predicted value of the second circumscribed rectangle and the predicted category information through the target area detection network, the training device can use the above predicted value, labeled
  • the parameter information of the quadrilateral, the predicted category information and the labeled category information are used to calculate the loss function value, and then the parameters of the target area detection network are updated according to the calculated loss function value.
  • the training device when calculating the loss function value, can separately parameterize the marked data and the predicted data, and then calculate the loss function value through the parameterized parameter value.
  • the quadrilateral can be characterized by the vertex coordinates of the three vertices Q 1 , Q 2 , Q 3 and the length of the side Q 3 Q 4 of the quadrilateral, that is, the parameter information of the quadrilateral with multiple degrees of freedom is in, is the coordinate of vertex Q 1 , is the coordinate of vertex Q 2 , is the coordinate of vertex Q 3 , and w Q is the length of edge Q 3 Q 4 .
  • the predicted value of the multi-degree-of-freedom quadrilateral is q 1 q 2 q 3 q 4
  • the parameter information of the predicted value of the multi-degree-of-freedom quadrilateral is:
  • the coordinates of the adjacent two vertices of the rectangle and the adjacent sides of the sides formed by the two vertices can be obtained. to characterize the length.
  • the first circumscribed rectangle R 1 R 2 R 3 R 4 can be characterized by the coordinates of the vertices R 1 and R 2 and the length of the side R 2 R 3 .
  • the parameters of the first circumscribed rectangle are Information is in, is the coordinate of vertex R 1 , is the coordinate of the vertex R 2 , and h R is the length of R 2 R 3 , that is, the height of the first circumscribed rectangle.
  • the first circumscribed rectangle may also be characterized by the coordinates of the vertices R 2 and R 3 and the length of the side R 1 R 2 , which will not be repeated in this embodiment of the present application.
  • the predicted value of the first circumscribed rectangle is r 1 r 2 r 3 r 4
  • the parameter information of the predicted value of the first circumscribed rectangle may be
  • the second circumscribed rectangle can be characterized by the coordinates of a vertex and the width and height of the circumscribed rectangle.
  • the second circumscribed rectangle K 1 K 2 K 3 K 4 can be characterized by the coordinates of the vertex K 1 , the lengths of the side K 1 K 2 and the side K 2 K 3 . That is, the parameter information of the second circumscribed rectangle may be in, is the coordinate of the vertex K 1 , w K is the width of the second circumscribed rectangle, that is, the length of the side K 1 K 2 , h K is the height of the second circumscribed rectangle, that is, the length of the side K 2 K 3 .
  • the predicted value of the second circumscribed rectangle is k 1 k 2 k 3 k 4
  • the parameter information of the predicted value of the second circumscribed rectangle may be
  • the training device can use the horizontal rectangular target area generated by the target area detection network in the process of predicting the above-mentioned each quadrilateral to parameterize it.
  • the training device can use the following formulas (1)-(3)
  • the multi-degree-of-freedom quadrilateral marked in the first industrial training sample and the parameter information of the predicted value of the multi-degree-of-freedom quadrilateral are parameterized.
  • the parameter information of the predicted value of the circumscribed rectangle is parameterized, and the parameter information of the second circumscribed rectangle and the predicted value of the second circumscribed rectangle is parameterized by formulas (1)-(4), thereby obtaining the parameter groups of each quadrilateral.
  • the parameter group of the first circumscribed rectangle can be expressed as The parameter group of the predicted value of the first circumscribed rectangle can be expressed as
  • formulas (1)-(4) are used to parameterize the second circumscribed rectangle and the parameter information of the predicted value of the second circumscribed rectangle, the value of i is 1.
  • the value of N is K,k.
  • the parameter group of the second circumscribed rectangle can be expressed as The parameter set of the predicted value of the second circumscribed rectangle can be expressed as
  • the training device After parameterizing the marked data and the predicted data respectively, the training device calculates the loss function value by the following formula (5) based on each parameter group after the above parameterization.
  • L cls ( ) is the classification loss function
  • l p is the prediction category information
  • l g is the labeling category information
  • ⁇ 1 , ⁇ 2 , and ⁇ 3 are preset balance factors, which are used to adjust the corresponding items when calculating the loss function value.
  • relative contribution in the process is the regression function of the parameters of the labeled multi-degree-of-freedom quadrilateral, is the regression function of the parameters of the first circumscribed rectangle, is the regression function of the parameters of the second circumscribed rectangle.
  • the training device can iteratively train the target area detection network under the guidance of the calculated loss function, thereby obtaining the target area detection model.
  • each quadrilateral obtained by labeling and each quadrilateral obtained by prediction are only an example provided by this embodiment of the present application, and in some possible scenarios, other parameter information can also be used to represent For each quadrilateral, for example, the corresponding quadrilateral is represented by the coordinates of the four vertices of each quadrilateral, which is not limited in this embodiment of the present application.
  • Step 603 Acquire multiple defective training samples.
  • the training device collects multiple defective training samples.
  • the multiple defect training samples are all images containing target regions. It should be noted that, in some possible cases, the multiple defect training samples do not have intersections with the above-mentioned multiple industrial training samples used to train the target area detection model, and in other possible cases, the multiple defects
  • the training samples may include industrial training samples that contain industrial defects in the above-mentioned plurality of industrial training samples.
  • Step 604 Extract a plurality of target regions from a plurality of defect training samples through a target region detection model.
  • the training device After training the target area detection model and acquiring the defect training samples, the training device extracts the target area from each defect training sample through the target area detection model.
  • the target area detection model since the target area detection model is obtained by marking quadrilaterals that may have industrial defects in the industrial training samples, the target area detection model uses a multi-degree-of-freedom quadrilateral as the detection frame.
  • a model capable of detecting quadrilateral-shaped target regions that may have industrial defects from each defect training sample when the quadrilateral used in training is a multi-degree-of-freedom quadrilateral, what is detected from each defect training sample will also be a multi-degree-of-freedom quadrilateral.
  • the training samples input to the defect detection network are usually required to be rectangular.
  • the training device after detecting the target area, can further determine the minimum value of the target area.
  • the bounding rectangle is filled with the background color between the minimum bounding rectangle and the target area, and then the minimum bounding rectangle is extracted from the defect training samples, and the extracted image is used as the input of the subsequent defect detection network.
  • a multi-degree-of-freedom quadrilateral can be used to mark the weld seam, so that the target area detection model can Target regions containing welds and shaped as multi-degree-of-freedom quadrilaterals are detected from the defect training samples.
  • the training device can determine the minimum circumscribed rectangle of the target area, fill the background color between the minimum circumscribed rectangle and the target area, and then extract the minimum circumscribed rectangle including the target area from the defect training sample.
  • Step 605 Acquire defect labeling information of target areas including industrial defects in the multiple target areas.
  • the training device can display multiple extracted target areas to the user, and the user can select the multiple target areas from the multiple target areas.
  • the target area containing industrial defects is selected from the area, and the industrial defects in the target area containing industrial defects are marked. After that, the training device obtains the defect annotation information generated when the user makes an annotation in each target area.
  • the user can use a horizontal rectangular callout box to mark the industrial defects contained in the target area, and input the defect type of the industrial defect contained in the horizontal rectangular callout box in the defect type option.
  • the training device acquires the position of the horizontal rectangular frame marked by the user in the target area, and uses the position as the position of the industrial defect contained in the target area.
  • the training device acquires the defect type of the industrial defect input by the user. The obtained location and defect type of industrial defects are used as defect annotation information in the target area.
  • the training device may obtain the position of the center point of the horizontal rectangular frame.
  • Step 606 Train the defect detection network by using the target area containing industrial defects and the defect annotation information of the corresponding target area to obtain a defect detection model.
  • the training device After acquiring the target area containing industrial defects and the defect annotation information of the corresponding target area, the training device trains the defect detection network through each target area containing industrial defects and the corresponding defect annotation information, thereby obtaining a defect detection model .
  • the training device inputs the smallest circumscribed rectangle containing the target area to the defect detection network, and the defect detection network identifies the industrial defects contained in the target area, thereby obtaining the target.
  • the predicted location and predicted defect type of the industrial defects contained within the zone Calculate the position loss function value according to the predicted position of the industrial defect and the labeled defect position contained in the defect label information of the target area, and calculate the type loss according to the predicted defect type of the industrial defect and the labeled defect type included in the defect label information. function value.
  • the parameters of the defect detection network are updated according to the position loss function value and the type loss function value, so as to obtain the defect detection model.
  • a multi-degree-of-freedom quadrilateral is used to mark the target area that may contain industrial defects in the industrial training sample, and then the target area detection network is trained through the industrial training sample to obtain the target area detection model, so that , the target area detection model can more accurately detect various target areas that may contain defects distributed in different directions and shapes in the image.
  • the target area detection model is obtained by first training, and then, the industrial defects contained in the defect training samples are roughly located by the target area detection model, so as to extract the target area, and then mark the target area through the target area. The fine location of the defect and the defect type are trained to the defect detection network to obtain the defect detection model.
  • the defect detection model when the target area extracted from the image to be recognized is detected by the defect detection model, more accurate positioning of industrial defects in the target area can be obtained, and specific defect types of industrial defects can be output, which not only improves the The accuracy of positioning is improved, and the dimension of defect identification is enriched.
  • the above embodiment mainly introduces the training process of the target area detection model and the defect detection model. Next, the process of using the target area detection model and the defect detection model to recognize the image to be recognized will be introduced.
  • FIG. 9 is a flowchart of a method for identifying an industrial defect provided by an embodiment of the present application. The method can be applied to the identification device in the industrial defect identification system described above, referring to FIG. 9 , the method includes the following steps:
  • Step 901 Acquire an image to be recognized, where the image to be recognized is an image reflecting an industrial product.
  • the identification device acquires the to-be-identified image, and the to-be-identified image is stored in the identification device, or the to-be-identified image is sent to the identification device by other devices.
  • the other device may be a device that collects the to-be-recognized image, or may be a device that stores the to-be-recognized image, which is not limited in this embodiment of the present application.
  • Step 902 Extract at least one target area from the image to be identified according to the target area detection model, where the at least one target area is an area that may have industrial defects.
  • the recognition device After acquiring the to-be-recognized image, the recognition device uses the to-be-recognized image as an input to the target area detection model, and extracts at least one target area from the to-be-recognized image through the target area detection model, wherein the at least one target area is possible
  • An area where an industrial defect exists, that is, the at least one target area refers to an area where a certain industrial defect may occur.
  • the recognition device determines the target area in the image to be recognized by using the target area detection model, and outputs the position of the target area in the image to be recognized. Afterwards, the identification device obtains a minimum circumscribed rectangle containing the target area according to the position of the target area in the image to be recognized.
  • the position of the target area in the to-be-recognized image may be represented by the coordinates of each vertex of the target area, or may be represented by other parameters according to the shape of the target area, which is not limited in this embodiment of the present application.
  • the target area detection model is obtained by training the quadrilaterals marked in the industrial training samples, the target area detection model can determine the quadrilateral target area from the image to be recognized.
  • the quadrilateral used in training is a multi-degree-of-freedom quadrilateral
  • the target area determined from the image to be recognized will also be a multi-degree-of-freedom quadrilateral.
  • the identification device After detecting the target area, the identification device The minimum circumscribed rectangle of the target area can be further determined, the background color is filled between the minimum circumscribed rectangle and the target area, and then the image to be recognized is cropped according to the minimum circumscribed rectangle of the target area, thereby obtaining an image containing the target area.
  • the weld seam is marked with a multi-degree-of-freedom quadrilateral frame in the industrial training sample.
  • a multi-degree-of-freedom target area including a weld can be identified from the to-be-recognized image.
  • the recognition device can determine the minimum circumscribed rectangle of the target area, fill the gap between the target area and the minimum circumscribed rectangle with the background color, and extract the minimum circumscribed rectangle from the image to be recognized, so as to obtain an image containing the target area. image.
  • Step 903 According to the at least one target area and the defect detection model, obtain the defect rough selection area of the target area containing industrial defects in the at least one target area, and the type of industrial defects contained in the defect rough selection area.
  • the identification device After extracting at least one target area through the target area detection model, the identification device uses the at least one target area as the input of the defect detection model, and performs inference through the defect detection model to obtain the defect of the target area containing industrial defects in the at least one target area The position of the rough selection area in the target area, and the defect type of the industrial defect contained in the defect rough selection area; then, according to the position of the defect rough selection area in the target area, the defect rough selection area is extracted from the target area.
  • the identification device inputs an image including the target area into the defect detection model.
  • the defect detection model processes the target area. It should be noted that since the target area is an area where industrial defects may occur, the target area may or may not contain industrial defects. Based on this, if the target area does not contain industrial defects, the defect detection model will output a recognition result indicating that the target area is free of defects. If the target area includes industrial defects, the defect detection model will be able to determine the defect roughing area in the target area and identify industrial defects in the defect roughing area, and then the defect detection model will output the defect roughing area. The location in the target area and the defect type of the industrial defect in the identified defect roughing area.
  • the identification device After obtaining the position of the rough selection area of the defect in the target area, the identification device cuts the target area according to the position, so as to obtain the rough selection area of the defect.
  • the position of the defect rough selection area in the target area can also be characterized by the vertex coordinates of the defect rough selection area, or by other parameters according to the shape of the defect rough selection area.
  • the target area extracted from the defect training samples is an industrial defect marked by a horizontal rectangular frame during the training of the defect detection model, in this step, the defects determined in the target area are coarse.
  • the selection area is also a horizontal rectangular area.
  • a target area containing the weld seam is extracted from the image to be recognized by the target area detection model, and the target area is input into the defect detection model, and the defect detection model processes the target area. , so that a horizontal rectangular area containing weld defects is determined in the target area. According to the determined position of the horizontal rectangular area, the horizontal rectangular area is extracted from the target area, so as to obtain a rough selection area of defects.
  • the defect roughing area is an area that contains weld defects and is smaller than the target area. In this way, the proportion of weld defects in the defect roughing area will be larger than that in the target area.
  • the target area extracted by the target area detection model is a large area where some industrial defects may occur, and the rough selection area of defects obtained by the defect detection model is further narrowed on the basis of the target area.
  • the proportion of industrial defects in the target area will be greater than that in the image to be identified, and the proportion in the rough selection area of defects will be greater than that in the target area. That is, by extracting the target area from the image to be recognized, and then extracting the defect rough selection area from the target area, the proportion of industrial defects relative to the background is gradually increased.
  • further positioning and size estimation of industrial defects in the defect roughing area can improve the accuracy of defect positioning and size estimation.
  • the above method is used to continuously enlarge the proportion of the industrial defects in the image, which is more conducive to the subsequent positioning and size estimation of the industrial defects.
  • the recognition device can detect the at least one target area with reference to the above method, so as to obtain an industrial defect from the at least one target area.
  • the defect roughing area of the target area and the type of industrial defects contained in the defect roughing area are examples of the defect roughing area.
  • Step 904 Determine the size and position of the industrial defects contained in the defect roughing area.
  • the identification device may further determine the industrial defects contained in the defect rough selection area by dividing the defect rough selection area. size and location.
  • the identification device firstly grayscales the defect rough selection area.
  • the identification device determines the size of the industrial defect contained in the defect roughing area and the position of the industrial defect in the defect roughing area according to the grayscale distribution in the grayscaled defect roughing area.
  • the position of the area in the target area, the position of the target area in the image to be identified, and the position of the industrial defects contained in the defect roughing area in the defect roughing area determine the industrial defects contained in the defect roughing area in the to-be-identified image. Location.
  • the identification device may use the adaptive threshold binarization method in the region, according to The grayscale distribution in the rough selection area of defects determines the location and size of industrial defects in this area.
  • the identification device determines the grayscale threshold value according to the grayscale value distribution in the grayscaled defect rough selection area. After that, the identification device compares the grayscale value of each pixel in the grayscale defect rough selection area with the grayscale threshold, and if the grayscale value of the corresponding pixel is less than the grayscale threshold, the corresponding pixel The gray value of the pixel is set to the first value, and if the gray value of the corresponding pixel is not less than the gray threshold, the gray value of the pixel is set to the second value.
  • the first value and the second value are not equal. For example, the first value is 0 and the second value is 255.
  • the identification device binarizes the grayscale value of each pixel in the grayscaled defect rough selection area. After that, the identification device determines the position and size of the industrial defect according to the pixel points with the gray value of the first value in the gray-scaled defect rough selection area.
  • the identification device After determining the position of the industrial defect in the defect roughing area, the identification device can pass the target area and the defect according to the position of the industrial defect in the defect roughing area and the position of the defect roughing area in the corresponding target area. Coordinate transformation between rough selection areas to obtain the position of the industrial defect in the target area. After that, the identification device obtains the industrial defect in the to-be-recognized image through the coordinate transformation between the target area and the to-be-recognized image according to the position of the industrial defect in the target area and the position of the target area in the to-be-recognized image. Location.
  • the identification device may also directly use the determined position of the industrial defect in the defect roughing area as the finally obtained position of the industrial defect.
  • the identification device can generate a detection report and feed back the detection report to the client.
  • the inspection report includes the defect type, size and location of the industrial defect detected in the image to be identified.
  • the detection report may further include other content, for example, may also include information about whether the industrial product in the image to be identified is qualified according to the above identification result, which is not limited in this embodiment of the present application.
  • the identification device generates a detection report that contains the identification result for indicating no defects in the target area, And feedback the detection report to the client.
  • the beneficial effects brought by the industrial defect identification method include at least the following four aspects:
  • a neural network detection model with a multi-degree-of-freedom quadrilateral as the detection frame is used, which can be suitable for different types of target areas, and the target area detection model has good versatility.
  • a target area is first extracted from the image to be recognized, and then a rough selection area of defects containing industrial defects is obtained from the target area.
  • a target area is first extracted from the image to be recognized, and then a rough selection area of defects containing industrial defects is obtained from the target area.
  • the embodiment of the present application further provides an identification device 10 as shown in FIG. 1 , and the modules and functions included in the identification device 10 are as described above, and details are not repeated here.
  • the target area detection module 101 in the identification device 10 is configured to perform steps 901 and 902 in the foregoing embodiments.
  • the rough defect detection module 102 is used to perform step 903 in the foregoing embodiment.
  • the defect fine identification module 103 is used to perform step 904 in the foregoing embodiment.
  • the model optimization apparatus 10 may further include a report generation module 104, and the report generation module 104 may be configured to generate a detection report including the identification result, and provide the detection report to the user.
  • the report generation module 104 may be configured to generate a detection report including the identification result, and provide the detection report to the user.
  • the present application also provides a training device 11 as shown in FIG. 1 .
  • the modules and functions included in the training device 11 are as described above, which will not be repeated here.
  • the target region detection model generation module 111 in the training device 11 may be used to perform steps 601 and 602 in the foregoing embodiments.
  • the target region extraction and defect annotation module 112 may be used to perform steps 603 to 605 in the foregoing embodiments.
  • the defect detection model generation module 113 may be used to perform step 606 in the previous embodiment.
  • This embodiment of the present application further provides a computing device 500 as shown in FIG. 5 .
  • the processor 501 in the computing device 500 reads a set of computer instructions stored in the memory 503 to perform the aforementioned industrial defect identification method.
  • each module in the identification device 10 provided in this embodiment of the present application can be distributed and deployed on multiple computers in the same environment or in different environments, the present application also provides a computing device as shown in FIG. 10 (also It may be referred to as a computer system), the computer system includes a plurality of computers 1000, and the structure of each computer 1000 is the same as or similar to the structure of the computing device 500 in FIG. 5, and will not be repeated here.
  • FIG. 10 also It may be referred to as a computer system
  • the computer system includes a plurality of computers 1000, and the structure of each computer 1000 is the same as or similar to the structure of the computing device 500 in FIG. 5, and will not be repeated here.
  • a communication path is established between each of the above computers 1000 through a communication network.
  • Each computer 1000 runs any one or more of the aforementioned target area detection module 101 , coarse defect detection module 102 , fine defect identification module 103 , and report generation module 104 .
  • Any computer 1000 may be a computer (eg, a server) in a cloud data center, or an edge computer, or a terminal computing device.
  • a computer program product for realizing industrial defect identification includes one or more computer instructions for industrial defect identification.
  • the computer program instructions are loaded and executed on a computer, the computer program instructions are generated in whole or in part according to the embodiments of the present application as described in FIG. 6 and FIG. 9 . process or function.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media.
  • the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, digital versatile disc (DVD)), or semiconductor media (eg, solid state disk (SSD)) )Wait.
  • references herein to "at least one” refers to one or more, and “plurality” refers to two or more.
  • “/” means or means, for example, A/B can mean A or B;
  • "and/or” in this document is only an association relationship to describe associated objects, indicating that There can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, A and B exist at the same time, and B exists alone.
  • words such as “first” and “second” are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that the words “first”, “second” and the like do not limit the quantity and execution order, and the words “first”, “second” and the like are not necessarily different.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

本申请实施例公开了一种工业缺陷识别方法、系统、计算设备及存储介质，属于AI领域。在本申请实施例中，首先从待识别图像中提取目标区域，然后再从目标区域中获得包含有工业缺陷的缺陷粗选区域。通过两次区域提取，提高了工业缺陷相对背景的尺寸占比，能够提高小尺寸的工业缺陷的检出概率。在检测出包含有工业缺陷的缺陷粗选区域之后，对缺陷粗选区域中的工业缺陷进行进一步定位和尺寸估计，能够有效提高缺陷定位和尺寸估计的精度。另外，本申请实施例中不仅对缺陷类型进行了识别，还能够对工业缺陷进行定位和尺寸估计，丰富了工业缺陷的识别维度。

Description

工业缺陷识别方法、系统、计算设备及存储介质
本申请要求于2020年11月13日提交中国知识产权局、申请号为202011268302.9、申请名称为“工业缺陷识别方法、系统、计算设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及人工智能(artificial intelligence，AI)技术领域，特别涉及一种工业缺陷识别方法、系统及存储介质。
背景技术
在工业领域中，工业产品在生产、制造和维护的过程中可能会出现工业缺陷，例如，焊缝缺陷。为了保证工业产品能够正常使用，对工业产品进行工业缺陷检测是必要的。当前，基于工业产品的图像进行检测是工业缺陷检测的重要手段之一。另外，随着近些年来神经网络模型在图像识别领域的广泛应用，已有一些通过神经网络模型对图像中的工业缺陷进行识别的方法出现。然而，目前通过神经网络模型进行工业缺陷识别的方法往往仅适合对大尺寸工业缺陷的识别，对小尺寸工业缺陷的识别能力有限，并且，识别结果往往仅包含对工业缺陷类型的识别，识别维度单一。
发明内容
本申请实施例提供了一种工业缺陷识别方法、系统、计算设备及存储介质，能够提高小尺寸的工业缺陷的检出概率，提高缺陷定位和尺寸估计的精度，丰富工业缺陷的识别维度。所述技术方案如下：
第一方面，提供了一种工业缺陷识别方法，所述方法包括：获取待识别图像，所述待识别图像为反映工业产品的图像；根据目标区域检测模型从所述待识别图像中提取至少一个目标区域，所述至少一个目标区域为可能存在工业缺陷的区域；根据所述至少一个目标区域和缺陷检测模型，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域，和所述缺陷粗选区域包含的工业缺陷的类型；确定所述缺陷粗选区域包含的工业缺陷的尺寸和位置。
在本申请实施例中，首先从待识别图像中提取目标区域，然后再从目标区域中获得包含有工业缺陷的缺陷粗选区域。通过两次区域提取，提高了工业缺陷相对背景的尺寸占比，能够提高小尺寸的工业缺陷的检出概率。在检测出包含有工业缺陷的缺陷粗选区域之后，对缺陷粗选区域中的工业缺陷进行进一步定位和尺寸估计，能够有效提高缺陷定位和尺寸估计的精度。另外，本申请实施例中不仅对缺陷类型进行了识别，还能够对工业缺陷进行定位和尺寸估计，丰富了工业缺陷的识别维度。
可选地，所述根据所述至少一个目标区域和缺陷检测模型，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域，和所述缺陷粗选区域包含的工业缺陷的类型的实现过程为：将所述至少一个目标区域作为所述缺陷检测模型的输入，通过所述缺陷检测模型进行推理，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域在目标区域中的位置，和所述缺陷粗选区域包含的工业缺陷的缺陷类型，其中，所述缺陷检测模型为利用缺陷训练样本训练完成的人工智能AI模型；根据所述缺陷粗选区域在所述目标区域中的位置，从所述目标区域中提取所述缺陷粗选区域。
其中，至少一个目标区域为可能发生工业缺陷的区域。这样，对于任意一个目标区域，该目标区域中可能包含有工业缺陷，也可能不包括。基于此，在将目标区域提取出来之后，可以进一步通过缺陷检测模型对目标区域检测，以确定该目标区域是否包含有工业缺陷，如果包含有工业缺陷，则该缺陷检测模型能够输出包含有该工业缺陷的缺陷粗选区域和该工业缺陷的类型。
在本申请实施例中，通过目标区域检测模型提取到的目标区域是一个某种工业缺陷可能发生的范围较大的区域，而通过缺陷检测模型得到缺陷粗选区域则是在目标区域的基础上进一步缩小范围后提取出的一个相较该目标区域更小的包含有工业缺陷的区域。这样，工业缺陷在该目标区域中的占比将大于在待识别图像中的占比，在缺陷粗选区域中的占比又将大于在目标区域中的占比。也即，通过从待识别图像中提取目标区域，然后再从目标区域中提取缺陷粗选区域，逐渐的提高了工业缺陷相对背景的占比，这样，能够提高小尺寸的工业缺陷的检出概率。
可选地，所述确定所述缺陷粗选区域包含的工业缺陷的尺寸和位置的实现过程可以为：根据所述缺陷粗选区域内的灰度分布，确定所述缺陷粗选区域包含的工业缺陷在所述缺陷粗选区域内的位置和尺寸；根据所述缺陷粗选区域在所述目标区域中的位置、所述目标区域在所述待识别图像中的位置以及所述缺陷粗选区域包含的工业缺陷在所述缺陷粗选区域内的位置，确定所述缺陷粗选区域包含的工业缺陷在所述待识别图像中的位置。
在本申请实施例中，在检测出包含有工业缺陷的缺陷粗选区域之后，根据缺陷粗选区域内的灰度分布，对缺陷粗选区域中的工业缺陷进行进一步定位和尺寸估计，能够有效提高缺陷定位和尺寸估计的精度。
可选地，在对缺陷粗选区域内的工业缺陷进行定位和尺寸估计之后，所述方法还包括：生成检测报告，所述检测报告包括在所述待识别图像中检测到的工业缺陷的缺陷类型、尺寸和位置；向用户提供所述检测报告。
可选地，所述工业缺陷为焊缝缺陷，所述目标区域为焊缝所在的区域。
可选地，所述方法还包括：获取多个工业训练样本，每个工业训练样本均对应有目标区域标注信息，所述目标区域标注信息包括相应工业训练样本中标注的多自由度四边形的位置信息和用于指示相应多自由度四边形为目标区域的标注类别信息；通过所述多个工业训练样本和每个工业训练样本的目标区域标注信息对目标区域检测网络进行训练，得到所述目标区域检测模型。
在本申请实施例中，采用多自由度四边形对工业训练样本中可能包含有工业缺陷的目标区域进行标注，进而通过该工业训练样本对目标区域检测网络进行训练，以获得目标区 域检测模型，这样，该目标区域检测模型能够更为准确的检测图像中以不同方向、形态分布的可能包含缺陷的各种目标区域。
可选地，所述通过所述多个工业训练样本和每个工业训练样本的目标区域标注信息对目标区域检测网络进行训练的实现过程可以为：根据第一工业训练样本的目标区域标注信息，确定第一外接矩形和第二外接矩形，所述第一外接矩形是指所述第一工业训练样本中标注的多自由度四边形的最小旋转外接矩形，所述第二外接矩形是指所述第一外接矩形的水平外接矩形；通过所述目标区域检测网络确定所述第一工业训练样本中的多自由度四边形的预测值、所述第一外接矩形的预测值、所述第二外接矩形的预测值和预测类别信息；根据所述第一工业训练样本的目标区域标注信息、所述第一外接矩形、所述第二外接矩形、所述第一工业训练样本中的多自由度四边形的预测值、所述第一外接矩形的预测值、所述第二外接矩形的预测值和所述预测类别信息，计算损失函数值；根据所述损失函数值对所述目标区域检测网络进行参数更新。
可选地，所述方法还包括：获取多个缺陷训练样本；通过所述目标区域检测模型从所述多个缺陷训练样本中提取多个目标区域；获取所述多个目标区域中包含有工业缺陷的目标区域的缺陷标注信息，所述缺陷标注信息用于指示相应目标区域内的工业缺陷的缺陷类型和位置；通过包含有工业缺陷的目标区域和相应目标区域的缺陷标注信息对缺陷检测网络进行训练，得到所述缺陷检测模型。
在本申请实施例中，先训练得到目标区域检测模型，之后，通过该目标区域检测模型对缺陷训练样本中包含的工业缺陷进行粗定位，以提取目标区域，进而通过该目标区域中标注的缺陷的精细位置和缺陷类型训练缺陷检测网络，以得到缺陷检测模型。这样，后续在通过该缺陷检测模型检测从待识别图像中提取出来的目标区域时，能够得到工业缺陷在该目标区域中更为准确的定位，并能够输出具体的工业缺陷的缺陷类型，不仅提高了定位的准确性，而且丰富了缺陷识别得到维度。
可选地，所述目标区域检测模型使用多自由度四边形作为检测框，所述多自由度四边形的各个边与所述待识别图像的各个边的角度为0度至180度中的任一数值。
在本申请实施例中，在目标区域提取环节，使用以多自由度四边形为检测框的神经网络检测模型，可以适合不同类型的目标区域，所述目标区域检测模型具有良好的通用性。
第二方面，提供了一种工业缺陷识别系统，所述工业缺陷识别系统具有实现上述第一方面中工业缺陷识别方法的功能。所述工业缺陷识别系统包括至少一个模块，该至少一个模块用于实现上述第一方面所提供的工业缺陷识别系统方法。
第三方面，提供了一种计算设备，所述计算设备的结构中包括处理器和存储器，所述存储器用于存储支持计算设备执行上述第一方面所提供的工业缺陷识别方法的程序，以及存储用于实现上述第一方面所提供的工业缺陷识别方法所涉及的数据。所述处理器被配置为用于执行所述存储器中存储的程序。所述计算设备还可以包括通信总线，该通信总线用于该处理器与存储器之间建立连接。
第四方面，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面所述的工业缺陷识别方法。
第五方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面所述的工业缺陷识别方法。
上述第二方面、第三方面、第四方面和第五方面所获得的技术效果与第一方面中对应的技术手段获得的技术效果近似，在这里不再赘述。
本申请实施例提供的技术方案带来的有益效果至少包括以下三个方面：
第一，在本申请实施例中，首先从待识别图像中提取目标区域，然后再从目标区域中获得包含有工业缺陷的缺陷粗选区域。通过两次区域提取，提高了工业缺陷相对背景的尺寸占比，能够提高小尺寸的工业缺陷的检出概率。
第二，在检测出包含有工业缺陷的缺陷粗选区域之后，对缺陷粗选区域中的工业缺陷进行进一步定位和尺寸估计，能够有效提高缺陷定位和尺寸估计的精度。
第三，本申请实施例中不仅对缺陷类型进行了识别，还能够对工业缺陷进行定位和尺寸估计，丰富了工业缺陷的识别维度。
附图说明
图1是本申请实施例提供的一种工业缺陷识别系统的结构示意图；
图2是本申请实施例提供的一种工业缺陷识别系统的部署示意图；
图3是本申请实施例提供的一种工业缺陷识别系统的应用示意图；
图4是本申请实施例提供的另一种工业缺陷识别系统的部署示意图；
图5是本申请实施例提供的一种计算设备的结构示意图；
图6是本申请实施例提供的一种训练目标区域检测模型和缺陷检测模型的流程图；
图7是本申请实施例提供的一种在工业训练样本中标注得到的多自由度四边形的示意图；
图8是本申请实施例提供的一种在工业训练样本中标注的多自由度四边形、生成的第一外接矩形和第二外接矩形的示意图；
图9是本申请实施例提供的一种工业缺陷识别方法流程图；
图10是本申请实施例提供的一种计算机系统的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
在对本申请实施例进行详细的解释说明之前，先对本申请实施例的应用场景予以介绍。
在工业领域中，工业产品在生产、制造和维护的过程可能会出现工业缺陷。例如，在油气管道、航天器、汽车等工业产品的制造或维护过程中，经常使用焊接，在焊接留下的焊缝处可能存在焊缝缺陷。再例如，在汽车某部件上，可能会由于生产或维护工艺的问题，导致该部件上存在裂纹、划痕等缺陷。工业产品中诸如此类的缺陷往往会影响到工业产品 的正常使用，基于此，对工业产品进行工业缺陷的检测是非常必要的。当前，基于工业产品的图像进行检测是工业缺陷检测的重要手段之一。其中，一种主要的方法是依靠人工从工业产品的图像中识别工业缺陷。然而这种方法不仅不利于工作人员的健康，识别质量也难以保证。近些年来，随着神经网络模型在图像识别领域的广泛应用，已有一些基于神经网络模型的工业缺陷识别方法出现，以替代人工识别。但是目前基于神经网络模型的工业缺陷识别方法仍存在一些问题，主要表现在两个方面，一方面是该种方法往往仅适用于大尺寸缺陷的识别，对于小尺寸缺陷的识别效果较差；另一方面，该种方法往往仅能识别图像中是否存在工业缺陷，但是无法对工业缺陷进行定位和尺寸估计，识别维度较为单一。基于此，本申请实施例提供了一种基于AI模型对工业产品图像中的工业缺陷进行识别方法，以丰富工业缺陷的识别维度，提高小尺寸工业缺陷的识别效果。
需要说明的是，本申请实施例提供的工业缺陷识别方法由工业缺陷识别系统01来执行。其中，该工业缺陷识别系统01可以由软件系统来实现，也可以由硬件设备来实现，还可以由软件系统和硬件设备结合来实现。
其中，当该工业缺陷识别系统01为软件系统时，参见图1，该工业缺陷识别系统01可以在逻辑上划分识别装置10和训练装置11。其中，识别装置10和训练装置11在逻辑上均可以划分为多个模块，每个模块具有不同的功能。示例性地，参见图1，该识别装置10包括目标区域检测模块101、缺陷粗检测模块102和缺陷精细识别模块103。需要说明的是，本申请实施例仅对识别装置10的结构和功能模块进行示例性划分，但是并不对其具体划分做任何限定。
目标区域检测模块101，用于获取待识别图像，并根据目标区域检测模型从待识别图像中提取至少一个目标区域。其中，待识别图像为包含工业产品的图像。至少一个目标区域是指可能存在工业缺陷的区域。例如，在焊缝缺陷的检测中，该目标区域可以是待识别图像中包含有完整焊缝的区域。其中，目标区域可以为一个多边形区域。示例性地，该目标区域为多自由度四边形。所谓多自由度四边形是指该四边形的各边与待识别图像的边之间的夹角或平行关系自由，不局限于垂直或平行关系，并且，该四边形的四条边之间的夹角或平行关系自由，也不局限于垂直或平行关系。也就是说，多自由度四边形的各条边与待识别图像的各个边的角度可以为0度至180度中的任一数值。并且，多自由四边形区域的相邻边之间可以垂直也可以不垂直，对边之间可以平行也可以不平行。
需要说明的是，目标区域检测模型为一个利用工业训练样本训练好的AI模型。目标区域检测模型用于从待识别图像中确定可能出现某种工业缺陷的感兴趣区域，也即目标区域，并输出目标区域在待识别图像中的位置。
缺陷粗检测模块102，用于与目标区域检测模块101和缺陷精细识别模块103进行通信，接收目标区域检测模块101提取的至少一个目标区域，并根据至少一个目标区域和缺陷检测模型，获得至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域和缺陷粗选区域包含的工业缺陷的类型。其中，缺陷检测模型是一个利用缺陷训练样本训练好的AI模型。并且，该缺陷检测模型用于从目标区域中进一步检测包含有工业缺陷的缺陷粗选区域，并输出该缺陷粗选区域在目标区域中的位置信息。例如，在焊缝缺陷的检测中，该 缺陷粗选区域可以是指从包含有完整焊缝的区域中提取到一个更小的包含有焊缝缺陷的区域。除此之外，该缺陷检测模型还用于输出缺陷粗选区域内包含的工业缺陷的类型。当然，需要说明的是，由于前述目标区域检测模块101提取的至少一个目标区域是工业缺陷可能出现的区域，也就是说，有些目标区域中也可能并不包含有工业缺陷，在这种情况下，缺陷检测模型在对这些目标区域进行检测后，将会输出用于指示目标区域内无缺陷的识别结果。
缺陷精细识别模块103，用于与缺陷粗检测模块102进行通信，接收缺陷粗检测模块102提取的缺陷粗选区域，并确定缺陷粗选区域包含的工业缺陷的尺寸和位置。
在一些可能的实现方式中，缺陷精细识别模块103还用于接收缺陷粗检测模块102发送的缺陷粗选区域在目标区域中的位置信息，以及与目标区域检测模块101进行通信，接收目标区域检测模块101发送的目标区域在待识别图像中的位置信息。之后，缺陷精细识别模块103根据缺陷粗选区域中的灰度分布，确定缺陷粗选区域中包含的工业缺陷的尺寸以及该工业缺陷在缺陷粗选区域中的位置。根据缺陷粗选区域在目标区域中的位置信息、包含该缺陷粗选区域的目标区域在待识别图像中的位置信息以及该工业缺陷在缺陷粗选区域中的位置，获得该工业缺陷在待识别图像中的位置。
可选地，在一些可能的情况中，参见图1，该识别装置10还可以包括报告生成模块104，该报告生成模块104用于与缺陷精细识别模块103进行通信，接收缺陷精细识别模块103发送的识别结果，进而根据该识别结果生成检测报告，将该检测报告提供给用户。其中，该识别结果包括在待识别图像中检测到的工业缺陷的类型、尺寸和位置，相应地，该检测报告包括上述识别结果，除此之外，该检测报告还可以包括其他内容，例如，还可以包括根据该识别结果确定的待识别图像中的工业产品是否合格的信息等，本申请实施例对此不作限定。
可选地，报告生成模块104还用于与缺陷粗检测模块102进行通信，接收缺陷粗检测模块102输出的用于指示某些目标区域内无缺陷的识别结果，进而根据该识别结果生成检测报告。
由前述介绍可知，目标区域检测模型和缺陷检测模型均为训练好的AI模型。在这两个模型被用于进行工业缺陷的检测之前，可由训练装置11对目标区域检测网络进行训练得到目标区域检测模型，对缺陷检测网络进行训练得到缺陷检测模型。参见图1，该训练装置11包括目标区域检测模型生成模块111、目标区域提取模块112、缺陷检测模型生成模块113以及存储模块114。
目标区域检测模型生成模块111，用于从存储模块114中获取多个工业训练样本，并通过获取的多个工业训练样本对目标区域检测网络进行训练，从而得到目标区域检测模型。其中，该多个工业训练样本为均包含目标区域的图像，该目标区域是指有可能出现工业缺陷的区域。另外，每个工业训练样本均对应有目标区域标注信息，该目标区域标识信息包括相应工业训练样本中标注的多自由度四边形的位置信息和用于指示相应多自由度四边形为目标区域的标注类别信息。
目标区域提取和缺陷标注模块112，用于从存储模块114中获取多个缺陷训练样本，并通过训练好的目标区域检测模型从每个缺陷训练样本中提取目标区域。其中，多个缺陷训 练样本均为包含有目标区域的训练样本。在从各个缺陷训练样本中提取到目标区域之后，目标区域提取和缺陷标注模块112还可以获取由用户对目标区域中的工业缺陷进行标注后得到的缺陷标注信息，其中，该缺陷标注信息用于指示相应目标区域内的工业缺陷的缺陷类型和位置。
缺陷检测模型生成模块113，用于与目标区域提取和缺陷标注模块112进行通信，接收目标区域提取和缺陷标注模块112发送的多个目标区域以及每个目标区域的缺陷标注信息，进而通过该多个目标区域和每个目标区域的缺陷标注信息对缺陷检测网络进行训练，从而得到缺陷检测模型。
在本申请实施例中，训练装置11与识别装置10为两个独立的装置。在这种情况下，经过训练装置11训练得到目标区域检测模型和缺陷检测模型将被部署至识别装置10中。可选地，训练装置11也可以是识别装置10的一部分。
上述介绍的工业缺陷识别系统01可以灵活地部署。例如，该工业缺陷识别系统01部署在云环境。云环境是云计算模式下利用基础资源向用户提供云服务的实体，云环境包括云数据中心和云服务平台。
云数据中心包括云服务提供商拥有的大量基础资源(包括计算资源、存储资源和网络资源)，云数据中心包括的计算资源可以是大量的计算设备(例如服务器)。该工业缺陷识别系统01可以是部署在云数据中心中的服务器或者虚拟机上的软件系统，该软件系统可以用于进行工业缺陷的识别，该软件系统可以分布式地部署在多个服务器上、或者分布式地部署在多个虚拟机上、或者分布式地部署在虚拟机和服务器上。例如，如图2所示，该工业缺陷识别系统01部署在云环境中。客户端02可以将待识别图像发送至该工业缺陷识别系统01，该工业缺陷识别系统01在接收到待识别图像之后，可以根据目标区域检测模型从待识别图像中提取至少一个目标区域，根据至少一个目标区域和缺陷检测模型，获得至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域和缺陷粗选区域包含的工业缺陷的类型。之后，确定缺陷粗选区域包含的工业缺陷的尺寸和位置。之后，该工业缺陷识别系统01将上述检测得到的工业缺陷的类型、位置和尺寸反馈给客户端02。
示例性地，图3为本申请中的工业缺陷识别系统01的一种应用示意图，如图3所示，工业缺陷识别系统01可以由云服务提供商部署在云数据中心，云服务提供商将工业缺陷识别系统01提供的功能抽象成为一项云服务，云服务平台供用户咨询和购买这项云服务。用户购买这项云服务后即可使用云数据中心的该工业缺陷识别系统01提供的工业缺陷识别服务。该工业缺陷识别系统01还可以由租户部署在租户租用的云数据中心的计算资源中，租户通过云服务平台购买云服务提供商提供的计算资源云服务，在购买的计算资源中运行该工业缺陷识别系统01，使得该工业缺陷识别系统01进行工业缺陷的识别。
可选地，该工业缺陷识别系统01还可以是边缘环境中运行在边缘计算设备上的软件系统或者是由边缘环境中的一个或多个边缘计算设备组成。所谓边缘环境是指某个应用场景中包括一个或多个边缘计算设备在内的设备集合，其中，该一个或多个边缘计算设备可以是一个数据中心内的计算设备或者是多个数据中心的计算设备。当工业缺陷识别系统01为软件系统时，工业缺陷识别系统01可以分布式地部署在多台边缘计算设备，也可以集中地部署在一台边缘计算设备。示例性地，如图4所示，该工业缺陷识别系统01分布式地部署 在某个企业的数据中心包括的边缘计算设备03中，该企业中的客户端04可以将待识别图像发送至该工业缺陷识别系统01，该工业缺陷识别系统01在接收到待识别图像之后，可以通过本申请实施例提供的方法识别该待识别图像中包含的工业缺陷，并将识别结果反馈给客户端04。
当该工业缺陷识别系统01为硬件设备时，该工业缺陷识别系统01可以为任意环境中的计算设备，例如，可以为前述介绍的边缘计算设备，也可以为前述介绍的云环境下的计算设备。图5是本申请实施例提供的一种计算设备500的结构示意图。该计算设备500包括处理器501，通信总线502，存储器503以及至少一个通信接口504。
处理器501可以是一个通用中央处理器(Central Processing Unit，CPU)，特定应用集成电路(application-specific integrated circuit，ASIC)，图形处理器(graphics processing unit，GPU)或其任意组合。处理器501可以包括一个或多个芯片，处理器501可以包括AI加速器，例如：神经网络处理器(neural processing unit，NPU)。
通信总线502可包括在计算设备500各个部件(例如，处理器501、存储器503、通信接口504)之间传送信息的通路。
存储器503可以是只读存储器(read-only memory，ROM)或可存储静态信息和指令的其它类型的静态存储设备，随机存取存储器(random access memory，RAM))或者可存储信息和指令的其它类型的动态存储设备，也可以是电可擦可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)或其它光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其它磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其它介质，但不限于此。存储器503可以是独立存在，通过通信总线502与处理器501相连接。存储器503也可以和处理器501集成在一起。存储器503可以存储计算机指令，当存储器503中存储的计算机指令被处理器501执行时，可以实现本申请的工业缺陷识别方法。另外，存储器503中还可以存储有处理器在执行上述方法的过程中所需的数据以及所产生的中间数据和/或结果数据。
通信接口504，使用任何收发器一类的装置，用于与其它设备或通信网络通信，如以太网，无线接入网(RAN)，无线局域网(Wireless Local Area Networks，WLAN)等。
在具体实现中，作为一种实施例，处理器501可以包括一个或多个CPU。
在具体实现中，作为一种实施例，计算机设备可以包括多个处理器。这些处理器中的每一个可以是一个单核(single-CPU)处理器，也可以是一个多核(multi-CPU)处理器。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。
在本申请实施例中，工业缺陷识别系统采用目标区域检测模型和缺陷检测模型来对待识别图像中的工业缺陷进行识别，其中，目标区域检测模型和缺陷检测模型均为训练好的AI模型。基于此，接下来首先对目标区域检测模型和缺陷检测模型的训练过程进行介绍。
图6是本申请实施例提供的一种训练目标区域检测模型和缺陷检测模型的流程图。该方法可以应用于前述工业缺陷识别系统中的训练装置中，参见图6，该方法包括以下步骤：
步骤601：获取多个工业训练样本，每个工业训练样本均对应有目标区域标注信息。
在本申请实施例中，训练装置收集多个工业训练样本。其中，该工业训练样本可以是由其他采集设备采集并发送至该训练装置的图像，例如，可以由工业环境中的摄像头采集并发送至该训练装置。当然，该工业训练样本也可以由采集设备采集之后发送到其他设备存储，再由其他设备发送至该训练装置。或者，工业训练样本也可以是客户端发送至工业缺陷识别系统进行识别后的图像。也即，训练装置可以收集经过工业缺陷识别系统中的识别装置识别之后的图像，将这些图像作为工业训练样本。
需要说明的是，该多个工业训练样本均为包含有目标区域的图像。其中，该目标区域是指可能存在工业缺陷的区域，该工业缺陷是指工业产品在生产、制造或维护过程中产生的缺陷，例如，该工业缺陷为焊缝缺陷，由于焊缝缺陷只出现在焊缝所在的区域，所以此时目标区域即为焊缝区域。再例如，对于某种工业缺陷，该工业缺陷只出现在汽车的某个部件上，则此时目标区域即为该部件所在的区域，本申请实施例在此不再赘述。
在获取到多个工业训练样本之后，训练装置向用户依次显示每个工业训练样本，以便用户对每个工业训练样本进行目标区域标注。之后，训练装置获取用户在每个工业训练样本中进行标注时产生的目标区域标注信息。
其中，用户采用四边形标注框在该工业训练样本中对目标区域进行标注，从而得到一个包含有目标区域的四边形。
需要说明的是，该四边形标注框为一个多自由度四边形标注框。相应地，标注得到的四边形也为一个多自由度四边形。所谓多自由度四边形是指该四边形的四条边与工业训练样本的四条边之间的角度为0-180度之间的任一数值。换句话说，该多自由四边形的各条边与工业训练样本的各条边之间的可以垂直或不垂直，也可以平行也可以不平行。除此之外，并且，该多自由度四边形的相临边之间可以垂直或不垂直，对边之间可以平行或不平行。
示例性地，图7是本申请实施例示出的在工业训练样本中标注得到的一个多自由度四边形。如图7所示，该多自由度四边形的边Q 1Q 2与工业训练样本的各条边既不垂直也不平行，其他三条边与该工业训练样本的各条边也既不垂直也不平行。另外，如图7中所示，多自由四边形的边Q 1Q 2和边Q 3Q 4平行，边Q 1Q 4和边Q 2Q 3不平行，并且，该多自由度四边形中每相邻的两条边之间也均不垂直。
以工业缺陷为焊缝缺陷，目标区域为焊缝区域为例，用户采用四边形标注框在工业训练样本中进行标注时，均可以采用该四边形标注框包围焊缝，得到一个包围焊缝的四边形。需要说明的是，由于焊缝通常是长条状的不规则的四边形，且焊缝在工业训练样本中并不是水平或竖直的，因此，在本申请实施例中，用户可以采用多自由度的四边形标注框来框选焊缝，这样，该多自由度的四边形标注框能够更好的适应焊缝的形状，更为紧密的包围该焊缝，从而能够尽可能的减少标注得到的多自由度四边形内不包含有焊缝缺陷的区域的面积。
用户在通过四边形标注框在工业训练样本中标注得到四边形之后，训练装置自动获取 该四边形在工业训练样本中的位置信息。其中，该位置信息可以包括四边形的四个顶点在工业训练样本中的坐标信息。
除此之外，针对每个工业训练样本，在用户标注得到四边形之后，训练装置可以生成该四边形的标注类别信息，该标注类别信息用于指示该四边形为目标区域。其中，标注类别信息可以为第一标识，示例性地，第一标识为“1”。相应的，训练装置可以将标注类别信息和该四边形的位置信息作为该工业训练样本对应的目标区域标注信息。
步骤602：通过多个工业训练样本和每个工业训练样本的目标区域标注信息对目标区域检测网络进行训练，得到目标区域检测模型。
在获取到多个工业训练样本和每个工业训练样本的目标区域标注信息之后，训练装置通过该多个工业训练样本和每个工业训练样本的目标区域标注信息对目标区域检测网络进行训练，从而得到目标区域检测模型。接下来以多个工业训练样本中的一个工业训练样本，且标注得到的四边形区域为多自由度四边形为例来对该过程进行说明，为了方便说明，下文中将该工业训练样本称为第一工业训练样本。
示例性地，训练装置根据第一工业训练样本的目标区域标注信息，确定第一外接矩形和第二外接矩形，第一外接矩形是指第一工业训练样本中标注的多自由度四边形的最小旋转外接矩形，第二外接矩形是指第一外接矩形的水平外接矩形；通过目标区域检测网络确定第一工业训练样本中的多自由度四边形的预测值、第一外接矩形的预测值、第二外接矩形的预测值和预测类别信息；根据第一工业训练样本的目标区域标注信息、第一外接矩形、第二外接矩形、第一工业训练样本中的多自由度四边形的预测值、第一外接矩形的预测值、第二外接矩形的预测值和预测类别信息，计算损失函数值；根据损失函数值对目标区域检测网络进行参数更新。
在本申请实施例中，考虑到多自由度四边形的自由度较高，回归难度比较大，所以训练装置首先在第一工业训练样本中确定标注的多自由度四边形的最小旋转外接矩形，也即第一外接矩形，之后，确定第一外接矩形的水平外接矩形，也即第二外接矩形。其中，第一外接矩形是指四条边与第一工业训练样本的四条边均不平行且面积最小的外接矩形。第二外接矩形是指第一外接矩形的外接矩形中面积最小且相邻两条边分别与第一工业训练样本的相邻两条边平行的外接矩形。例如，如图8所示，第一工业训练样本中标注的多自由度四边形为Q 1Q 2Q 3Q 4，第一外接矩形为R 1R 2R 3R 4，第二外接矩形为K 1K 2K 3K 4。
另外，训练装置通过目标区域检测网络在第一工业训练样本中检测，从而获得标注的多自由度四边形的预测值、第一外接矩形的预测值、第二外接矩形的预测值和预测类别信息。该预测类别信息用于指示标注的多自由度四边形的预测值为前景区域还是背景区域。例如，如果标注的多自由度四边形的预测值为前景区域，则预测类别信息为第一标识，如果标注的多自由度四边形的预测值为背景区域，则预测类别信息为第二标识。
在通过目标区域检测网络得到标注的多自由度四边形的预测值、第一外接矩形的预测值、第二外接矩形的预测值和预测类别信息之后，训练装置可以利用上述预测值、标注得到的各个四边形的参数信息、预测类别信息以及标注类别信息来计算损失函数值，进而根据计算得到的损失函数值来对目标区域检测网络进行参数更新。
其中，在计算损失函数值时，训练装置可以对标注得到的数据和预测得到的数据分别 进行参数化，进而通过参数化后的参数值来计算损失函数值。
示例性地，以第一工业训练样本中标注的多自由度四边形为一组对边平行、另一组对边不平行的四边形为例，例如，如图8所示，此时，该多自由度四边形可以通过该四边形的三个顶点Q 1、Q 2、Q 3的顶点坐标和边Q 3Q 4的长度来表征，也即，该多自由度四边形的参数信息为
Figure PCTCN2021124254-appb-000001
其中，
Figure PCTCN2021124254-appb-000002
为顶点Q 1的坐标，
Figure PCTCN2021124254-appb-000003
为顶点Q 2的坐标，
Figure PCTCN2021124254-appb-000004
为顶点Q 3的坐标，w Q为边Q 3Q 4的长度。同理，假设该多自由度四边形的预测值为q 1q 2q 3q 4，则该多自由度四边形的预测值的参数信息为
Figure PCTCN2021124254-appb-000005
对于第一工业训练样本中的多自由度四边形的最小旋转外接矩形，也即第一外接矩形，可以通过该矩形的相邻的两个顶点的坐标和这两个顶点组成的边的相邻边的长度来表征。例如，如图8所示，第一外接矩形R 1R 2R 3R 4可以通过顶点R 1和R 2的坐标以及边R 2R 3的长度来表征，此时，第一外接矩形的参数信息为
Figure PCTCN2021124254-appb-000006
其中，
Figure PCTCN2021124254-appb-000007
为顶点R 1的坐标，
Figure PCTCN2021124254-appb-000008
为顶点R 2的坐标，h R为R 2R 3的长度，也即该第一外接矩形的高。或者，第一外接矩形也可以通过顶点R 2和R 3的坐标以及边R 1R 2的长度来表征，本申请实施例对此不再赘述。同理，假设该第一外接矩形的预测值为r 1r 2r 3r 4，则第一外接矩形的预测值的参数信息可以为
Figure PCTCN2021124254-appb-000009
对于第二外接矩形，由于第二外接矩形为水平外接矩形，因此，第二外接矩形可以通过一个顶点的坐标和该外接矩形的宽和高来表征。例如，如图8中所示，第二外接矩形K 1K 2K 3K 4可以通过顶点K 1的坐标、边K 1K 2和边K 2K 3的长度来表征。也即，第二外接矩形的参数信息可以为
Figure PCTCN2021124254-appb-000010
其中，
Figure PCTCN2021124254-appb-000011
为顶点K 1的坐标，w K为第二外接矩形的宽，也即，边K 1K 2的长度，h K为第二外接矩形的高，也即，边K 2K 3的长度。同理，假设第二外接矩形的预测值为k 1k 2k 3k 4，则第二外接矩形的预测值的参数信息可以为
Figure PCTCN2021124254-appb-000012
对于上述标注得到的各个四边形的参数信息和预测得到的各个四边形的参数信息，训练装置可以采用目标区域检测网络在预测上述各个四边形的过程中所生成的水平矩形目标区域来对其进行参数化。假设在采用目标区域检测网络预测上述各个四边形的过程中所生成的水平矩形目标区域的参数信息为(x,y,w,h)，则训练装置可以通过下述公式(1)-(3)对第一工业训练样本中标注的多自由度四边形、该多自由度四边形的预测值的参数信息进行参数化，通过公式(1)、(2)和(4)对第一外接矩形和第一外接矩形的预测值的参数信息进行参数化，通过公式(1)-(4)对第二外接矩形和第二外接矩形的预测值的参数信息进行参数化，从而得到各个四边形的参数组。
Figure PCTCN2021124254-appb-000013
Figure PCTCN2021124254-appb-000014
Figure PCTCN2021124254-appb-000015
Figure PCTCN2021124254-appb-000016
其中，当采用公式(1)-(3)对第一工业训练样本中标注的多自由度四边形和该多自由度四边形的预测值的参数信息进行参数化时，i的取值为1,2,3。N的取值为Q,q。这样，该多自由度四边形的参数组可以表示为
Figure PCTCN2021124254-appb-000017
该多自由度四边形的预测值的参数组可以表示为
Figure PCTCN2021124254-appb-000018
当采用公式(1)、(2)和(4)对第一外接矩形和第一外接矩形的预测值的参数信息进行参数化时，i的取值为1,2。N的取值为R,r。这样，第一外接矩形的参数组可以表示为
Figure PCTCN2021124254-appb-000019
第一外接矩形的预测值的参数组可以表示为
Figure PCTCN2021124254-appb-000020
当采用公式(1)-(4)对第二外接矩形和第二外接矩形的预测值的参数信息进行参数化时，i的取值为1。N的取值为K,k。这样，第二外接矩形的参数组可以表示为
Figure PCTCN2021124254-appb-000021
第二外接矩形的预测值的参数组可以表示为
Figure PCTCN2021124254-appb-000022
在将标注得到的数据和预测得到的数据分别进行参数化之后，训练装置基于上述参数化后的各个参数组，通过下述公式(5)来计算损失函数值。
Figure PCTCN2021124254-appb-000023
其中，L cls()为分类损失函数，l p为预测类别信息，l g为标注类别信息，λ 1、λ 2、λ 3为预设的平衡因子，用于调整相应项在计算损失函数值的过程中的相对贡献；
Figure PCTCN2021124254-appb-000024
为标注的多自由度四边形的参数的回归函数，
Figure PCTCN2021124254-appb-000025
为第一外接矩形的参数的回归函数，
Figure PCTCN2021124254-appb-000026
为第二外接矩形的参数的回归函数。
在通过上述公式获得损失函数值之后，训练装置可以计算得到的损失函数的引导下迭代训练目标区域检测网络，从而得到目标区域检测模型。
需要说明的是，上文中用于表征标注得到的各个四边形和预测得到的各个四边形的参数仅是本申请实施例提供的一种示例，在一些可能的场景中，也可以通过其他参数信息来表征各个四边形，例如，通过各个四边形的四个顶点的坐标来表征相应四边形，本申请实施例对此不做限定。
步骤603：获取多个缺陷训练样本。
在本申请实施例中，训练装置收集多个缺陷训练样本。其中，该多个缺陷训练样本均为包含有目标区域的图像。需要说明的是，在一些可能的情况下，该多个缺陷训练样本与上述用于训练得到目标区域检测模型的多个工业训练样本不存在交集，在另一些可能的情况中，该多个缺陷训练样本中可以包含有上述多个工业训练样本中包含有工业缺陷的工业训练样本。
步骤604：通过目标区域检测模型从多个缺陷训练样本中提取多个目标区域。
在训练得到目标区域检测模型并获取到缺陷训练样本之后，训练装置通过目标区域检测模型从每个缺陷训练样本中提取目标区域。
需要说明的是，由于目标区域检测模型是通过在工业训练样本中标注可能存在工业缺陷的四边形训练得到的，所以该目标区域检测模型以多自由度四边形作为检测框，这样，通过该目标区域检测模型，能够从每个缺陷训练样本中检测出可能存在工业缺陷且形状为四边形的目标区域。其中，当训练时所采用的四边形为多自由度四边形，则从每个缺陷训练样本中检测到的也将为多自由度四边形。然而，由于后续训练缺陷检测网络时，输入到缺陷检测网络的训练样本通常要求为矩形，因此，在本申请实施例中，在检测出目标区域之后，训练装置可以进一步的确定该目标区域的最小外接矩形，在最小外接矩形和该目标区域之间填充背景色，然后从缺陷训练样本中提取该最小外接矩形，将提取的图像作为后续缺陷检测网络的输入。
示例性地，对于检测焊缝缺陷的目标区域检测模型，在用于训练该目标区域检测模型的各个工业训练样本中可以通过多自由度四边形来标注焊缝，这样，通过该目标区域检测模型能够从缺陷训练样本中检测到包含有焊缝且形状为多自由度四边形的目标区域。训练装置可以确定该目标区域的最小外接矩形，并在最小外接矩形和该目标区域之间填充背景色，之后，从缺陷训练样本中提取包含有该目标区域的最小外接矩形。
步骤605：获取多个目标区域中包含有工业缺陷的目标区域的缺陷标注信息。
在从每个缺陷训练样本中提取得到目标区域之后，由于该目标区域可能包含有工业缺陷，也可能不包含，所以训练装置可以向用户显示提取到的多个目标区域，由用户从多个目标区域中挑选出包含有工业缺陷的目标区域，并对包含有工业缺陷的目标区域内的工业缺陷进行标注。之后，训练装置获取用户在每个目标区域中进行标注时产生的缺陷标注信息。
其中，用户可以采用水平矩形标注框来标注目标区域中包含的工业缺陷，并在缺陷类型选项中输入该水平矩形标注框内包含的工业缺陷的缺陷类型。相应地，训练装置获取用户在目标区域内标注的水平矩形标注框的位置，将该位置作为目标区域内包含的工业缺陷的位置。与此同时，训练装置获取用户输入的工业缺陷的缺陷类型。将获取到的工业缺陷的位置和缺陷类型作为目标区域的缺陷标注信息。
需要说明的是，在通过水平矩形标注框标注目标区域中包含的工业缺陷时，该水平矩形标注框可以在包围该工业缺陷的情况下面积尽可能的小，也即，该水平矩形标注框可以紧紧包围该工业缺陷。训练装置在获取水平矩形标注框的位置时，可以获取该水平矩形标注框的中心点的位置。
步骤606：通过包含有工业缺陷的目标区域和相应目标区域的缺陷标注信息对缺陷检测网络进行训练，得到缺陷检测模型。
在获取到包含有工业缺陷的目标区域以及相应目标区域的缺陷标注信息之后，训练装置通过每个包含有工业缺陷的目标区域和对应的缺陷标注信息对缺陷检测网络进行训练，从而得到缺陷检测模型。
其中，由前述步骤604中的介绍可知，在提取目标区域时，提取的可以是一个包含有目标区域的最小外接矩形。基于此，对于任一个包含有工业缺陷的目标区域，训练装置将 包含该目标区域的最小外接矩形输入至缺陷检测网络，缺陷检测网络对该目标区域中包含的工业缺陷进行识别，从而得到该目标区域内包含的工业缺陷的预测位置和预测缺陷类型。根据该工业缺陷的预测位置和该目标区域的缺陷标注信息中包含的标注的缺陷位置计算位置损失函数值，根据该工业缺陷的预测缺陷类型和缺陷标注信息中包含的标注的缺陷类型计算类型损失函数值。之后，根据该位置损失函数值和类型损失函数值对缺陷检测网络进行参数更新，从而得到缺陷检测模型。
在本申请实施例中，通过多自由度四边形对工业训练样本中可能包含有工业缺陷的目标区域进行标注，进而通过该工业训练样本对目标区域检测网络进行训练，以获得目标区域检测模型，这样，该目标区域检测模型能够更为准确的检测图像中以不同方向、形态分布的可能包含缺陷的各种目标区域。另外，在本申请实施例中，先训练得到目标区域检测模型，之后，通过该目标区域检测模型对缺陷训练样本中包含的工业缺陷进行粗定位，以提取目标区域，进而通过该目标区域中标注的缺陷的精细位置和缺陷类型训练缺陷检测网络，以得到缺陷检测模型。这样，后续在通过该缺陷检测模型检测从待识别图像中提取出来的目标区域时，能够得到工业缺陷在该目标区域中更为准确的定位，并能够输出具体的工业缺陷的缺陷类型，不仅提高了定位的准确性，而且丰富了缺陷识别得到维度。
上述实施例主要介绍了目标区域检测模型和缺陷检测模型的训练过程，接下来对采用该目标区域检测模型和缺陷检测模型对待识别图像进行识别的过程进行介绍。
图9是本申请实施例提供的一种工业缺陷的识别方法流程图。该方法可以应用于前述介绍的工业缺陷识别系统中的识别装置中，参见图9，该方法包括以下步骤：
步骤901：获取待识别图像，该待识别图像为反映工业产品的图像。
在本申请实施例中，识别装置获取待识别图像，该待识别图像是该识别装置中存储的，或者是该待识别图像是其他设备发送至该识别装置的。其中，其他设备可以为采集该待识别图像的设备，也可以是存储该待识别图像的设备，本申请实施例对此不做限定。
步骤902：根据目标区域检测模型从待识别图像中提取至少一个目标区域，该至少一个目标区域为可能存在工业缺陷的区域。
在获取到待识别图像之后，识别装置将该待识别图像作为目标区域检测模型的输入，通过该目标区域检测模型从该待识别图像中提取至少一个目标区域，其中，该至少一个目标区域为可能存在工业缺陷的区域，也即，该至少一个目标区域是指某种工业缺陷可能发生的区域。
示例性地，以一个目标区域为例，识别装置通过目标区域检测模型在待识别图像中确定目标区域，并输出目标区域在待识别图像中的位置。之后，识别装置根据目标区域在待识别图像中的位置，获得包含该目标区域的最小外接矩形。其中，目标区域在待识别图像中的位置可以通过目标区域的各个顶点的坐标来表征，也可以根据目标区域的形状，采用别的参数来表征，本申请实施例对此不做限定。
需要说明的是，由于目标区域检测模型是通过在工业训练样本中标注的四边形训练得到的，所以通过该目标区域检测模型，能够从待识别图像中确定出形状为四边形的目标区域。其中，当训练时所采用的四边形为多自由度四边形，则从待识别图像中确定出的目标 区域也将为多自由度四边形。由于后续检测第一区域中的工业缺陷时需要通过缺陷检测模型来检测，而输入到缺陷检测模型的图像通常要求为矩形，因此，在本申请实施例中，在检测出目标区域之后，识别装置可以进一步的确定目标区域的最小外接矩形，在最小外接矩形和目标区域之间填充背景色，然后根据目标区域的最小外接矩形对待识别图像进行裁剪，从而得到包含有目标区域的图像。
例如，在焊缝缺陷识别的场景中，在训练目标区域检测模型时，在工业训练样本中通过多自由度四边形标注框标注焊缝，这样，在采用该目标区域检测模型对包含有焊缝的待识别图像进行检测时，则能够从该待识别图像中识别得到多自由度的包含有焊缝的目标区域。之后，识别装置可以确定目标区域的最小外接矩形，并在目标区域和最小外接矩形之间的空隙中填充背景色，将该最小外接矩形从待识别图像中提取出来，从而得到包含有目标区域的图像。
步骤903：根据至少一个目标区域和缺陷检测模型，获得至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域，和缺陷粗选区域包含的工业缺陷的类型。
在通过目标区域检测模型提取得到至少一个目标区域之后，识别装置将至少一个目标区域作为缺陷检测模型的输入，通过缺陷检测模型进行推理，获得至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域在目标区域中的位置，和缺陷粗选区域包含的工业缺陷的缺陷类型；之后，根据缺陷粗选区域在目标区域中的位置，从目标区域中提取缺陷粗选区域。
其中，仍以至少一个目标区域中的一个目标区域为例，识别装置将包含有该目标区域的图像输入至缺陷检测模型。缺陷检测模型对该目标区域进行处理。需要说明的是，由于目标区域是工业缺陷可能出现的区域，因此，该目标区域中可能包含有工业缺陷，也可能不包含。基于此，如果该目标区域中不包含有工业缺陷，则缺陷检测模型将会输出用于指示该目标区域内无缺陷的识别结果。如果该目标区域中包括有工业缺陷，则缺陷检测模型将能够在该目标区域中确定出缺陷粗选区域并对缺陷粗选区域内的工业缺陷进行识别，之后，缺陷检测模型输出缺陷粗选区域在该目标区域内的位置以及识别得到的缺陷粗选区域内的工业缺陷的缺陷类型。识别装置在得到缺陷粗选区域在目标区域内的位置之后，根据该位置对目标区域进行裁剪，从而得到缺陷粗选区域。其中，缺陷粗选区域在目标区域内的位置同样可以采用缺陷粗选区域的顶点坐标来表征，或者是根据缺陷粗选区域的形状采用其他参数来表征。
需要说明的是，如果前述在训练缺陷检测模型时，在从缺陷训练样本提取到的目标区域内是通过水平矩形标注框标注的工业缺陷，则在本步骤中，在目标区域中确定的缺陷粗选区域也是一个水平矩形区域。
例如，在焊缝缺陷识别场景中，通过目标区域检测模型从待识别图像中提取得到一个包含焊缝的目标区域，将该目标区域输入至该缺陷检测模型，该缺陷检测模型对目标区域进行处理，从而在该目标区域中确定出一个包含有焊缝缺陷的水平矩形区域。根据确定出的水平矩形区域的位置，将该水平矩形区域从目标区域中提取出来，从而得到缺陷粗选区域。此时，缺陷粗选区域是一个包含有焊缝缺陷且面积比目标区域更小的区域。这样，焊缝缺陷在缺陷粗选区域中的占比将大于在目标区域中的占比。
由上述介绍可知，通过目标区域检测模型提取到的目标区域是一个某种工业缺陷可能发生的范围较大的区域，而通过缺陷检测模型得到缺陷粗选区域则是在目标区域的基础上进一步缩小范围后提取出的一个相较该目标区域更小的包含有工业缺陷的区域。这样，工业缺陷在该目标区域中的占比将大于在待识别图像中的占比，在缺陷粗选区域中的占比又将大于在目标区域中的占比。也即，通过从待识别图像中提取目标区域，然后再从目标区域中提取缺陷粗选区域，逐渐的提高了工业缺陷相对背景的占比。在此基础上，进一步地对缺陷粗选区域中的工业缺陷进行定位和尺寸估计，能够提高缺陷定位和尺寸估计的精度。尤其是对于小尺寸的工业缺陷，通过上述的方法不断放大该工业缺陷在图像中的占比，更有利于后续对该工业缺陷的定位和尺寸估计。
对于从待识别图像中提取到的至少一个目标区域中的每个目标区域，识别装置均能够参考上述方法对该至少一个目标区域进行检测，从而从该至少一个目标区域中获得包含有工业缺陷的目标区域的缺陷粗选区域和缺陷粗选区域包含的工业缺陷的类型。
步骤904：确定缺陷粗选区域包含的工业缺陷的尺寸和位置。
在通过步骤903获得缺陷粗选区域以及该缺陷粗选区域内包含的工业缺陷的缺陷类型，则识别装置可以进一步的通过对缺陷粗选区域进行分割来确定缺陷粗选区域内包含的工业缺陷的尺寸和位置。
在一种可能的实现方式中，如果缺陷粗选区域不是灰度图像，识别装置先对该缺陷粗选区域进行灰度化，当然，如果该缺陷粗选区域本身为灰度图像，则不必再执行灰度化操作。在此基础上，识别装置根据灰度化的缺陷粗选区域内的灰度分布，确定缺陷粗选区域包含的工业缺陷的尺寸以及该工业缺陷在缺陷粗选区域内的位置，根据缺陷粗选区域在目标区域中的位置、目标区域在待识别图像中的位置以及缺陷粗选区域包含的工业缺陷在缺陷粗选区域内的位置，确定缺陷粗选区域包含的工业缺陷在待识别图像中的位置。
需要说明的是，由于在图像中工业缺陷部分的灰度通常与非缺陷部分的灰度存在差别，所以，在本申请实施例中，识别装置可以采用区域内自适应阈值二值化方法，根据缺陷粗选区域内的灰度分布，确定该区域内的工业缺陷的位置和尺寸。
其中，识别装置根据灰度化的缺陷粗选区域内的灰度值分布，确定灰度阈值。之后，识别装置将灰度化的缺陷粗选区域内的每个像素点的灰度值与该灰度阈值进行比较，如果相应像素点的灰度值小于该灰度阈值，则将相应像素点的灰度值设置为第一数值，如果相应像素点的灰度值不小于该灰度阈值，则将该像素点的灰度值设置为第二数值。其中，第一数值和第二数值不相等。例如，第一数值为0，第二数值为255。通过上述方法，识别装置将灰度化的缺陷粗选区域内的各个像素点的灰度值二值化。之后，识别装置根据灰度化的缺陷粗选区域内灰度值为第一数值的像素点确定工业缺陷的位置和尺寸。
在确定出该工业缺陷在缺陷粗选区域内的位置之后，识别装置根据该工业缺陷在缺陷粗选区域内的位置以及缺陷粗选区域在相应的目标区域内的位置，通过该目标区域和缺陷粗选区域之间的坐标转换，得到该工业缺陷在目标区域内的位置。之后，识别装置再根据该工业缺陷在目标区域内的位置以及该目标区域在待识别图像中的位置，通过目标区域和待识别图像之间的坐标转换，得到该工业缺陷在待识别图像中的位置。
需要说明的是，在一些可能的情况中，识别装置也可以直接将确定出的该工业缺陷在 缺陷粗选区域内的位置作为最终得到的该工业缺陷的位置。
在得到该工业缺陷的位置和尺寸之后，识别装置可以生成检测报告，并向客户端反馈该检测报告。其中，该检测报告包括在待识别图像中检测到的工业缺陷的缺陷类型、尺寸和位置。可选地，该检测报告中还可以包括其他内容，例如，还可以包括根据上述的识别结果确定得到的待识别图像中的工业产品是否合格的信息等，本申请实施例对此不做限定。
可选地，如果缺陷检测模型在对目标区域检测时输出的是用于指示目标区域内无缺陷的识别结果，则识别装置生成包含有用于指示该目标区域内无缺陷的识别结果的检测报告，并向客户端反馈该检测报告。
综上可知，本申请实施例提供的工业缺陷识别方法带来的有益效果至少包括以下四个方面：
第一，在本申请实施例中，在目标区域提取环节，使用以多自由度四边形为检测框的神经网络检测模型，可以适合不同类型的目标区域，所述目标区域检测模型具有良好的通用性。
第二，在本申请实施例中，首先从待识别图像中提取目标区域，然后再从目标区域中获得包含有工业缺陷的缺陷粗选区域。通过两次区域提取，提高了工业缺陷相对背景的尺寸占比，能够提高小尺寸的工业缺陷的检出概率。
第三，在检测出包含有工业缺陷的缺陷粗选区域之后，对缺陷粗选区域中的工业缺陷进行进一步定位和尺寸估计，能够有效提高缺陷定位和尺寸估计的精度。
第四，本申请实施例中不仅对缺陷类型进行了识别，还能够对工业缺陷进行定位和尺寸估计，丰富了工业缺陷的识别维度。
本申请实施例还提供了如图1中所示的识别装置10，该识别装置10包括的模块和功能如前文的描述，在此不再赘述。
在一些实施例中，识别装置10中的目标区域检测模块101用于执行前述实施例中的步骤901和902。缺陷粗检测模块102用于执行前述实施例中的步骤903。缺陷精细识别模块103用于执行前述实施例中的步骤904。
可选地，该模型优化装置10还可以包括报告生成模块104，该报告生成模块104可以用于生成包括识别结果的检测报告，并向用户提供该检测报告。
本申请还提供了如图1中所示的训练装置11，该训练装置11包括的模块和功能如前文的描述，在此不再赘述。
在一些实施例中，训练装置11中的目标区域检测模型生成模块111可以用于执行前述实施例中的步骤601和步骤602。目标区域提取和缺陷标注模块112可以用于执行前述实施例中的步骤603至步骤605。缺陷检测模型生成模块113可以用于执行前述实施例中的步骤606。
本申请实施例还提供了一种如图5所示的计算设备500。计算设备500中的处理器501读取存储器503中存储的一组计算机指令以执行前述的工业缺陷识别方法。
由于本申请实施例提供的识别装置10中的各个模块可以分布式的部署在同一环境或不同环境的多个计算机上，因此，本申请还提供了一种如图10所示的计算设备(也可以称 为计算机系统)，该计算机系统包括多个计算机1000，每个计算机1000的结构与前述图5中的计算设备500的结构相同或相似，在此不再赘述。
上述每个计算机1000间通过通信网络建立通信通路。每个计算机1000上运行前述目标区域检测模块101、缺陷粗检测模块102、缺陷精细识别模块103、报告生成模块104中的任意一个或多个。任一计算机1000可以为云数据中心中的计算机(例如：服务器)，或边缘计算机，或终端计算设备。
上述各个附图对应的流程的描述各有侧重，某个流程中没有详述的部分，可以参见其他流程的相关描述。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。实现工业缺陷识别的计算机程序产品包括一个或多个进行工业缺陷识别的计算机指令，在计算机上加载和执行这些计算机程序指令时，全部或部分地产生按照本申请实施例图6和图9所述的流程或功能。
所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如：同轴电缆、光纤、数据用户线(digital subscriber line，DSL))或无线(例如：红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如：软盘、硬盘、磁带)、光介质(例如：数字通用光盘(digital versatile disc，DVD))、或者半导体介质(例如：固态硬盘(solid state disk，SSD))等。
应当理解的是，本文提及的“至少一个”是指一个或多个，“多个”是指两个或两个以上。在本文的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，为了便于清楚描述本申请实施例的技术方案，在本申请的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述并不用以限制本申请实施例，凡在本申请实施例的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请实施例的保护范围之内。

Claims (19)

  1. 一种工业缺陷识别方法，其特征在于，所述方法包括：
    获取待识别图像，所述待识别图像为反映工业产品的图像；
    根据目标区域检测模型从所述待识别图像中提取至少一个目标区域，所述至少一个目标区域为可能存在工业缺陷的区域；
    根据所述至少一个目标区域和缺陷检测模型，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域，和所述缺陷粗选区域包含的工业缺陷的类型；
    确定所述缺陷粗选区域包含的工业缺陷的尺寸和位置。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述至少一个目标区域和缺陷检测模型，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域，和所述缺陷粗选区域包含的工业缺陷的类型，包括：
    将所述至少一个目标区域作为所述缺陷检测模型的输入，通过所述缺陷检测模型进行推理，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域在目标区域中的位置，和所述缺陷粗选区域包含的工业缺陷的缺陷类型，其中，所述缺陷检测模型为利用缺陷训练样本训练完成的人工智能AI模型；
    根据所述缺陷粗选区域在所述目标区域中的位置，从所述目标区域中提取所述缺陷粗选区域。
  3. 根据权利要求1或2所述的方法，其特征在于，所述确定所述缺陷粗选区域包含的工业缺陷的尺寸和位置，包括：
    根据所述缺陷粗选区域内的灰度分布，确定所述缺陷粗选区域包含的工业缺陷在所述缺陷粗选区域内的位置和尺寸；
    根据所述缺陷粗选区域在所述目标区域中的位置、所述目标区域在所述待识别图像中的位置以及所述缺陷粗选区域包含的工业缺陷在所述缺陷粗选区域内的位置，确定所述缺陷粗选区域包含的工业缺陷在所述待识别图像中的位置。
  4. 根据权利要求1-3任一所述的方法，其特征在于，所述方法还包括：
    生成检测报告，所述检测报告包括在所述待识别图像中检测到的工业缺陷的缺陷类型、尺寸和位置；
    向用户提供所述检测报告。
  5. 根据权利要求1-4任一所述的方法，其特征在于，所述工业缺陷为焊缝缺陷，所述目标区域为焊缝所在的区域。
  6. 根据权利要求1-5任一所述的方法，其特征在于，所述方法还包括：
    获取多个工业训练样本，每个工业训练样本均对应有目标区域标注信息，所述目标区域标注信息包括相应工业训练样本中标注的多自由度四边形的位置信息和用于指示相应多自由度四边形为目标区域的标注类别信息；
    通过所述多个工业训练样本和每个工业训练样本的目标区域标注信息对目标区域检测网 络进行训练，得到所述目标区域检测模型。
  7. 根据权利要求1-6任一所述的方法，其特征在于，所述方法还包括：
    获取多个缺陷训练样本；
    通过所述目标区域检测模型从所述多个缺陷训练样本中提取多个目标区域；
    获取所述多个目标区域中包含有工业缺陷的目标区域的缺陷标注信息，所述缺陷标注信息用于指示相应目标区域内的工业缺陷的缺陷类型和位置；
    通过包含有工业缺陷的目标区域和相应目标区域的缺陷标注信息对缺陷检测网络进行训练，得到所述缺陷检测模型。
  8. 根据权利要求1-7任一所述的方法，其特征在于，所述目标区域检测模型使用多自由度四边形作为检测框，所述多自由度四边形的各个边与所述待识别图像的各个边的角度为0度至180度中的任一数值。
  9. 一种工业缺陷识别系统，其特征在于，所述工业缺陷识别系统包括：
    目标区域检测模块，用于获取待识别图像，所述待识别图像为反映工业产品的图像；根据目标区域检测模型从所述待识别图像中提取至少一个目标区域，所述至少一个目标区域为可能存在工业缺陷的区域；
    缺陷粗检测模块，用于根据所述至少一个目标区域和缺陷检测模型，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域，和所述缺陷粗选区域包含的工业缺陷的类型；
    缺陷精细识别模块，用于确定所述缺陷粗选区域包含的工业缺陷的尺寸和位置。
  10. 根据权利要求9所述的系统，其特征在于，所述缺陷粗检测模块用于：
    将所述至少一个目标区域作为所述缺陷检测模型的输入，通过所述缺陷检测模型进行推理，获得所述至少一个目标区域中包含有工业缺陷的目标区域的缺陷粗选区域在目标区域中的位置，和所述缺陷粗选区域包含的工业缺陷的缺陷类型，其中，所述缺陷检测模型为利用缺陷训练样本训练完成的人工智能AI模型；
    根据所述缺陷粗选区域在所述目标区域中的位置，从所述目标区域中提取所述缺陷粗选区域。
  11. 根据权利要求9或10所述的系统，其特征在于，所述缺陷精细识别模块用于：
    根据所述缺陷粗选区域内的灰度分布，确定所述缺陷粗选区域包含的工业缺陷在所述缺陷粗选区域内的位置和尺寸；
    根据所述缺陷粗选区域在所述目标区域中的位置、所述目标区域在所述待识别图像中的位置以及所述缺陷粗选区域包含的工业缺陷在所述缺陷粗选区域内的位置，确定所述缺陷粗选区域包含的工业缺陷在所述待识别图像中的位置。
  12. 根据权利要求9-11任一所述的系统，其特征在于，所述系统还包括报告生成模块，所述报告生成模块用于：
    生成检测报告，所述检测报告包括在所述待识别图像中检测到的工业缺陷的缺陷类型、 尺寸和位置；
    向用户提供所述检测报告。
  13. 根据权利要求9-12任一所述的系统，其特征在于，所述工业缺陷为焊缝缺陷，所述目标区域为焊缝所在的区域。
  14. 根据权利要求9-13任一所述的系统，其特征在于，所述系统还用于：
    获取多个工业训练样本，每个工业训练样本均对应有目标区域标注信息，所述目标区域标注信息包括相应工业训练样本中标注的多自由度四边形的位置信息和用于指示相应多自由度四边形为目标区域的标注类别信息；
    通过所述多个工业训练样本和每个工业训练样本的目标区域标注信息对目标区域检测网络进行训练，得到所述目标区域检测模型。
  15. 根据权利要求9-14任一所述的系统，其特征在于，所述系统还用于：
    获取多个缺陷训练样本；
    通过所述目标区域检测模型从所述多个缺陷训练样本中提取多个目标区域；
    获取所述多个目标区域中包含有工业缺陷的目标区域的缺陷标注信息，所述缺陷标注信息用于指示相应目标区域内的工业缺陷的缺陷类型和位置；
    通过包含有工业缺陷的目标区域和相应目标区域的缺陷标注信息对缺陷检测网络进行训练，得到所述缺陷检测模型。
  16. 根据权利要求9-15任一所述的系统，其特征在于，所述目标区域检测模型使用多自由度四边形作为检测框，所述多自由度四边形的各个边与所述待识别图像的各个边的角度为0度至180度中的任一数值。
  17. 一种计算设备，其特征在于，所述计算设备包括处理器和存储器，所述存储器用于存储一组计算机指令，当所述处理器执行所述一组计算机指令时，所述计算设备执行上述权利要求1至8任一项所述的方法。
  18. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序代码，当所述计算机程序代码被计算设备执行时，所述计算设备执行权利要求1至8中任一项所述的方法。
  19. 一种计算机程序产品，其特征在于，当所述计算机程序产品在计算设备上运行时，使得所述计算设备执行权利要求1至8中任一项所述的方法。
PCT/CN2021/124254 2020-11-13 2021-10-16 工业缺陷识别方法、系统、计算设备及存储介质 WO2022100366A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21890893.7A EP4231229A4 (en) 2020-11-13 2021-10-16 METHOD AND SYSTEM FOR DETECTING INDUSTRIAL DEFECTS, COMPUTER DEVICE AND STORAGE MEDIUM
US18/316,757 US20230281784A1 (en) 2020-11-13 2023-05-12 Industrial Defect Recognition Method and System, Computing Device, and Storage Medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011268302.9A CN114565548A (zh) 2020-11-13 2020-11-13 工业缺陷识别方法、系统、计算设备及存储介质
CN202011268302.9 2020-11-13

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/316,757 Continuation US20230281784A1 (en) 2020-11-13 2023-05-12 Industrial Defect Recognition Method and System, Computing Device, and Storage Medium

Publications (1)

Publication Number Publication Date
WO2022100366A1 true WO2022100366A1 (zh) 2022-05-19

Family

ID=81600823

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/124254 WO2022100366A1 (zh) 2020-11-13 2021-10-16 工业缺陷识别方法、系统、计算设备及存储介质

Country Status (4)

Country Link
US (1) US20230281784A1 (zh)
EP (1) EP4231229A4 (zh)
CN (1) CN114565548A (zh)
WO (1) WO2022100366A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953404A (zh) * 2023-03-14 2023-04-11 常州微亿智造科技有限公司 应用于工业质检的切图方法及装置
CN116452791A (zh) * 2023-03-27 2023-07-18 广州市斯睿特智能科技有限公司 多相机点位的缺陷区域定位方法、系统、装置及存储介质
CN116468725A (zh) * 2023-06-13 2023-07-21 北京航空航天大学杭州创新研究院 基于预训练模型的工业缺陷检测方法、装置及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116758040B (zh) * 2023-06-26 2024-02-20 上海感图网络科技有限公司 镀铜板表面褶皱缺陷检测方法、装置、设备及存储介质
CN117078665B (zh) * 2023-10-13 2024-04-09 东声(苏州)智能科技有限公司 产品表面缺陷检测方法、装置、存储介质和电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105190724A (zh) * 2013-03-11 2015-12-23 林肯环球股份有限公司 在虚拟现实环境中提供增强的教学和训练的系统和方法
CN109087294A (zh) * 2018-07-31 2018-12-25 深圳辰视智能科技有限公司 一种产品缺陷检测方法、系统及计算机可读存储介质
CN110389127A (zh) * 2019-07-03 2019-10-29 浙江大学 一种金属陶瓷零件识别及表面缺陷检测系统和方法
US20200134800A1 (en) * 2018-10-29 2020-04-30 International Business Machines Corporation Precision defect detection based on image difference with respect to templates

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10142165A (ja) * 1996-11-14 1998-05-29 Kubota Corp 疵検出装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105190724A (zh) * 2013-03-11 2015-12-23 林肯环球股份有限公司 在虚拟现实环境中提供增强的教学和训练的系统和方法
CN109087294A (zh) * 2018-07-31 2018-12-25 深圳辰视智能科技有限公司 一种产品缺陷检测方法、系统及计算机可读存储介质
US20200134800A1 (en) * 2018-10-29 2020-04-30 International Business Machines Corporation Precision defect detection based on image difference with respect to templates
CN110389127A (zh) * 2019-07-03 2019-10-29 浙江大学 一种金属陶瓷零件识别及表面缺陷检测系统和方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4231229A4

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953404A (zh) * 2023-03-14 2023-04-11 常州微亿智造科技有限公司 应用于工业质检的切图方法及装置
CN116452791A (zh) * 2023-03-27 2023-07-18 广州市斯睿特智能科技有限公司 多相机点位的缺陷区域定位方法、系统、装置及存储介质
CN116452791B (zh) * 2023-03-27 2024-03-22 广州市斯睿特智能科技有限公司 多相机点位的缺陷区域定位方法、系统、装置及存储介质
CN116468725A (zh) * 2023-06-13 2023-07-21 北京航空航天大学杭州创新研究院 基于预训练模型的工业缺陷检测方法、装置及存储介质
CN116468725B (zh) * 2023-06-13 2023-09-05 北京航空航天大学杭州创新研究院 基于预训练模型的工业缺陷检测方法、装置及存储介质

Also Published As

Publication number Publication date
US20230281784A1 (en) 2023-09-07
CN114565548A (zh) 2022-05-31
EP4231229A4 (en) 2024-04-10
EP4231229A1 (en) 2023-08-23

Similar Documents

Publication Publication Date Title
WO2022100366A1 (zh) 工业缺陷识别方法、系统、计算设备及存储介质
US20220036533A1 (en) Image defect detection method and apparatus, electronic device, storage medium and product
KR20190063839A (ko) 제조 공정에서 딥러닝을 활용한 머신 비전 기반 품질검사 방법 및 시스템
WO2020156409A1 (zh) 数据处理方法、瑕疵的检测方法、计算设备及存储介质
WO2022007451A1 (zh) 目标检测方法、装置、计算机可读介质及电子设备
CN111680750B (zh) 图像识别方法、装置和设备
CN109255767B (zh) 图像处理方法和装置
CN107341804B (zh) 点云数据中平面的确定方法及装置、图像叠加方法及设备
CN115908988B (zh) 一种缺陷检测模型生成方法、装置、设备以及存储介质
CN115880536A (zh) 数据处理方法、训练方法、目标对象检测方法及装置
KR20200001984A (ko) 계통적 결함들에 기반한 반도체 웨이퍼의 유도식 검사
Shamsollahi et al. A timely object recognition method for construction using the mask R-CNN architecture
CN113793349A (zh) 目标检测方法及装置、计算机可读存储介质、电子设备
CN111079752A (zh) 识别红外图像中的断路器的方法、装置及可读存储介质
CN112668365A (zh) 一种物料入库识别方法、装置、设备及存储介质
JP4262297B2 (ja) クラスタ生成装置、欠陥分類装置、クラスタ生成方法およびプログラム
CN106331696B (zh) 丢帧检测方法和装置
CN116109627B (zh) 基于迁移学习和小样本学习缺陷检测方法、装置及介质
CN117575985B (zh) 一种汽车零部件铸造监管方法、装置、设备及介质
CN113763467B (zh) 图像处理方法、装置、计算设备及介质
CN117215164B (zh) 一种光刻仿真方法及装置
CN117974635B (zh) 电缆通道检测方法、装置、电子设备与计算机可读介质
JP2005083843A (ja) 欠陥検出装置、クラスタ生成装置、欠陥分類装置、欠陥検出方法、クラスタ生成方法およびプログラム
CN115375987B (zh) 一种数据标注方法、装置、电子设备及存储介质
CN117934809A (zh) 一种多模型级联的目标检测方法、系统、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21890893

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021890893

Country of ref document: EP

Effective date: 20230517

NENP Non-entry into the national phase

Ref country code: DE