WO2024082925A1 - 一种表面缺陷数据增强方法、装置、电子设备及存储介质 - Google Patents
一种表面缺陷数据增强方法、装置、电子设备及存储介质 Download PDFInfo
- Publication number
- WO2024082925A1 WO2024082925A1 PCT/CN2023/121002 CN2023121002W WO2024082925A1 WO 2024082925 A1 WO2024082925 A1 WO 2024082925A1 CN 2023121002 W CN2023121002 W CN 2023121002W WO 2024082925 A1 WO2024082925 A1 WO 2024082925A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- surface defect
- data
- defect data
- enhanced
- enhancement
- Prior art date
Links
- 230000007547 defect Effects 0.000 title claims abstract description 331
- 238000000034 method Methods 0.000 title claims abstract description 130
- 230000009466 transformation Effects 0.000 claims description 44
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 6
- 238000007500 overflow downdraw method Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 36
- 230000000694 effects Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 9
- 230000004927 fusion Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000002708 enhancing effect Effects 0.000 description 7
- 238000012549 training Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000000717 retained effect Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 2
- 101100233916 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR5 gene Proteins 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000010287 polarization Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 1
- 101000827703 Homo sapiens Polyphosphoinositide phosphatase Proteins 0.000 description 1
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 1
- 102100023591 Polyphosphoinositide phosphatase Human genes 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration using non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Definitions
- the present disclosure relates to the technical field of image data processing, and in particular to a surface defect data enhancement method, device, electronic device and storage medium.
- machine vision surface defect detection is one of the more mature applications of machine learning in industry, including defect detection methods based on visual features and defect detection methods based on deep learning.
- defect detection methods based on visual features
- defect detection methods based on deep learning.
- data enhancement steps need to be adopted.
- the disclosed embodiment provides a surface defect data enhancement method for realizing data enhancement at a specific required position during surface defect detection.
- Some embodiments of the present disclosure provide a surface defect data enhancement method, which may include: obtaining surface defect data to be enhanced; obtaining information about a part to be enhanced; dividing the surface defect data according to the information about the part to be enhanced; and performing data enhancement on the divided surface defect data.
- a surface defect data enhancement method may include: obtaining surface defect data to be enhanced; obtaining information about a part to be enhanced; dividing the surface defect data according to the information about the part to be enhanced; and performing data enhancement on the divided surface defect data.
- the surface defect data enhancement method may further include: performing initial data enhancement on the surface defect data to obtain surface defect data after initial data enhancement; dividing the surface defect data according to the information of the part to be enhanced may include: dividing the surface defect data after initial data enhancement according to the information of the part to be enhanced.
- performing initial data enhancement on the surface defect data not only the data is enhanced in quantity, but also the effect of dividing the surface defect data is effectively guaranteed.
- the initial data enhancement method may include: data enhancement based on geometric transformation.
- the initial data enhancement of the surface defect data is achieved through a variety of methods. Any one or more of geometric transformation, color transformation and pixel transformation can be used to achieve the initial enhancement of the sample, thereby providing data support for the subsequent segmentation of the surface defect data and improving the data segmentation effect.
- the data enhancement methods for the surface defect data after division may include: at least one of data enhancement based on geometric transformation, data enhancement based on color transformation, and data enhancement based on pixel transformation.
- data enhancement of surface defect data is achieved through a variety of methods. Any one or more of geometric transformation, color transformation and pixel transformation can be used to achieve secondary enhancement of samples.
- secondary enhancement is performed on the surface defect data after division;
- the purpose of initial enhancement is to expand the global sample to improve the regional division effect;
- the purpose of secondary enhancement is to expand the divided samples again; the combination of initial enhancement and secondary enhancement can effectively improve the enhancement effect of each sample data after division, so that surface defect detection of the part to be enhanced can be achieved in a targeted manner.
- the surface defect data to be enhanced may include: gear surface defect data to be enhanced; the said dividing the surface defect data according to the information of the part to be enhanced may include: dividing the gear surface defect data into tooth surface defect data and tooth bottom defect data according to the information of the part to be enhanced.
- the gear surface defect data is divided into tooth surface defect data and tooth bottom defect data according to the information of the part to be enhanced, so as to realize data enhancement for a specific position of the gear; then, the tooth surface defect data and the tooth bottom defect data are respectively enhanced, and then the surface defect detection network is trained by the tooth surface defect enhancement data and the tooth bottom defect enhancement data, so that the subsequent targeted defect detection of different gear parts can be realized when the gear surface is subjected to defect detection.
- the gear surface defect data is divided into tooth surface defect data and tooth bottom defect data by using a feature point matching method, thereby realizing data enhancement for a specific position of the gear; then, data enhancement is performed on the tooth surface defect data and the tooth bottom defect data respectively, and then the surface defect detection network is trained by using the tooth surface defect enhancement data and the tooth bottom defect enhancement data, so that targeted defect detection of different gear parts can be realized when defect detection is performed on the gear surface later.
- the feature point matching method may include two steps: feature point extraction and feature point matching, wherein the feature point extraction may use SIFT algorithm, SURF algorithm, FAST algorithm, BRIEF algorithm or ORB algorithm to extract feature points, and the feature point matching method may use a matching method based on feature point similarity.
- data enhancement is performed on the divided surface defect data respectively, which may include: respectively acquiring low-frequency components of the divided surface defect data; respectively enhancing the divided surface defect data according to the low-frequency components; Perform data enhancement.
- the low-frequency components of the digital image represent the grayscale value and brightness of the image
- the high-frequency components represent the edge, noise and detail information of the image
- the image detail information is effectively retained by performing data enhancement on the low-frequency components and retaining the high-frequency components.
- the quality of the expanded data is also guaranteed, and the data enhancement effect of the surface defect data is effectively improved.
- Using the data after the above enhancement method to train the surface defect detection network can obtain better training and learning effects, and further improve the detection accuracy of the surface defect detection network.
- the method of obtaining the low-frequency component may be: using Fourier transform to obtain the high-frequency component and the low-frequency component of the divided surface defect data, and performing data enhancement on the divided surface defect data according to the low-frequency component may include: performing data enhancement on the low-frequency component of the divided surface defect data by Poisson fusion.
- Some other embodiments of the present disclosure also provide a surface defect data enhancement device, which may include: a module for acquiring surface defect data to be enhanced, the module for acquiring surface defect data to be enhanced being configured to acquire surface defect data to be enhanced; a module for acquiring information about a part to be enhanced, the module for acquiring information about the part to be enhanced being configured to acquire information about the part to be enhanced; a data division module, the module for dividing the surface defect data according to the information about the part to be enhanced; and a data enhancement module, the module for performing data enhancement on the divided surface defect data respectively.
- the surface defect data enhancement device may further include:
- the primary data enhancement module is configured to perform primary data enhancement on the surface defect data to be enhanced acquired by the surface defect data acquisition module to be enhanced; the data division module is also configured to divide the surface defect data after the primary data enhancement according to the information of the part to be enhanced.
- the data enhancement method of the initial data enhancement module may include: at least one of data enhancement based on geometric transformation, data enhancement based on color transformation, and data enhancement based on pixel transformation.
- the data enhancement module may perform data enhancement on the divided surface defect data in a manner that includes: at least one of data enhancement based on geometric transformation, data enhancement based on color transformation, and data enhancement based on pixel transformation.
- the surface defect data to be enhanced acquired by the surface defect data acquisition module to be enhanced may include: gear surface defect data to be enhanced.
- the data division module specifically divides the gear surface defect data into tooth surface defect data and tooth bottom defect data according to the information of the part to be enhanced.
- the part information to be enhanced acquired by the part information acquisition module may include: a tooth surface data template and a tooth bottom data template.
- the data division module specifically divides the gear surface defect data into tooth surface defect data and tooth bottom defect data using a feature point matching method for the tooth surface data template and the tooth bottom data template.
- the data enhancement module may include: a low-frequency component acquisition unit, the low-frequency component
- the data acquisition unit is configured to acquire the low-frequency components of the divided surface defect data
- the data enhancement unit is configured to perform data enhancement on the divided surface defect data according to the low-frequency components.
- Still other embodiments of the present disclosure provide an electronic device, which may include: a processor, a memory, and a bus, wherein the processor and the memory communicate with each other via the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method described above.
- Some further embodiments of the present disclosure provide a computer-readable storage medium having a computer program stored thereon.
- the computer program is executed by a processor, the method described above is executed.
- FIG1 is a schematic flow chart of a surface defect data enhancement method performed by an electronic device (eg, a server) according to an embodiment of the present disclosure
- FIG2 is a schematic diagram of a flow chart of a surface defect data enhancement method provided by an embodiment of the present disclosure applied to a gear surface defect data enhancement scenario;
- FIG3 is a flow chart of a method for performing data enhancement on divided surface defect data respectively provided by an embodiment of the present disclosure
- FIG4 is a schematic structural diagram of a surface defect data enhancement device provided by an embodiment of the present disclosure.
- FIG5 is a schematic diagram of a flow chart of a surface defect data enhancement method provided by an embodiment of the present disclosure in a gear surface defect data enhancement scenario
- FIG. 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.
- the term "and/or" is only a description of the association relationship of associated objects, indicating that three relationships may exist.
- a and/or B may represent: A exists alone, A and B exist at the same time, and B exists alone.
- the character "/" in this article generally indicates that the associated objects before and after are in an "or" relationship.
- multiple refers to more than two (including two).
- multiple groups refers to more than two groups (including two groups), and “multiple pieces” refers to more than two pieces (including two pieces).
- the surface defect data enhancement method provided in the embodiment of the present disclosure can be executed by an electronic device, where the electronic device refers to a device terminal or server with the function of executing a computer program, such as a smart phone, a personal computer, a tablet computer, a personal digital assistant or a mobile Internet device.
- a server refers to a device that provides computing services through a network, such as an x86 server and a non-x86 server, and a non-x86 server includes a mainframe, a minicomputer and a UNIX server.
- FIG. 1 a flow chart of a surface defect data enhancement method performed by an electronic device (e.g., a server) provided in an embodiment of the present disclosure; the method can be applied to an electronic device (e.g., a server), and the main idea of the method is: by obtaining information about the part to be enhanced, the surface defect data is divided, and the divided surface defect data is enhanced respectively, thereby realizing data expansion for specific required locations.
- the implementation of the above surface defect data enhancement method may include:
- Step S110 Acquire surface defect data to be enhanced
- Step S120 obtaining information of the part to be enhanced
- Step S130 dividing the surface defect data according to the information of the part to be enhanced
- Step S140 performing data enhancement on the divided surface defect data.
- the surface defect data to be enhanced may be a surface defect image to be enhanced, which may be an infrared image, a remote sensing image, a visible light image, a polarization image, etc., as long as surface defect detection can be realized.
- the part information to be enhanced may include at least the template data of the part to be enhanced, and may also include the name of the part to be enhanced.
- the part information to be enhanced may be uploaded by a user, or retrieved from a related open source database, or of course, a crawler may be used to crawl related template data.
- step S130 the data in the surface defect data is compared with the template data according to the template data of the part to be enhanced. Parts whose similarity meets a preset threshold are considered to match the data template, thereby achieving the division of the surface defect data.
- the data enhancement method for the divided surface defect data may be a basic data enhancement method such as size transformation and color transformation, or a data enhancement method based on deep learning, such as a data enhancement method based on a generative adversarial network.
- step S140 several types of divided surface defect data can be selected from the divided surface defect data for enhancement, or one type of data can be enhanced.
- the gear surface defect data is first divided into tooth surface defect data and tooth bottom defect data. Then, according to the specific application scenario, you can choose to enhance only the tooth surface defect data or the tooth bottom defect data, or you can enhance both the tooth surface defect data and the tooth bottom defect data.
- step S130 before dividing the surface defect data according to the information of the part to be enhanced in step S130, it may also include: performing initial data enhancement on the surface defect data to obtain surface defect data after the initial data enhancement.
- This implementation may include:
- Step S150 performing initial data enhancement on the surface defect data to obtain surface defect data after the initial data enhancement.
- Step S130 specifically includes: dividing the surface defect data after the initial data enhancement according to the information of the part to be enhanced.
- the purpose of the initial data enhancement of the surface defect data is to achieve effective division of the surface defect data and then to enhance the data after division.
- the method of initial enhancement combined with secondary enhancement can effectively improve the enhancement effect of each sample data after division.
- the initial data enhancement method in step S150 may include: at least one of data enhancement based on geometric changes, data enhancement based on color changes, and data enhancement based on pixel transformation.
- the initial data enhancement of surface defect data can be achieved through a variety of methods. Any one or more of geometric transformation, color transformation and pixel transformation can be used to achieve the initial enhancement of the sample, thereby providing data support for the subsequent division of surface defect data and improving the data division effect.
- the data enhancement method in step S140 may include: at least one of data enhancement based on geometric changes, data enhancement based on color changes, and data enhancement based on pixel transformation.
- the disclosed embodiments implement data enhancement of surface defect data through a variety of methods, and may use any one or more of geometric transformation, color transformation, and pixel transformation to implement secondary enhancement of samples.
- secondary enhancement is performed on the surface defect data after division; the purpose of initial enhancement is to expand the global sample to improve the regional division effect; the purpose of secondary enhancement is to expand the sample after division; the combination of initial enhancement and secondary enhancement can effectively improve the enhancement effect of each sample data after division.
- the first method is data enhancement based on geometric changes. It performs spatial geometric transformation on the data set to achieve data enhancement, mainly including flipping, rotation, cropping, scaling, shifting and edge filling.
- the second method is data enhancement based on color changes, which adjusts the color space of digital images to achieve data enhancement, mainly including: brightness adjustment and chromaticity adjustment on the color channel, color space conversion, etc.
- the third method is data enhancement based on pixel transformation, which realizes data enhancement based on the pixels of digital images, mainly including: noise, blur, image fusion, information deletion, etc.
- data enhancement methods based on image fusion can include:
- SMOTE method This method maps the extracted image features to the feature space. After determining the sampling rate, it selects several adjacent samples, randomly selects a line from them, and randomly selects a point on the line as a new sample point. Repeat this process until the samples are balanced.
- MIXUP method This method randomly extracts two pieces of data from the dataset, and then performs linear weighted summation on the pixel values of the extracted image data in accordance with the fusion ratio of the Beat distribution. At the same time, the One-hot vector labels corresponding to the samples are also weighted summed accordingly. The loss of the generated new samples and the weighted summed labels is predicted, and reverse derivation is performed to update the parameters. At the same time, batch data is extracted and randomly scattered before weighted summation.
- CUTMIX method This method fills randomly selected areas with patch areas of other images.
- Sample Pairing method This method first randomly selects two images from the data set, then takes the average value of the pixels, and finally superimposes them to synthesize a new sample.
- the present disclosure proposes the following image fusion method:
- Poisson fusion This method uses the gradient field of the background image as a guidance field to calculate the fused gradient field. It can reconstruct the image pixels in the synthesis area using the interpolation method based on the gradient information of the original image and the boundary information of the target defect image.
- the surface defect data to be enhanced may include: gear surface defect data to be enhanced; and the gear surface defect data is divided into tooth surface defect data and tooth bottom defect data according to the information of the part to be enhanced.
- this implementation may include:
- Step S210 Acquire surface defect data of the gear to be enhanced
- Step S220 obtaining information of the part to be enhanced
- Step S230 dividing the gear surface defect data into tooth surface defect data and tooth bottom defect data according to the information of the part to be enhanced;
- Step S240 performing data enhancement on the tooth surface defect data and the tooth bottom defect data respectively.
- the gear surface defect data to be enhanced in step S210 can be an image of the gear surface defect, which can be an infrared image, a remote sensing image, a visible light image, a polarization image, etc., as long as it can realize surface defect detection.
- the information of the part to be enhanced in step S220 includes the template data of the part to be enhanced, which can be uploaded by the user or retrieved from the relevant open source database.
- Step S230 compares the data in the surface defect data with the template data according to the template data of the part to be enhanced of the gear. The parts whose similarity meets the preset threshold can be considered to match the data template, thereby realizing the division of the gear surface defect data.
- Step S240 can use basic data enhancement methods such as size transformation and color transformation to enhance the divided surface defect data respectively, or can use data enhancement methods based on deep learning.
- the data enhancement method based on deep learning is, for example, a data enhancement method based on a generative adversarial network.
- the gear surface defect data is divided into tooth surface defect data and tooth bottom defect data through the information of the part to be enhanced, thereby realizing data enhancement for the specific position of the gear; then, the tooth surface defect data and the tooth bottom defect data are enhanced respectively, and then the surface defect detection network is trained through the tooth surface defect enhancement data and the tooth bottom defect enhancement data, so that targeted defect detection of different gear parts can be realized when defect detection is performed on the gear surface subsequently.
- the tooth part information to be enhanced in step S220 may include: a tooth surface data template and a tooth bottom data template.
- the gear surface defect data is divided into tooth surface defect data and tooth bottom defect data according to the part information to be enhanced, which may include: for the tooth surface data template and the tooth bottom data template, using a feature point matching method to divide the gear surface defect data into tooth surface defect data and tooth bottom defect data.
- This implementation may include:
- Step S210 Acquire surface defect data of the gear to be enhanced
- Step S220 obtaining information of the part to be enhanced, including a tooth surface data template and a tooth bottom data template;
- Step S230 for the tooth surface data template and the tooth bottom data template, the gear surface defect data is divided into tooth surface defect data and tooth bottom defect data by using a feature point matching method;
- Step S240 performing data enhancement on the tooth surface defect data and the tooth bottom defect data respectively.
- the feature point matching method is used to divide the gear surface defect data into tooth surface defect data and tooth bottom defect data, thereby realizing data enhancement for specific positions of the gear; then, the tooth surface defect data and the tooth bottom defect data are enhanced respectively, and then the surface defect detection network is trained with the tooth surface defect enhancement data and the tooth bottom defect enhancement data, so that targeted defect detection of different gear parts can be achieved when defect detection is performed on the gear surface in the subsequent process.
- the feature point matching method in step S230 may include two steps: feature point extraction and feature point matching, wherein feature point extraction may use SIFT algorithm, SURF algorithm, FAST algorithm, BRIEF algorithm or ORB algorithm to extract feature points, and feature point matching may use a matching method based on feature point similarity, wherein different distances may be selected as a measure of similarity according to different feature descriptors, and if it is a floating point type descriptor, its Euclidean distance may be used; for binary descriptors, its Hamming distance may be used.
- the data enhancement of the divided surface defect data in step S140 may include: respectively obtaining low-frequency components of the divided surface defect data; and respectively enhancing the divided surface defect data according to the low-frequency components.
- this implementation may include:
- Step S140-1 respectively obtaining low-frequency components of the divided surface defect data
- Step S140 - 2 performing data enhancement on the divided surface defect data according to the low-frequency components.
- the image details are effectively retained.
- the quality of the expanded data is also guaranteed, and the data enhancement effect of the surface defect data is effectively improved.
- Using the data after the above enhancement method to train the surface defect detection network can obtain better training and learning effects, and further improve the detection accuracy of the surface defect detection network.
- the method of obtaining the low-frequency component in step S140-1 can be: using Fourier transform to obtain the high-frequency component and low-frequency component of the image, wherein the low-frequency component represents the gray value and brightness of the image, and the high-frequency component represents the edge, noise and detail information of the image. Therefore, step S140-2 performs data enhancement on the divided surface defect data according to the low-frequency component.
- FIG. 4 shows a schematic diagram of the structure of a surface defect data enhancement device provided by an embodiment of the present disclosure.
- an embodiment of the present disclosure provides a surface defect data enhancement device 400, and the surface defect data enhancement device 400 may include:
- a module 410 for acquiring data of surface defects to be enhanced wherein the module 410 may be configured to acquire data of surface defects to be enhanced;
- a module 420 for acquiring information about a part to be enhanced wherein the module 420 may be configured to acquire information about a part to be enhanced;
- a data division module 430 wherein the data division module 430 may be configured to divide the surface defect data according to the information of the part to be enhanced;
- the data enhancement module 440 may be configured to perform data enhancement on the divided surface defect data.
- the surface defect data enhancement device 400 may further include:
- a primary data enhancement module 450 wherein the primary data enhancement module 450 may be configured to perform primary data enhancement on the surface defect data to be enhanced acquired by the surface defect data to be enhanced acquisition module 410;
- the data division module 430 may be configured to divide the surface defect data after the initial data enhancement according to the information of the part to be enhanced.
- the data enhancement method of the initial data enhancement module 450 may include: at least one of data enhancement based on geometric transformation, data enhancement based on color transformation, and data enhancement based on pixel transformation.
- the data enhancement module 440 may perform data enhancement on the divided surface defect data in a manner that includes: at least one of data enhancement based on geometric transformation, data enhancement based on color transformation, and data enhancement based on pixel transformation.
- the surface defect data to be enhanced acquired by the surface defect data acquisition module 410 may include gear surface defect data to be enhanced.
- the data division module 430 specifically divides the gear surface defect data into tooth surface defect data and tooth bottom defect data according to the information of the part to be enhanced.
- the part information to be enhanced acquired by the part information acquisition module 420 may include: a tooth surface data template and a tooth bottom data template.
- the data division module 430 specifically divides the gear surface defect data into tooth surface defect data and tooth bottom defect data by using a feature point matching method for the tooth surface data template and the tooth bottom data template.
- the data enhancement module 440 may include:
- a low frequency component acquisition unit wherein the low frequency component acquisition unit can be configured to acquire the surface defects after segmentation.
- the low-frequency components of the data can be configured to acquire the surface defects after segmentation.
- a data enhancement unit wherein the data enhancement unit may be configured to perform data enhancement on the divided surface defect data according to the low-frequency components.
- the device corresponds to the above-mentioned surface defect data enhancement method embodiment and can perform the various steps involved in the above-mentioned method embodiment.
- the specific functions of the device can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here.
- the device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.
- OS operating system
- FIG. 5 a schematic flow diagram of a surface defect data enhancement method provided by an embodiment of the present disclosure in a gear surface defect data enhancement application scenario; an embodiment of the present disclosure provides a gear surface defect data enhancement method, which includes:
- Step S510 Acquire surface defect data of the gear to be enhanced
- Step S520 performing initial data enhancement on the gear surface defect data to be enhanced, and obtaining the gear surface defect data after the initial data enhancement;
- the data is expanded by translation, rotation, random cropping and other methods.
- the scale of translation varies from 0 to 30%
- the scale of rotation angle varies from 0 to 180°
- the proportion of random cropping varies from 0 to 40%.
- Step S530 obtaining information of the part to be enhanced, including a tooth surface data template and a tooth bottom image template;
- Step S540 Based on the tooth surface data template and the tooth bottom image template, the SURF algorithm is used to extract feature points, and the Euclidean distance is used as the similarity measurement to match the feature points, screen out the tooth surface and tooth bottom areas, and divide the gear surface defect data after the initial data enhancement into tooth surface defect data and tooth bottom defect data;
- the SURF algorithm used in the embodiment of the present disclosure extracts feature points through the Hessian matrix.
- a corresponding Hessian matrix for each pixel point (x, y) in the image and its formula is as follows:
- the Hessian matrix of the filtered image I(x,y) is as follows:
- Lxx (x,y, ⁇ ), Lxy (x,y, ⁇ ), Lxy (x,y, ⁇ ) and Lyy (x,y, ⁇ ) represent the convolution of I(x,y) and the second-order derivative of the Gaussian function, respectively.
- the located feature points are respectively calculated with the data template for Euclidean distance. If the value of the Euclidean distance is less than the set threshold, The value is a successful match. After matching all digital images in the surface defect data, the tooth surface and tooth bottom areas can be screened out.
- Step S550 performing data enhancement on the tooth surface defect data and the tooth bottom defect data respectively;
- Step S550-1 obtaining high-frequency components and low-frequency components of defect data through Fourier transform
- Step S550 - 2 Perform data enhancement on the low-frequency components of the defect data by using Poisson fusion.
- Poisson fusion The specific process of Poisson fusion is as follows: The core idea of Poisson fusion is not to directly superimpose the two images to be fused, but to let the target image grow a new image in the fusion part according to the gradient field of the source image.
- S is a two-dimensional real number set
- the closed subset of S is ⁇ , and the boundary of S is Closed subset
- f * is the function of the set S- ⁇ part (if it is an image, it refers to the pixel value of all pixels)
- f is the function of the set ⁇
- v is the vector field of the set ⁇
- the method for solving the minimum gradient is as follows:
- the gradient is smooth, that is:
- the low-frequency components of the image are interpolated by solving the Poisson equation.
- the Poisson fusion method is used to enhance the low-frequency components and fully retain the high-frequency information of the image, that is, the characteristic texture information of the target defect image is fully retained, so that the enhanced data has higher training accuracy in the neural network and better learning effect.
- An electronic device 600 may include a central processing unit CPU 601, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory ROM 602 or computer program instructions loaded from a storage unit into a random access memory RAM 603. Various programs and data required for device operation may also be stored in the RAM 603.
- the CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604.
- An I/O interface 605 is also connected to the bus 604.
- the I/O interface 605 may include: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc.
- the communication unit 609 allows the device to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.
- CPU 601 performs the various methods and processes described above, such as method steps S110 to S140 in the embodiment of the present disclosure.
- method steps S110 to S140 in the embodiment of the present disclosure may be implemented as a computer software program, which is tangibly contained in a non-transitory computer-readable storage medium, such as a storage unit.
- part or all of the computer program may be loaded and/or installed on the device via ROM 602 and/or communication unit 609.
- the computer program is loaded into RAM 603 and executed by CPU 601, one or more steps of method steps S110 to S140 in the embodiment of the present disclosure described above may be executed.
- CPU 601 may be configured to execute method steps S110 to S140 in the embodiment of the present disclosure by any other appropriate means (e.g., by means of firmware).
- exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
- FPGAs field programmable gate arrays
- ASICs application specific integrated circuits
- ASSPs application specific standard products
- SOCs systems on chip
- CPLDs complex programmable logic devices
- the program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions/operations specified in the flow chart and/or block diagram.
- the program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
- a non-transitory computer-readable storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
- a non-transitory computer-readable storage medium may be a machine-readable signal medium or a machine-readable storage medium.
- a non-transitory computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
- non-transitory computer-readable storage media would include electrical connections based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), Read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
- RAM random access memory
- ROM Read-only memory
- EPROM or flash memory erasable programmable read-only memory
- CD-ROM compact disk read-only memory
- magnetic storage device or any suitable combination of the above.
- the disclosed devices and methods can be implemented in other ways.
- the device embodiments described above are merely schematic.
- the division of the units is only a logical function division of the above method. There may be other division methods in actual implementation.
- multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
- Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
- the functional modules in the various embodiments of the present disclosure may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
- the present disclosure provides a surface defect data enhancement method, device, electronic device and storage medium, wherein the surface defect data enhancement method comprises: obtaining surface defect data to be enhanced; obtaining information of a part to be enhanced; dividing the surface defect data according to the information of the part to be enhanced; and performing data enhancement on the divided surface defect data.
- the surface defect data enhancement method comprises: obtaining surface defect data to be enhanced; obtaining information of a part to be enhanced; dividing the surface defect data according to the information of the part to be enhanced; and performing data enhancement on the divided surface defect data.
- the surface defect data enhancement device of the present disclosure is reproducible and can be used in a variety of industrial applications.
- the surface defect data enhancement device of the present disclosure can be used in any application that requires the surface defect data enhancement device to detect surface defects at a specific location.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Manufacturing Of Magnetic Record Carriers (AREA)
Abstract
本公开提供一种表面缺陷数据增强方法、装置、电子设备及存储介质，其中表面缺陷数据增强方法包括：获取待增强的表面缺陷数据；获取待增强部位信息；根据待增强部位信息对表面缺陷数据进行划分；对划分后的表面缺陷数据进行数据增强。通过根据待增强部位信息对表面缺陷数据进行划分，然后对划分后的表面缺陷数据分别进行数据增强，实现了针对特定位置的数据增强，从而可以实现针对特定位置的表面缺陷检测。
Description
相关申请的交叉引用
本公开要求于2022年10月17日提交于中国国家知识产权局的申请号为202211264174.X、名称为“一种表面缺陷数据增强方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
本公开涉及图像数据处理技术领域，具体而言，涉及一种表面缺陷数据增强方法、装置、电子设备及存储介质。
目前，机器视觉表面缺陷检测是机器学习在工业上较成熟的应用之一，包括基于视觉特征的缺陷检测方法和基于深度学习的缺陷检测方法等。在进行表面缺陷检测时，若存在数据量缺乏或数据质量不高的情况，需要采用数据增强的步骤。
相关数据增强方法大都直接采用几何变换、像素变换以及裁剪等基础变换方式，缺陷数据的扩充范围为图片全局，无法在某些特定的需求位置进行数据扩充，导致在表面缺陷检测时无法对特定需求位置进行缺陷检测。
发明内容
本公开实施例提供了一种表面缺陷数据增强方法，用以实现在表面缺陷检测时对特定需求位置进行数据增强。
本公开的一些实施例提供一种表面缺陷数据增强方法，所述表面缺陷数据增强方法可以包括：获取待增强的表面缺陷数据；获取待增强部位信息；根据待增强部位信息对表面缺陷数据进行划分；对划分后的表面缺陷数据进行数据增强。在上述方案的实现过程中，通过根据待增强部位信息对表面缺陷数据进行划分，然后对划分后的表面缺陷数据分别进行数据增强，实现了针对特定位置的数据增强，从而可以实现针对特定位置的表面缺陷检测。
可选地，在本公开实施例中，在所述根据待增强部位信息对表面缺陷数据进行划分前，所述表面缺陷数据增强方法还可以包括：对所述表面缺陷数据进行初次数据增强，获得经过初次数据增强后的表面缺陷数据；所述根据待增强部位信息对表面缺陷数据进行划分，可以包括：根据待增强部位信息对初次数据增强后的表面缺陷数据进行划分。在上述方案的实现过程中，通过对表面缺陷数据进行初次数据增强，不仅在数量上实现了对数据进行增强，还有效保证了对表面缺陷数据进行划分的效果。
可选地，在本公开实施例中，初次数据增强的方式可以包括：基于几何变换的数据增
强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。在上述方案的实现过程中，通过多种方式来实现对表面缺陷数据的初次数据增强，可以采用几何变换、色彩变换和像素变换方式中的任意一种或几种来实现对样本的初次增强，从而为后续对表面缺陷数据进行划分提供数据支撑，改善数据划分效果。
可选地，在本公开实施例中，对划分后的表面缺陷数据分别进行数据增强的增强方式，可以包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。在上述方案的实现过程中，通过多种方式来实现对表面缺陷数据的数据增强，可以采用几何变换、色彩变换和像素变换方式中的任意一种或几种来实现对样本的再次增强，区别于初次增强的全局增强后方式，再次增强是针对划分后的表面缺陷数据进行的；初次增强目的在于对全局样本进行扩充，以改善区域划分效果；再次增强的目的在于对划分后的样本进行再次扩充；采用初次增强配合再次增强的方式，可以有效改善针对划分后各样本数据的增强效果，从而可以有针对性地实现待增强部位的表面缺陷检测。
可选地，在本公开实施例中，待增强的表面缺陷数据可以包括：待增强的齿轮表面缺陷数据；所述根据待增强部位信息对表面缺陷数据进行划分，可以包括：根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。在上述方案的实现过程中，通过待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据，实现了针对齿轮特定位置的数据增强；然后通过对齿面缺陷数据和齿底缺陷数据分别进行数据增强，进而通过齿面缺陷增强数据和齿底缺陷增强数据对表面缺陷检测网络进行训练，从而使得后续在针对齿轮表面进行缺陷检测时实现对不同齿轮部位的针对性缺陷检测。
可选地，在本公开实施例中，待增强部位信息可以包括：齿面数据模板与齿底数据模板；所述根据待增强部位信息对齿轮表面缺陷数据进行划分，可以包括：针对齿面数据模板和齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。在上述方案的实现过程中，基于齿面数据模板与齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据，实现了针对齿轮特定位置的数据增强；然后通过对齿面缺陷数据和齿底缺陷数据分别进行数据增强，进而通过齿面缺陷增强数据和齿底缺陷增强数据对表面缺陷检测网络进行训练，从而使得后续在针对齿轮表面进行缺陷检测时实现对不同齿轮部位的针对性缺陷检测。
可选地，在本公开实施例中，所述特征点匹配法可以包括特征点提取和特征点匹配两个步骤，其中，所述特征点提取可以采用SIFT算法、SURF算法、FAST算法、BRIEF算法或ORB算法来提取特征点，所述特征点匹配方式可以采用基于特征点相似度的匹配方式。
可选地，在本公开实施例中，对划分后的表面缺陷数据分别进行数据增强，可以包括：分别获取划分后的表面缺陷数据的低频分量；根据低频分量对划分后的表面缺陷数据分别
进行数据增强。在上述方案的实现过程中，由于数字图像的低频分量表征图像的灰度值以及亮度等信息，高频分量表征图像的边缘、噪声以及细节信息，通过对低频分量进行数据增强而保留高频分量的方式，使得图像的细节信息得到有效保留，在对数量进行增强的基础上，还保证了扩充数据的质量，有效改善了表面缺陷数据的数据增强效果。使用经过上述增强方式之后的数据对表面缺陷检测网络进行训练，可以获得更好的训练及学习效果，进一步提高表面缺陷检测网络的检测精度。
可选地，在本公开实施例中，所述获取低频分量的方式可以为：采用傅里叶变换获得所述划分后的表面缺陷数据的高频分量和低频分量，根据所述低频分量对所述划分后的表面缺陷数据分别进行数据增强可以包括：对所述划分后的表面缺陷数据的所述低频分量采用泊松融合的方式进行数据增强。
本公开的另一些实施例还提供了一种表面缺陷数据增强装置，所述表面缺陷数据增强装置可以包括：待增强表面缺陷数据获取模块，所述待增强表面缺陷数据获取模块被配置成用于获取待增强的表面缺陷数据；待增强部位信息获取模块，所述待增强部位信息获取模块被配置成用于获取待增强部位信息；数据划分模块，所述数据划分模块被配置成用于根据待增强部位信息对表面缺陷数据进行划分；数据增强模块，所述数据增强模块被配置成用于对划分后的表面缺陷数据分别进行数据增强。
可选地，在本公开实施例中，表面缺陷数据增强装置还可以包括：
初次数据增强模块，所述初次数据增强模块被配置成用于对待增强表面缺陷数据获取模块获取的待增强的表面缺陷数据进行初次数据增强；所述数据划分模块还被配置成根据待增强部位信息对初次数据增强后的表面缺陷数据进行划分。
可选地，在本公开实施例中，初次数据增强模块的数据增强方式可以包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。
可选地，在本公开实施例中，数据增强模块对划分后的表面缺陷数据分别进行数据增强的增强方式可以包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。
可选地，在本公开实施例中，待增强表面缺陷数据获取模块所获取的待增强的表面缺陷数据可以包括：待增强的齿轮表面缺陷数据。数据划分模块具体为：根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。
可选地，在本公开实施例中，待增强部位信息获取模块所获取的待增强部位信息可以包括：齿面数据模板与齿底数据模板。数据划分模块具体为：针对齿面数据模板和齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。
可选地，在本公开实施例中，数据增强模块可以包括：低频分量获取单元，所述低频分
量获取单元被配置成用于获取划分后的表面缺陷数据的低频分量；数据增强单元，所述数据增强单元被配置成用于根据低频分量对划分后的表面缺陷数据分别进行数据增强。
本公开的又一些实施例还提供了一种电子设备，所述电子设备可以包括：处理器、存储器和总线，处理器和存储器通过总线完成相互间的通信；存储器存储有可被处理器执行的程序指令，处理器调用程序指令能够执行如上面描述的方法。
本公开的再一些实施例还提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如上面描述的方法。
本公开的其他特征和优点将在随后的说明书阐述，并且，部分地从说明书中变得显而易见，或者通过实施本公开实施例了解。本公开的目的和其他优点可通过在所写的说明书、权利要求书、以及附图中所特别指出的结构来实现和获得。
为了更清楚地说明本公开实施例的技术方案，下面将对本公开实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1为本公开实施例提供的电子设备(例如服务器)执行的表面缺陷数据增强方法的流程示意图；
图2为本公开实施例提供的表面缺陷数据增强方法应用于齿轮表面缺陷数据增强场景下的流程示意图；
图3为本公开实施例提供的对划分后的表面缺陷数据分别进行数据增强的方法的流程示意图；
图4为本公开实施例提供的表面缺陷数据增强装置的结构示意图；
图5为本公开实施例提供的表面缺陷数据增强方法在齿轮表面缺陷数据增强场景下的流程示意图；
图6为本公开实施例提供的电子设备的结构示意图。
下面将结合附图对本公开技术方案的实施例进行详细的描述。以下实施例仅用于更加清楚地说明本公开的技术方案，因此只作为示例，而不能以此来限制本公开的保护范围。
除非另有定义，本文所使用的所有的技术和科学术语与属于本公开的技术领域的技术人员通常理解的含义相同；本文中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本公开；本公开的说明书和权利要求书及上述附图说明中的术语“包括”和“具有”以及它们的任何变形，意图在于覆盖不排他的包含。
在本公开实施例的描述中，技术术语“第一”“第二”等仅用于区别不同对象，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量、特定顺序或主次关系。在本公开实施例的描述中，“多个”的含义是两个以上，除非另有明确具体的限定。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本公开的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
在本公开实施例的描述中，术语“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
在本公开实施例的描述中，术语“多个”指的是两个以上(包括两个)，同理，“多组”指的是两组以上(包括两组)，“多片”指的是两片以上(包括两片)。
在本公开实施例的描述中，技术术语“中心”“纵向”“横向”“长度”“宽度”“厚度”“上”“下”“前”“后”“左”“右”“竖直”“水平”“顶”“底”“内”“外”“顺时针”“逆时针”“轴向”“径向”“周向”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本公开实施例和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本公开实施例的限制。
在本公开实施例的描述中，除非另有明确的规定和限定，技术术语“安装”“相连”“连接”“固定”等术语应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或成一体；也可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通或两个元件的相互作用关系。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本公开实施例中的具体含义。
需要说明的是，本公开实施例提供的表面缺陷数据增强方法可以被电子设备执行，这里的电子设备是指具有执行计算机程序功能的设备终端或者服务器，设备终端例如：智能手机、个人电脑、平板电脑、个人数字助理或者移动上网设备等。服务器是指通过网络提供计算服务的设备，服务器例如：x86服务器以及非x86服务器，非x86服务器包括：大型机、小型机和UNIX服务器。
请参见图1示出的本公开实施例提供的电子设备(例如服务器)执行的表面缺陷数据增强方法的流程示意图；该方法可以应用于电子设备(例如服务器)，该方法的主要思路为：通过获取待增强部位信息实现对表面缺陷数据的划分，从而对划分后的表面缺陷数据分别进行数据增强，实现了针对特定需求位置进行数据扩充。上述表面缺陷数据增强方法的实施方式可以包括：
步骤S110：获取待增强的表面缺陷数据；
步骤S120：获取待增强部位信息；
步骤S130：根据待增强部位信息对表面缺陷数据进行划分；
步骤S140：对划分后的表面缺陷数据分别进行数据增强。
在步骤S110中，待增强的表面缺陷数据可以为待增强的表面缺陷图像，可以是红外图像、遥感图像、可见光图像以及偏振图像等，能够实现表面缺陷检测即可。
在步骤S120中，待增强部位信息可以至少包括待增强部位的模板数据，还可以包括待增强部位的名称。待增强部位信息可以由用户上传，也可以在相关开源数据库中检索获取，当然也可以使用爬虫爬取相关的模板数据。
在步骤S130中，根据待增强部位的模板数据，将表面缺陷数据中的数据与模板数据进行对比，两者相似度满足预设阈值的即可认为是与数据模板相匹配的部位，从而实现对表面缺陷数据的划分。
在步骤S140中，对划分后的表面缺陷数据分别进行数据增强的方式可以采用例如尺寸变换和色彩变换等基础数据增强方式，也可以采用基于深度学习的数据增强方法。其中，基于深度学习的数据增强方法例如基于生成对抗网络的数据增强方法等。
通过根据待增强部位信息对表面缺陷数据进行划分，然后对划分后的表面缺陷数据分别进行数据增强，实现了针对特定位置的数据增强，从而可以实现针对特定位置的表面缺陷检测。
作为上述表面缺陷数据增强方法的一种可选实施方式，在步骤S140中可以在划分后的表面缺陷数据中选择若干种类的划分后的表面缺陷数据进行增强，也可以针对一种类型的数据进行增强，例如在对齿轮表面缺陷数据进行数据增强时，首先将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据，然后根据具体的应用场景，可以选择仅对齿面缺陷数据或齿底缺陷数据进行数据增强，也可以对齿面缺陷数据和齿底缺陷数据进行增强。
作为上述表面缺陷数据增强方法的一种可选实施方式，在步骤S130根据待增强部位信息对表面缺陷数据进行划分前，还可以包括：对表面缺陷数据进行初次数据增强，获得经过初次数据增强后的表面缺陷数据。该实施方式可以包括：
步骤S150：对所述表面缺陷数据进行初次数据增强，获得经过初次数据增强后的表面缺陷数据。
步骤S130具体为：根据待增强部位信息对初次数据增强后的表面缺陷数据进行划分。
其中，对表面缺陷数据进行初次数据增强的目的在于：能够实现对表面缺陷数据的有效划分，后续配合对划分后数据的再次增强。采用初次增强配合再次增强的方式，可以有效改善划分后各样本数据的增强效果。
作为上述表面缺陷数据增强方法的一种可选实施方式，步骤S150中的初次数据增强方式可以包括：基于几何变化的数据增强、基于色彩变化的数据增强以及基于像素变换的数据增强中的至少一者。
通过多种方式来实现对表面缺陷数据的初次数据增强，可以采用几何变换、色彩变换和像素变换方式中的任意一种或几种来实现对样本的初次增强，从而为后续对表面缺陷数据进行划分提供数据支撑，改善数据划分效果。
作为上述表面缺陷数据增强方法的一种可选实施方式，步骤S140中的数据增强方式可以包括：基于几何变化的数据增强、基于色彩变化的数据增强以及基于像素变换的数据增强中的至少一者。
本公开实施例通过多种方式来实现对表面缺陷数据的数据增强，可以采用几何变换、色彩变换和像素变换方式中的任意一种或几种来实现对样本的再次增强，区别于初次增强的全局增强后方式，再次增强是针对划分后的表面缺陷数据进行的；初次增强目的在于对全局样本进行扩充，以改善区域划分效果；再次增强的目的在于对划分后的样本进行扩充；采用初次增强配合再次增强的方式，可以有效改善划分后各样本数据的增强效果。
下面详细介绍三种数据增强方式：
第一种方式，基于几何变化的数据增强，对数据集进行空间几何变换以实现数据增强，主要包括：翻转、旋转、裁剪、缩放、移位与边缘填充等方式。
第二种方式，基于色彩变化的数据增强，对数字图像色彩空间进行调节以实现数据增强，主要包括：在色彩通道上进行亮度调节和色度调剂、色彩空间转换等方式。
第三种方式，基于像素变换的数据增强，以数字图像的像素为基础实现数据增强，主要包括：噪声、模糊、图像融合、信息删除等。
需要指出的是，基于图像融合的数据增强方式可以包括：
(1)SMOTE方法：该方法将提取的图像特征映射到特征空间，在确定采样倍率后选取几个最相邻的样本，从中随机选取一个连线，并在连线上随机选取一个点作为新的样本点，重复至样本均衡。
(2)MIXUP方法：该方法再数据集中随机抽取两条数据，然后将抽取到的图像数据的像素值进行符合Beat分布的融合比例的线性加权求和，同时将样本对应的One-hot向量标签也对应加权求和，预测生成的新样本与加权求和后的标签的损失，进行反向求导并更新参数，同时抽取批量数据并进行随机打散后进行加权求和。
(3)CUTMIX方法：该方法将随机选中的区域填充其他图像的补丁区域。
(4)Sample Pairing方法：该方法首先从数据集中随机选择两种图片，再经像素取平均值，最后叠加合成一个新的样本。
上述四种方法在获取新的样本之后均会在缺陷边缘部分出现像素梯度变化不均的问题，使得卷积网络在进行特征学习时的效果不佳。
因此本公开实施例提出下述图像融合方式：
泊松融合：该方法使用背景图像的梯度场作为指导场计算融合梯度场，可以根据原图像的梯度信息以及目标缺陷图像的边界信息，利用插值的方法重新构建出合成区域内的图像像素。
作为上述表面缺陷数据增强方法的一种可选实施方式，待增强的表面缺陷数据可以包括：待增强的齿轮表面缺陷数据；根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。如图2所示，该实施方式可以包括：
步骤S210：获取待增强的齿轮表面缺陷数据；
步骤S220：获取待增强部位信息；
步骤S230：根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据；
步骤S240：分别对齿面缺陷数据和齿底缺陷数据进行数据增强。
其中，步骤S210中的待增强的齿轮表面缺陷数据可以是齿轮表面缺陷的图像，可以是红外图像、遥感图像、可见光图像以及偏振图像等，能够实现表面缺陷检测即可。步骤S220中的待增强部位信息包括待增强部位的模板数据，可以由用户上传，也可以在相关开源数据库中检索获取。步骤S230根据齿轮待增强部位的模板数据，将表面缺陷数据中的数据与模板数据进行对比，两者相似度满足预设阈值的即可认为是与数据模板相匹配的部位，从而实现对齿轮表面缺陷数据的划分。步骤S240对划分后的表面缺陷数据分别进行数据增强的方式可以采用例如尺寸变换和色彩变换等基础数据增强方式，也可以采用基于深度学习的数据增强方法。其中，基于深度学习的数据增强方法例如基于生成对抗网络的数据增强方法等。
通过待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据，实现了针对齿轮特定位置的数据增强；然后通过对齿面缺陷数据和齿底缺陷数据分别进行数据增强，进而通过齿面缺陷增强数据和齿底缺陷增强数据对表面缺陷检测网络进行训练，从而使得后续在针对齿轮表面进行缺陷检测时实现对不同齿轮部位的针对性缺陷检测。
作为上述表面缺陷数据增强方法的一种可选实施方式，步骤S220中待增强齿部位信息可以包括：齿面数据模板与齿底数据模板。步骤230中根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据，可以包括：针对齿面数据模板和齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。该实施方式可以包括：
步骤S210：获取待增强的齿轮表面缺陷数据；
步骤S220：获取待增强部位信息，包括齿面数据模板与齿底数据模板；
步骤S230：针对齿面数据模板和齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据；
步骤S240：分别对齿面缺陷数据和齿底缺陷数据进行数据增强。
基于齿面数据模板与齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据，实现了针对齿轮特定位置的数据增强；然后通过对齿面缺陷数据和齿底缺陷数据分别进行数据增强，进而通过齿面缺陷增强数据和齿底缺陷增强数据对表面缺陷检测网络进行训练，从而使得后续在针对齿轮表面进行缺陷检测时实现对不同齿轮部位的针对性缺陷检测。
作为上述表面缺陷数据增强方法的一种可选实施方式，步骤S230中的特征点匹配法可以包括特征点提取和特征点匹配两个步骤，其中特征点提取可以采用SIFT算法、SURF算法、FAST算法、BRIEF算法或ORB算法等来提取特征点，特征点匹配方式可以采用基于特征点相似度的匹配方式，其中根据特征描述子的不同，可以选择不同的距离来作为相似度的度量，若是浮点类型的描述子，可以使用其欧式距离；对于二进制的描述子，可以使用其汉明距离。
作为上述表面缺陷数据增强方法的一种可选实施方式，步骤S140中对划分后的表面缺陷数据分别进行数据增强，可以包括：分别获取划分后的表面缺陷数据的低频分量；根据低频分量对划分后的表面缺陷数据分别进行数据增强。如图3所示，该实施方式可以包括：
步骤S140-1：分别获取划分后的表面缺陷数据的低频分量；
步骤S140-2：根据低频分量对划分后的表面缺陷数据分别进行数据增强。
通过对低频分量进行数据增强，而保留高频分量，使得图像的细节信息得到有效保留，在对数量进行增强的基础上，还保证了扩充数据的质量，有效改善了表面缺陷数据的数据增强效果。使用经过上述增强方式之后的数据对表面缺陷检测网络进行训练，可以获得更好的训练及学习效果，进一步提高表面缺陷检测网络的检测精度。
作为上述表面缺陷数据增强方法的一种可选实施方式，步骤S140-1中获取低频分量的方式可以为：采用傅里叶变换获得图像的高频分量和低频分量，其中低频分量代表图像的灰度值以及亮度等信息，高频分量代表图像的边缘、噪声以及细节信息。因此步骤S140-2根据低频分量对划分后的表面缺陷数据分别进行数据增强。
通过采用傅里叶变化对低频分量进行数据增强，而保留高频分量，使得图像的细节信息得到有效保留，在对数量进行增强的基础上，还保证了扩充数据的质量，有效改善了表面缺陷数据的数据增强效果。使用经过上述增强方式之后的数据对表面缺陷检测网络进行
训练，可以获得更好的训练及学习效果，进一步提高表面缺陷检测网络的检测精度。
请参见图4示出的本公开实施例提供的表面缺陷数据增强装置的结构示意图；基于同一种发明构思，本公开实施例提供了一种表面缺陷数据增强装置400，所述表面缺陷数据增强装置400可以包括：
待增强表面缺陷数据获取模块410，所述待增强表面缺陷数据获取模块410可以被配置成用于获取待增强的表面缺陷数据；
待增强部位信息获取模块420，所述待增强部位信息获取模块420可以被配置成用于获取待增强部位信息；
数据划分模块430，所述数据划分模块430可以被配置成用于根据待增强部位信息对表面缺陷数据进行划分；
数据增强模块440，所述数据增强模块440可以被配置成用于对划分后的表面缺陷数据分别进行数据增强。
可选地，在本公开实施例中，表面缺陷数据增强装置400还可以包括：
初次数据增强模块450，所述初次数据增强模块450可以被配置成用于对待增强表面缺陷数据获取模块410获取的待增强的表面缺陷数据进行初次数据增强；
在本实施例中，数据划分模块430可以被配置成用于根据待增强部位信息对初次数据增强后的表面缺陷数据进行划分。
可选地，在本公开实施例中，初次数据增强模块450的数据增强方式可以包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。
可选地，在本公开实施例中，数据增强模块440对划分后的表面缺陷数据分别进行数据增强的增强方式可以包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。
可选地，在本公开实施例中，待增强表面缺陷数据获取模块410所获取的待增强的表面缺陷数据可以包括：待增强的齿轮表面缺陷数据。数据划分模块430具体为：根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。
可选地，在本公开实施例中，待增强部位信息获取模块420所获取的待增强部位信息可以包括：齿面数据模板与齿底数据模板。数据划分模块430具体为：针对齿面数据模板和齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。
可选地，在本公开实施例中，数据增强模块440可以包括：
低频分量获取单元，所述低频分量获取单元可以被配置成用于获取划分后的表面缺陷
数据的低频分量；
数据增强单元，所述数据增强单元可以被配置成用于根据低频分量对划分后的表面缺陷数据分别进行数据增强。
应理解的是，该装置与上述的表面缺陷数据增强方法实施例对应，能够执行上述方法实施例涉及的各个步骤，该装置具体的功能可以参见上文中的描述，为避免重复，此处适当省略详细描述。该装置包括至少一个能以软件或固件(firmware)的形式存储于存储器中或固化在装置的操作系统(operating system，OS)中的软件功能模块。
请参见图5示出的本公开实施例提供的表面缺陷数据增强方法在齿轮表面缺陷数据增强应用场景下的流程示意；本公开实施例提供了一种齿轮表面缺陷数据增强方法，该实施方式包括：
步骤S510：获取待增强的齿轮表面缺陷数据；
步骤S520：对待增强的齿轮表面缺陷数据进行初次数据增强，获取经过初次数据增强后的齿轮表面缺陷数据；
采用平移、旋转、随机裁剪等方式对数据进行基础扩充，根据数据样本中图像尺寸的大小，平移的尺度变化为0～30％，旋转角度变化尺度为0～180°，随机裁剪的比例为0～40％；
步骤S530：获取待增强部位信息，包括齿面数据模板和齿底图像模板；
步骤S540：基于齿面数据模板和齿底图像模板，采用SURF算法提取特征点，以欧式距离为相似度度量，对特征点进行匹配，筛选出齿面和齿底区域，将经过初次数据增强的齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据；
本公开实施例中所采用的SURF算法通过Hessian矩阵进行特征点的提取，针对图像中每一个像素点(x,y)都有对应的Hessian矩阵，其公式如下所示：
因为选取的特征点要具有尺度无关性，所以在构造矩阵前，需要先对图像进行高斯滤波。滤波后图像I(x,y)的Hessian矩阵如下所示：
其中，Lxx(x,y,σ)、Lxy(x,y,σ)、Lxy(x,y,σ)和Lyy(x,y,σ)分别表示I(x,y)和高斯函数二阶导数的卷积。
将定位到的特征点分别与数据模板进行欧式距离求解，若欧氏距离的值小于设定的阈
值即为匹配成功，对表面缺陷数据中的所有数字图像进行匹配后即可筛选出齿面与齿底区域。
欧式距离的计算方法为：
步骤S550：分别对齿面缺陷数据和齿底缺陷数据进行数据增强；
步骤S550-1：通过傅里叶变换获得缺陷数据的高频分量和低频分量；
步骤S550-2：对缺陷数据的低频分量采用泊松融合的方式进行数据增强。
针对齿轮表面缺陷检测这一应用场景，由于齿底表面缺陷对齿轮的影响较小，而齿面的表面缺陷对齿轮的影响较大，所以在将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据后，可以着重对齿面缺陷数据进行数据增强，由此可以获得针对齿面缺陷数据进行数据增强后的训练数据，使得齿轮表面缺陷检测网络可以更加着重于齿面的表面缺陷检测。
泊松融合的具体过程为：泊松融合的核心思想不是让需要融合的两张图像直接叠加，而是让目标图像在融合部分根据源图像的梯度场生长出新的图像。
S为二维实数集的闭合子集，Ω为S的边界为闭合子集，f*为集合S-Ω部分的函数(若是图像，则就是指所有像素的像素值)，f为集合Ω的函数，v为集合Ω的向量场，求解梯度最小值的方式如下：
在边界一致的条件下，梯度平滑，即：
代入到拉格朗日方程进行求解，变化最小的解即为泊松等式的解，即：
对图像的低频分量通过泊松方程求解插值。
上述表面缺陷数据增强方法在齿轮表面缺陷数据增强应用场景下：
(1)实现在指定齿轮位置区域，即在齿面位置和齿底位置上进行准确的数据增强；
(2)实现了梯度域上的连续；
(3)采用泊松融合的方式对低频分量进行数据增强，充分保留图像的高频信息，即充分保留了目标缺陷图像的特征纹理信息，使得增强后的数据在神经网络中的训练准确度更高，学习效果更优。
请参见图6示出的本公开实施例提供的电子设备的结构示意图。本公开实施例提供的
一种电子设备600，可以包括：中央处理单元CPU 601，其可以根据存储在只读存储器ROM 602中的计算机程序指令或者从存储单元加载到随机访问存储器RAM 603中的计算机程序指令，来执行各种适当的动作和处理。在RAM 603中，还可以存储设备操作所需的各种程序和数据。CPU 601、ROM 602以及RAM 603通过总线604彼此相连。I/O接口605也连接至总线604。
设备中的多个部件连接至I/O接口605，可以包括：输入单元606，例如键盘、鼠标等；输出单元607，例如各种类型的显示器、扬声器等；存储单元608，例如磁盘、光盘等；以及通信单元609，例如网卡、调制解调器、无线通信收发机等。通信单元609允许设备通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。
CPU601执行上文所描述的各个方法和处理，例如本公开实施例中方法步骤S110～S140。例如，在一些实施例中，本公开实施例中方法步骤S110～S140可被实现为计算机软件程序，其被有形地包含于非暂态计算机可读存储介质，例如存储单元。在一些实施例中，计算机程序的部分或者全部可以经由ROM 602和/或通信单元609而被载入和/或安装到设备上。当计算机程序加载到RAM 603并由CPU 601执行时，可以执行上文描述的本公开实施例中方法步骤S110～S140的一个或多个步骤。备选地，在其他实施例中，CPU 601可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行本公开实施例中方法步骤S110～S140。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。
在本公开的上下文中，非暂态计算机可读存储介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。非暂态计算机可读存储介质可以是机器可读信号介质或机器可读储存介质。非暂态计算机可读存储介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。非暂态计算机可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、
只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD‐ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
在本公开所提供的实施例中，应该理解到，所揭露装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种上述方法的逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
另外，作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
再者，在本公开各个实施例中的各功能模块可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或两个以上模块集成形成一个独立的部分。
以上所述仅为本公开的实施例而已，并不用于限制本公开的保护范围，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。
本公开提供了一种表面缺陷数据增强方法、装置、电子设备及存储介质，其中表面缺陷数据增强方法包括：获取待增强的表面缺陷数据；获取待增强部位信息；根据待增强部位信息对表面缺陷数据进行划分；对划分后的表面缺陷数据进行数据增强。通过根据待增强部位信息对表面缺陷数据进行划分，然后对划分后的表面缺陷数据分别进行数据增强，实现了针对特定位置的数据增强，从而可以实现针对特定位置的表面缺陷检测。
此外，可以理解的是，本公开的表面缺陷数据增强装置是可以重现的，并且可以用在多种工业应用中。例如，本公开的表面缺陷数据增强装置可以用于需要用表面缺陷数据增强装置对特定位置进行表面缺陷检测的任何应用。
Claims (14)
- 一种表面缺陷数据增强方法，其中，所述表面缺陷数据增强方法包括：获取待增强的表面缺陷数据；获取待增强部位信息；根据待增强部位信息对表面缺陷数据进行划分；对划分后的表面缺陷数据进行数据增强。
- 根据权利要求1所述的表面缺陷数据增强方法，其中，在所述根据待增强部位信息对表面缺陷数据进行划分前，所述表面缺陷数据增强方法还包括：对所述表面缺陷数据进行初次数据增强，获得经过初次数据增强后的表面缺陷数据；所述根据待增强部位信息对表面缺陷数据进行划分，包括：根据待增强部位信息对初次数据增强后的表面缺陷数据进行划分。
- 根据权利要求2所述的表面缺陷数据增强方法，其中，所述初次数据增强的方式包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。
- 根据权利要求1至3中任一项所述的表面缺陷数据增强方法，其中，所述对划分后的表面缺陷数据分别进行数据增强的增强方式，包括：基于几何变换的数据增强、基于色彩变换的数据增强、以及基于像素变换的数据增强中的至少一者。
- 根据权利要求1至4中任一项所述的表面缺陷数据增强方法，其中，所述待增强的表面缺陷数据包括：待增强的齿轮表面缺陷数据；所述根据待增强部位信息对表面缺陷数据进行划分，包括：根据待增强部位信息将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。
- 根据权利要求5所述的表面缺陷数据增强方法，其中，所述待增强部位信息包括：齿面数据模板与齿底数据模板；所述根据待增强部位信息对齿轮表面缺陷数据进行划分，包括：针对齿面数据模板和齿底数据模板，采用特征点匹配法将齿轮表面缺陷数据划分为齿面缺陷数据和齿底缺陷数据。
- 根据权利要求6所述的表面缺陷数据增强方法，其中，所述特征点匹配法包括特征点提取和特征点匹配两个步骤，其中，所述特征点提取采用SIFT算法、SURF算法、FAST算法、BRIEF算法或ORB算法来提取特征点，所述特征点匹配方式采用基于特征点相似度的匹配方式。
- 根据权利要求1至7中任一项所述的表面缺陷数据增强方法，其中，所述对划分后 的表面缺陷数据分别进行数据增强，包括：分别获取划分后的表面缺陷数据的低频分量；根据所述低频分量对所述划分后的表面缺陷数据分别进行数据增强。
- 根据权利要求8所述的表面缺陷数据增强方法，其中，所述获取低频分量的方式为：采用傅里叶变换获得所述划分后的表面缺陷数据的高频分量和低频分量，根据所述低频分量对所述划分后的表面缺陷数据分别进行数据增强包括：对所述划分后的表面缺陷数据的所述低频分量采用泊松融合的方式进行数据增强。
- 一种表面缺陷数据增强装置，其中，所述表面缺陷数据增强装置包括：待增强表面缺陷数据获取模块，所述待增强表面缺陷数据获取模块被配置成用于获取待增强的表面缺陷数据；待增强部位信息获取模块，所述待增强部位信息获取模块被配置成用于获取待增强部位信息；数据划分模块，所述数据划分模块被配置成用于根据待增强部位信息对表面缺陷数据进行划分；数据增强模块，所述数据增强模块被配置成用于对划分后的表面缺陷数据分别进行数据增强。
- 根据权利要求10所述的表面缺陷数据增强装置，其中，所述表面缺陷数据增强装置还包括：初次数据增强模块，所述初次数据增强模块被配置成用于对待增强表面缺陷数据获取模块获取的待增强的表面缺陷数据进行初次数据增强；所述数据划分模块还被配置成根据所述待增强部位信息对初次数据增强后的表面缺陷数据进行划分。
- 根据权利要求10或11所述的表面缺陷数据增强装置，其中，所述数据增强模块包括：低频分量获取单元，所述低频分量获取单元被配置成用于获取划分后的表面缺陷数据的低频分量；数据增强单元，所述数据增强单元被配置成用于根据所述低频分量对所述划分后的表面缺陷数据分别进行数据增强。
- 一种电子设备，其中，所述电子设备包括：处理器、存储器和总线，其中，所述处理器和所述存储器通过所述总线完成相互间的通信；所述存储器存储有能够被所述处理器执行的程序指令，所述处理器调用所述程序指令能够执行根据权利要求1至7中任一项所述的方法。
- 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有计算机程序， 所述计算机程序被处理器运行时执行根据权利要求1至7中任一项所述的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211264174.X | 2022-10-17 | ||
| CN202211264174.XA CN115330803B (zh) | 2022-10-17 | 2022-10-17 | 一种表面缺陷数据增强方法、装置、电子设备及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024082925A1 true WO2024082925A1 (zh) | 2024-04-25 |
Family
ID=83915494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2023/121002 WO2024082925A1 (zh) | 2022-10-17 | 2023-09-25 | 一种表面缺陷数据增强方法、装置、电子设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115330803B (zh) |
| WO (1) | WO2024082925A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115330803B (zh) * | 2022-10-17 | 2023-01-24 | 菲特（天津）检测技术有限公司 | 一种表面缺陷数据增强方法、装置、电子设备及存储介质 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107633491A (zh) * | 2017-09-26 | 2018-01-26 | 中国科学院长春光学精密机械与物理研究所 | 一种基于目标检测的区域图像增强方法及存储介质 |
| CN109211919A (zh) * | 2018-09-03 | 2019-01-15 | 珠海格力智能装备有限公司 | 磁瓦崩缺区域的识别方法及装置 |
| CN110969611A (zh) * | 2019-12-03 | 2020-04-07 | 广州特种承压设备检测研究院 | 管道焊缝缺陷检测方法、装置、系统及存储介质 |
| CN113256608A (zh) * | 2021-06-17 | 2021-08-13 | 常州微亿智造科技有限公司 | 工件缺陷检测方法和装置 |
| CN115330803A (zh) * | 2022-10-17 | 2022-11-11 | 菲特（天津）检测技术有限公司 | 一种表面缺陷数据增强方法、装置、电子设备及存储介质 |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105510348B (zh) * | 2015-12-31 | 2019-03-08 | 南京协辰电子科技有限公司 | 一种印制电路板的缺陷检测方法、装置及检测设备 |
| CN109660821B (zh) * | 2018-11-27 | 2021-09-14 | Oppo广东移动通信有限公司 | 视频处理方法、装置、电子设备及存储介质 |
| CN110977767B (zh) * | 2019-11-12 | 2021-07-02 | 长沙长泰机器人有限公司 | 一种铸件瑕疵分布检测方法以及铸件打磨方法 |
| CN114663293A (zh) * | 2020-12-23 | 2022-06-24 | 杭州海康威视数字技术股份有限公司 | 一种图像增强方法、装置、电子设备及内窥镜系统 |
| CN113726981B (zh) * | 2021-09-30 | 2022-05-17 | 北京耐德佳显示技术有限公司 | 图像数据处理方法、电子设备及存储介质 |
| CN114359305A (zh) * | 2021-12-31 | 2022-04-15 | Oppo广东移动通信有限公司 | 图像处理方法、装置、电子设备和计算机可读存储介质 |
| CN114638294A (zh) * | 2022-03-10 | 2022-06-17 | 深圳市腾盛精密装备股份有限公司 | 一种数据增强方法、装置、终端设备及存储介质 |
-
2022
- 2022-10-17 CN CN202211264174.XA patent/CN115330803B/zh active Active
-
2023
- 2023-09-25 WO PCT/CN2023/121002 patent/WO2024082925A1/zh unknown
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107633491A (zh) * | 2017-09-26 | 2018-01-26 | 中国科学院长春光学精密机械与物理研究所 | 一种基于目标检测的区域图像增强方法及存储介质 |
| CN109211919A (zh) * | 2018-09-03 | 2019-01-15 | 珠海格力智能装备有限公司 | 磁瓦崩缺区域的识别方法及装置 |
| CN110969611A (zh) * | 2019-12-03 | 2020-04-07 | 广州特种承压设备检测研究院 | 管道焊缝缺陷检测方法、装置、系统及存储介质 |
| CN113256608A (zh) * | 2021-06-17 | 2021-08-13 | 常州微亿智造科技有限公司 | 工件缺陷检测方法和装置 |
| CN115330803A (zh) * | 2022-10-17 | 2022-11-11 | 菲特（天津）检测技术有限公司 | 一种表面缺陷数据增强方法、装置、电子设备及存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115330803B (zh) | 2023-01-24 |
| CN115330803A (zh) | 2022-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10453204B2 (en) | Image alignment for burst mode images | |
| US10803554B2 (en) | Image processing method and device | |
| US8687923B2 (en) | Robust patch regression based on in-place self-similarity for image upscaling | |
| CN109683699B (zh) | 基于深度学习实现增强现实的方法、装置及移动终端 | |
| CN109753971B (zh) | 扭曲文字行的矫正方法及装置、字符识别方法及装置 | |
| CN108229301B (zh) | 眼睑线检测方法、装置和电子设备 | |
| WO2024082925A1 (zh) | 一种表面缺陷数据增强方法、装置、电子设备及存储介质 | |
| CN110570440A (zh) | 一种基于深度学习边缘检测的图像自动分割方法及装置 | |
| CN111402170A (zh) | 图像增强方法、装置、终端及计算机可读存储介质 | |
| CN110992265A (zh) | 图像处理方法及模型、模型的训练方法及电子设备 | |
| CN110930296A (zh) | 图像处理方法、装置、设备及存储介质 | |
| CN110211195B (zh) | 生成图像集合的方法、装置、电子设备和计算机可读存储介质 | |
| EP4229540A1 (en) | Systems and methods for image detection | |
| CN115330940B (zh) | 一种三维重建方法、装置、设备和介质 | |
| JP5832656B2 (ja) | 画像中のテキストの検出を容易にする方法及び装置 | |
| WO2023147717A1 (zh) | 文字检测方法、装置、电子设备和存储介质 | |
| WO2022247406A1 (en) | Systems and methods for determining key frame images of video data | |
| CN112528811A (zh) | 行为识别方法和装置 | |
| CN112836692A (zh) | 用于处理图像的方法、装置、设备和介质 | |
| CN111435445A (zh) | 字符识别模型的训练方法及装置、字符识别方法及装置 | |
| CN108647605B (zh) | 一种结合全局颜色与局部结构特征的人眼凝视点提取方法 | |
| Qiu et al. | The infrared moving target extraction and fast video reconstruction algorithm | |
| GB2557417A (en) | Image alignment for burst mode images | |
| CN107038705B (zh) | 视网膜图像出血区域分割方法、装置和计算设备 | |
| JP6910622B2 (ja) | 画像処理システム |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23878918 Country of ref document: EP Kind code of ref document: A1 |