CN114638294A - Data enhancement method and device, terminal equipment and storage medium - Google Patents
Data enhancement method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN114638294A CN114638294A CN202210237047.4A CN202210237047A CN114638294A CN 114638294 A CN114638294 A CN 114638294A CN 202210237047 A CN202210237047 A CN 202210237047A CN 114638294 A CN114638294 A CN 114638294A
- Authority
- CN
- China
- Prior art keywords
- defect
- picture
- target
- sample
- pictures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000007547 defect Effects 0.000 claims abstract description 406
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 5
- 230000002950 deficient Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000001514 detection method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 241001292396 Cirrhitidae Species 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Image Processing (AREA)
Abstract
The application is applicable to the technical field of industrial vision, and provides a data enhancement method, a data enhancement device, terminal equipment and a storage medium. Acquiring a defect sample picture, generating a defect picture according to a defect area in the defect sample picture, and storing the defect picture in a preset defect sample set; if the defect pictures in the defect sample set meet preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures; and training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures, thereby improving the accuracy of the model identification result.
Description
Technical Field
The application belongs to the technical field of industrial vision, and particularly relates to a data enhancement method and device, a terminal device and a storage medium.
Background
In the industry, deep learning is more and more emphasized by people as a learning type algorithm, has certain reasoning and generalization capability, and can train a deep learning model through product pictures of qualified products and unqualified products, so that the model can classify and detect the product pictures in the industry to judge whether the products are qualified.
However, in an actual environment, the number of normal sample pictures corresponding to qualified products is far greater than that of defect sample pictures of unqualified products, and when the deep learning model is trained by using the data, the accuracy of the model identification result is low due to low richness of the sample pictures.
Disclosure of Invention
The embodiment of the application provides a data enhancement method, a data enhancement device, terminal equipment and a storage medium, and can solve the problem of low accuracy of a model identification result.
In a first aspect, an embodiment of the present application provides a data enhancement method, including:
acquiring a defect sample picture, generating a defect picture according to a defect area in the defect sample picture, and storing the defect picture in a preset defect sample set;
if the defect pictures in the defect sample set meet preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures;
and training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures.
In one embodiment, after storing the defect picture in a preset defect sample set, the method further includes:
and if the defect pictures in the defect sample set do not meet the preset conditions, returning to execute the step of obtaining the defect sample pictures until the defect pictures in the defect sample set meet the preset conditions.
In an embodiment, the generating a corresponding defect picture according to a defect area in the defect sample picture includes:
determining the target defect type of the defect pictures in the defect sample set;
determining a target area from the defect areas according to the target defect type, wherein the defect type of the target area is different from the target defect type;
and generating the defect picture according to the target area.
In an embodiment, the determining the target defect type of the defect picture in the defect sample set includes:
acquiring the defect shape of the defect picture in the defect sample set;
and determining the target defect type of the defect picture according to the defect shape.
In an embodiment, the generating the defect picture according to the target area includes:
performing minimum external rectangle processing on the target area to obtain a rectangular area;
setting pixels of a boundary region to zero, the boundary region being a region other than the target region within the rectangular region;
and generating the defect picture according to each pixel of the rectangular area.
In an embodiment, the randomly overlaying the defect pictures in the defect sample set onto a preset number of normal sample pictures includes:
randomly selecting a target defect picture from the defect sample set;
rotating the target defect picture in a preset mode to obtain a rotated target defect picture;
randomly selecting a target position from the normal sample picture, and covering the rotated target defect picture on the target position.
In an embodiment, the rotating the target defect picture in a preset manner to obtain a rotated target defect picture includes:
determining the rotation angle and the rotation direction in the preset mode;
and mapping the position of each pixel in the target defect picture according to the rotation angle and the rotation direction to obtain the rotated target defect picture.
In a second aspect, an embodiment of the present application provides a data enhancement apparatus, including:
the image generation module is used for acquiring a defect sample image, generating a defect image according to a defect area in the defect sample image and storing the defect image in a preset defect sample set;
the image covering module is used for randomly covering the defect images in the defect sample set onto a preset number of normal sample images to obtain a preset number of target sample images if the defect images in the defect sample set meet preset conditions;
and the training module is used for training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures.
In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements any of the steps of the data enhancement method when executing the computer program.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the data enhancement methods described above.
In a fifth aspect, embodiments of the present application provide a computer program product, which, when run on a terminal device, causes the terminal device to execute any one of the data enhancement methods in the first aspect.
In the embodiment of the application, a defect sample picture is obtained, a defect picture, namely a defect picture only containing defects, is generated according to a defect area in the defect sample picture, and the defect picture is stored in a preset defect sample set. If the defect pictures in the defect sample set meet the preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures, namely obtaining a certain number of possible defect pictures of unqualified products, and finally training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures, so that the normal sample pictures are covered by the defect pictures to obtain the possible defect pictures of the unqualified products, the data of the samples are enhanced, the richness of the samples is improved, and the accuracy of model identification results is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flowchart of a data enhancement method provided in an embodiment of the present application;
FIG. 2 is a sample picture of a defect provided by an embodiment of the present application;
FIG. 3 is a defect picture provided by an embodiment of the present application;
FIG. 4 is a normal sample picture provided by an embodiment of the present application;
FIG. 5 is a sample picture of a target provided by an embodiment of the present application;
FIG. 6 is a schematic structural diagram of a data enhancement device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Fig. 1 is a schematic flowchart of a data enhancement method in an embodiment of the present application, where an execution subject of the method may be a terminal device, and as shown in fig. 1, the data enhancement method may include the following steps:
step S101, a defect sample picture is obtained, a defect picture is generated according to a defect area in the defect sample picture, and the defect picture is stored in a preset defect sample set.
In this embodiment, the terminal device may obtain a sample picture for training a model, and select a sample picture with a defect from the sample pictures, where the sample picture with the defect is the defect sample picture, as shown in fig. 2, where a crescent defect exists in fig. 2. The terminal device can identify a region with a defect in the acquired defect sample picture, namely the defect region, through the visualization software, and then generate a defect picture according to the defect region in a preset mode, wherein the defect picture only has a defect, as shown in fig. 3, so that data enhancement can be performed based on the defect picture at a later stage. The terminal equipment can store the generated defect picture in a preset defect sample set so as to extract the defect picture in a later period.
Specifically, terminal equipment can utilize Halcon to read defect sample picture to show in QT's QLabel control, and can further enlarge or reduce defect sample picture, thereby enlarge the defect region of defect sample picture, until the pixel definition in this defect region accords with preset definition threshold, this definition threshold is the threshold that corresponds when the pixel is clear can distinguish, improves the precision that defect picture acquireed through enlarging defect sample picture. The terminal equipment can also name the defect picture according to the generation time of the defect picture after acquiring the defect picture.
In one embodiment, the step S101 may include: the terminal device may utilize an operator for drawing a region in Halcon to divide a target region from the defect region, where the target region includes all or part of the defect region, so as to generate a defect picture from the target region, where the dividing of the target region may be a control instruction issued by a user, and the terminal device is prompted to divide the target region from the defect region in a manner of circle, scratch, cut, and the like based on the control instruction. The target area can be divided by the terminal equipment according to the preset random algorithm, so that the target area is divided according to the position, the shape, the angle, the area and other information in the defect area, and the richness of the defect picture is improved.
In one embodiment, in order to improve the determination efficiency of the defect picture while satisfying the richness of the defect picture, the step S101 may include: and the terminal equipment determines the target defect types of the defect pictures in the defect sample set, wherein different defect pictures in the defect sample set correspond to different target defect types. The terminal device determines a target area from the defect area according to the target defect type, that is, the defect type of the target area is different from the target defect type, that is, the defect type which does not exist in the defect sample set is determined according to the target defect type which exists in the defect sample set, so as to determine a corresponding target area based on the non-existing defect type, and finally, the defect picture is generated according to the target area.
It can be understood that, since the target area for generating the defect picture may be a part of the defect area, the defect sample picture acquired by the terminal device in step S101 may be a defect sample picture of which the defect picture has been generated, or a defect sample picture of which the defect picture has not been generated and acquired by the terminal device again. If the defect sample picture of the defect picture is generated, different target regions can be determined from the defect sample picture through different information such as positions, shapes, angles, areas and the like, so that the defect pictures of different defect types are generated, the richness of the defect pictures can be improved, and the pertinence of the defect pictures is stronger.
In an embodiment, the determining the defect type that does not exist in the defect sample set according to the target defect type may include: and obtaining each defect type corresponding to the possible defects of the product, and comparing the defect type with the target defect type to determine the defect type which does not exist in the defect sample set.
In an embodiment, the determining the target defect type of the defect picture in the defect sample set may include: and the terminal equipment acquires the defect shapes of the defect pictures in the defect sample set, so that the target defect types of the defect pictures are determined according to the defect shapes, namely the defect shapes of the defect pictures in the defect sample set are different.
In an embodiment, the generating the defect picture according to the target area may include: the terminal device performs minimum enclosing rectangle processing on the target area to obtain a rectangular area, determines an area except the target area in the rectangular area as a boundary area in the rectangular area, namely the boundary area is a part without defects, processes the boundary area, for example, sets pixels in the boundary area to be zero, and generates the defect picture according to each pixel of the rectangular area.
Specifically, the terminal device may use a Halcon operator to make a minimum bounding rectangle for the target region, extract all pixel values in the minimum bounding rectangle, maintain the pixels in the target region unchanged, and set the pixels in the boundary region to zero, thereby generating one picture according to each pixel in the rectangular region.
In one embodiment, after step S101, the method may further include: if the defect pictures in the defect sample set do not meet the preset conditions, which indicates that the abundance of the defect pictures currently used for data enhancement does not meet the requirements, the method returns to execute the step of obtaining the defect sample pictures, wherein the defect sample pictures can be the defect sample pictures of which the defect pictures are generated last time or the defect sample pictures of which the defect pictures are not generated until the defect pictures in the defect sample set meet the preset conditions. The preset condition may be set according to a user requirement, for example, the condition is set as the number of defect types corresponding to the defect picture, and when the number of different defect types in the defect sample set satisfies a preset threshold, it indicates that the condition is satisfied. It can be understood that, if a defect sample picture based on a generated defect picture cannot obtain a defect picture different from the target defect type in the sample set, the defect sample picture obtained again is the defect sample picture without the generated defect picture.
Exemplarily, it is set that 10 defect sample pictures and 1000 normal sample pictures exist in the sample pictures currently used for training the model, and the defect sample picture acquired by the terminal device in step S101 is a defect sample picture that is not generated with a defect picture and is acquired by the terminal device again, where the condition is that the number of different defect types in the defect sample set is 10, so that one defect sample picture can generate one defect picture correspondingly, 10 defect pictures with different defect types can be generated according to the 10 defect sample pictures, and the 10 defect pictures are stored in the defect sample set.
And S102, if the defect pictures in the defect sample set meet preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures.
In this embodiment, if the defect pictures in the defect sample set satisfy the preset condition, which indicates that the abundance of the defect pictures currently used for data enhancement meets the requirement, the terminal device may perform data enhancement by using the defect pictures in the defect sample set, that is, randomly overlay the defect pictures in the defect sample set onto a normal sample picture, where the normal sample picture is a picture without defects, as shown in fig. 4, so as to obtain a corresponding target sample picture through defect overlay, and as shown in fig. 5, generate the target sample picture by changing the defect type of the defect and changing the background where the defect is located, thereby increasing the type of the defect sample.
In one embodiment, the step S102 may include: and the terminal equipment randomly selects a target defect picture from the defect sample set through a random algorithm, rotates the target defect picture in a preset mode to obtain a rotated target defect picture, randomly selects a target position from the normal sample picture, and covers the rotated target defect picture on the target position.
It can be understood that, if the distance between the target position and the boundary of the normal sample picture is smaller than the preset distance, whether the rotated target defect picture exceeds the boundary when being covered on the normal sample picture is judged according to the size of the rotated target defect picture and the target point, and if the rotated target defect picture does not exceed the boundary, the rotated target defect picture can be directly covered on the target position; and if the target position exceeds the boundary, modifying the target position according to the size of the target defect picture, so that the target defect picture is prevented from exceeding the boundary when covering the normal sample picture.
In an embodiment, the rotating the target defect picture in a preset manner to obtain the rotated target defect picture may include: the terminal device may determine a rotation angle and a rotation direction in the preset manner, where the rotation angle and the rotation direction may be randomly generated, so as to perform position mapping on each pixel in the target defect picture according to the rotation angle and the rotation direction, and obtain a rotated target defect picture. The terminal device can map the position of each pixel in the target defect picture according to the rotation angle and the rotation direction by using an opencv algorithm, so that the phenomenon of black edges of the generated target sample picture is avoided.
Illustratively, based on the above example, the terminal device first randomly selects a target defect picture from a defect sample set having 10 defect pictures, then determines the rotated target defect picture, randomly selects a coordinate point, i.e. the target position, from a normal sample picture, and covers the rotated target defect picture on the coordinate point in the normal sample picture, so as to obtain the target sample picture, and when the target sample picture is generated, the defect picture is covered on other normal sample pictures in 1000 normal sample pictures by using the cyclic algorithm in the above manner, so as to obtain 1000 target sample pictures in total.
Step S103, training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures.
In this embodiment, under the condition that the number of the defective sample pictures is small, the terminal device obtains a preset number of normal sample pictures through data enhancement, and trains the model based on the preset number of normal sample pictures, that is, trains the model through the training data after the data enhancement, trains a neural network model with strong robustness, improves the accuracy of the recognition result when the model is used for recognition in the industrial field, and avoids the occurrence of a large increase in the over-inspection rate while reducing the over-inspection rate.
It can be understood that if the model is suitable for classification detection of product pictures in industry, so as to determine whether a product is NG or OK, the model can be trained by using the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures, so that the model can accurately perform classification detection on the pictures to be recognized, and the pictures to be recognized are the product pictures.
Illustratively, based on the above example, the terminal device may send 10 defect sample pictures, 1000 normal sample pictures, and 1000 target sample pictures into the Resnet18 network model for training, and train a model a, where the model a may meet the accuracy requirement for classifying and detecting product pictures, for example, classifying 1000 OK pictures and 150 NG pictures with the model a may obtain a missed detection rate of 1.3%, and an over-detection rate of 0. In addition, in order to form comparison so as to judge whether the model trained by the method meets requirements or not, the terminal device can also compare the model trained by the defect sample picture and the normal sample picture, namely, train the model according to 10 defect sample pictures and 1000 normal sample pictures, train the model B, classify 1000 OK pictures and 150 NG pictures by using the model B, so that the undetected rate is 100% and the over-detection rate is 0, and thus, after the model is trained by the data enhancement method, the undetected rate of the model can be reduced from 100% to 1.3%, the undetected rate is reduced in a leap manner, and the application effect of the model is greatly improved.
In the embodiment of the application, a defect sample picture is obtained, a defect picture, namely a defect picture only containing defects, is generated according to a defect area in the defect sample picture, and the defect picture is stored in a preset defect sample set. If the defect pictures in the defect sample set meet the preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures, namely obtaining a certain number of possible defect pictures of unqualified products, and finally training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures, so that the normal sample pictures are covered by the defect pictures to obtain the possible defect pictures of the unqualified products, the data of the samples are enhanced, the richness of the samples is improved, and the accuracy of model identification results is improved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Corresponding to the above-mentioned data enhancement method, fig. 6 is a schematic structural diagram of a data enhancement device in an embodiment of the present application, and as shown in fig. 6, the data enhancement device may include:
the picture generating module 601 is configured to obtain a defect sample picture, generate a defect picture according to a defect area in the defect sample picture, and store the defect picture in a preset defect sample set.
The image covering module 602 is configured to, if the defect images in the defect sample set meet a preset condition, randomly cover the defect images in the defect sample set onto a preset number of normal sample images to obtain a preset number of target sample images.
A training module 603, configured to train a preset model according to the defect sample pictures, the preset number of normal sample pictures, and the preset number of target sample pictures.
In one embodiment, the data enhancement apparatus may further include:
and the execution module is used for returning to execute the defect sample picture acquisition if the defect pictures in the defect sample set do not meet the preset conditions until the defect pictures in the defect sample set meet the preset conditions.
In one embodiment, the image generating module 601 may include:
and the type determining unit is used for determining the target defect type of the defect picture in the defect sample set.
And the area determining unit is used for determining a target area from the defect areas according to the target defect type, wherein the defect type of the target area is different from the target defect type.
And the picture generating unit is used for generating the defect picture according to the target area.
In one embodiment, the type determining unit may include:
and the shape acquisition subunit is used for acquiring the defect shape of the defect picture in the defect sample set.
And the type determining subunit is used for determining the target defect type of the defect picture according to the defect shape.
In one embodiment, the picture generating unit may include:
and the area processing subunit is used for performing minimum circumscribed rectangle processing on the target area to obtain a rectangular area.
And a pixel setting subunit, configured to set a pixel of a boundary region to zero, where the boundary region is a region other than the target region within the rectangular region.
And the picture generation subunit is used for generating the defect picture according to each pixel of the rectangular area.
In one embodiment, the picture overlay module 602 may include:
and the picture selecting unit is used for randomly selecting a target defect picture from the defect sample set.
And the picture rotating unit is used for rotating the target defect picture in a preset mode to obtain the rotated target defect picture.
And the picture covering unit is used for randomly selecting a target position from the normal sample picture and covering the rotated target defect picture on the target position.
In one embodiment, the picture rotation unit may include:
and the information determining subunit is used for determining the rotation angle and the rotation direction in the preset mode.
And the position mapping subunit is used for performing position mapping on each pixel in the target defect picture according to the rotation angle and the rotation direction to obtain the rotated target defect picture.
In the embodiment of the application, a defect sample picture is obtained, a defect picture, namely a defect picture only containing defects, is generated according to a defect area in the defect sample picture, and the defect picture is stored in a preset defect sample set. If the defect pictures in the defect sample set meet the preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures, namely obtaining a certain number of possible defect pictures of unqualified products, and finally training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures, so that the normal sample pictures are covered by the defect pictures to obtain the possible defect pictures of the unqualified products, the data of the samples are enhanced, the richness of the samples is improved, and the accuracy of model identification results is improved.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the apparatus and the module described above may refer to corresponding processes in the foregoing system embodiments and method embodiments, and are not described herein again.
Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. For convenience of explanation, only portions related to the embodiments of the present application are shown.
As shown in fig. 7, the terminal device 7 of this embodiment includes: at least one processor 700 (only one shown in fig. 7), a memory 701 connected to the processor 700, and a computer program 702, such as a data enhancement program, stored in the memory 701 and executable on the at least one processor 700. The processor 700 implements the steps of the data enhancement method embodiments, such as the steps S101 to S103 shown in fig. 1, when executing the computer program 702. Alternatively, the processor 700 implements the functions of the modules in the device embodiments, for example, the functions of the modules 601 to 603 shown in fig. 6, when executing the computer program 702.
Illustratively, the computer program 702 may be divided into one or more modules, which are stored in the memory 701 and executed by the processor 700 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 702 in the terminal device 7. For example, the computer program 702 may be divided into a picture generation module 601, a picture overlay module 602, and a training module 603, and the specific functions of the modules are as follows:
the image generating module 601 is configured to obtain a defect sample image, generate a defect image according to a defect area in the defect sample image, and store the defect image in a preset defect sample set;
a picture covering module 602, configured to randomly cover the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures if the defect pictures in the defect sample set meet a preset condition;
a training module 603, configured to train a preset model according to the defect sample pictures, the preset number of normal sample pictures, and the preset number of target sample pictures.
The terminal device 7 may include, but is not limited to, a processor 700 and a memory 701. It will be understood by those skilled in the art that fig. 7 is only an example of the terminal device 7, and does not constitute a limitation to the terminal device 7, and may include more or less components than those shown, or combine some components, or different components, such as an input-output device, a network access device, a bus, etc.
The Processor 700 may be a Central Processing Unit (CPU), and the Processor 700 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 701 may be an internal storage unit of the terminal device 7 in some embodiments, for example, a hard disk or a memory of the terminal device 7. In other embodiments, the memory 701 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 7. Further, the memory 701 may include both an internal storage unit of the terminal device 7 and an external storage device. The memory 701 is used for storing an operating system, an application program, a Boot Loader (Boot Loader), data, and other programs, such as a program code of the computer program. The above-described memory 701 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned functions may be distributed as different functional units and modules according to needs, that is, the internal structure of the apparatus may be divided into different functional units or modules to implement all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the above modules or units is only one logical function division, and there may be other division manners in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. The computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method of data enhancement, comprising:
acquiring a defect sample picture, generating a defect picture according to a defect area in the defect sample picture, and storing the defect picture in a preset defect sample set;
if the defect pictures in the defect sample set meet preset conditions, randomly covering the defect pictures in the defect sample set onto a preset number of normal sample pictures to obtain a preset number of target sample pictures;
and training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures.
2. The data enhancement method of claim 1, further comprising, after storing the defect picture in a preset set of defect samples:
and if the defect pictures in the defect sample set do not meet the preset conditions, returning to execute the step of obtaining the defect sample pictures until the defect pictures in the defect sample set meet the preset conditions.
3. The data enhancement method according to claim 1, wherein the generating of the corresponding defect picture according to the defect area in the defect sample picture comprises:
determining a target defect type of a defect picture in the defect sample set;
determining a target area from the defect area according to the target defect type, wherein the defect type of the target area is different from the target defect type;
and generating the defect picture according to the target area.
4. The data enhancement method of claim 3, wherein the determining the target defect type for the defective picture in the defect sample set comprises:
acquiring the defect shape of the defect picture in the defect sample set;
and determining the target defect type of the defect picture according to the defect shape.
5. The data enhancement method of claim 3, wherein the generating the defect picture according to the target region comprises:
performing minimum external rectangle processing on the target area to obtain a rectangular area;
setting pixels of a boundary region to be zero, wherein the boundary region is a region except for the target region in the rectangular region;
and generating the defect picture according to each pixel of the rectangular area.
6. The data enhancement method of any one of claims 1 to 5, wherein the randomly overlaying the defect pictures in the defect sample set onto a preset number of normal sample pictures comprises:
randomly selecting a target defect picture from the defect sample set;
rotating the target defect picture in a preset mode to obtain a rotated target defect picture;
and randomly selecting a target position from the normal sample picture, and covering the rotated target defect picture on the target position.
7. The data enhancement method of claim 6, wherein the rotating the target defect picture in a preset manner to obtain a rotated target defect picture comprises:
determining a rotation angle and a rotation direction in the preset mode;
and mapping the position of each pixel in the target defect picture according to the rotation angle and the rotation direction to obtain the rotated target defect picture.
8. A data enhancement device, comprising:
the image generation module is used for acquiring a defect sample image, generating a defect image according to a defect area in the defect sample image and storing the defect image in a preset defect sample set;
the image covering module is used for randomly covering the defect images in the defect sample set onto a preset number of normal sample images to obtain a preset number of target sample images if the defect images in the defect sample set meet preset conditions;
and the training module is used for training a preset model according to the defect sample pictures, the preset number of normal sample pictures and the preset number of target sample pictures.
9. A terminal device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, characterized in that said processor implements the steps of a data enhancement method according to any one of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a data enhancement method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210237047.4A CN114638294A (en) | 2022-03-10 | 2022-03-10 | Data enhancement method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210237047.4A CN114638294A (en) | 2022-03-10 | 2022-03-10 | Data enhancement method and device, terminal equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114638294A true CN114638294A (en) | 2022-06-17 |
Family
ID=81948279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210237047.4A Pending CN114638294A (en) | 2022-03-10 | 2022-03-10 | Data enhancement method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114638294A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115330803A (en) * | 2022-10-17 | 2022-11-11 | 菲特（天津）检测技术有限公司 | Surface defect data enhancement method and device, electronic equipment and storage medium |
| CN115346109A (en) * | 2022-08-02 | 2022-11-15 | 北京新岳纵横科技有限公司 | IOU (input/output Unit) strategy based enhanced sample generation method |
| CN115908402A (en) * | 2022-12-30 | 2023-04-04 | 胜科纳米(苏州)股份有限公司 | Defect analysis method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807359A (en) * | 2019-09-19 | 2020-02-18 | 江苏濠汉信息技术有限公司 | Pole tower defect inspection method based on model deployed in unmanned aerial vehicle |
| WO2020042741A1 (en) * | 2018-08-27 | 2020-03-05 | 北京百度网讯科技有限公司 | Battery detection method and device |
| CN111047576A (en) * | 2019-12-12 | 2020-04-21 | 珠海博明视觉科技有限公司 | Surface defect sample generation tool |
| CN112446429A (en) * | 2020-11-27 | 2021-03-05 | 广东电网有限责任公司肇庆供电局 | CGAN (Carrier grade Access network) -based routing inspection image data small sample expansion method |
| CN112906769A (en) * | 2021-02-04 | 2021-06-04 | 国网河南省电力公司电力科学研究院 | Power transmission and transformation equipment image defect sample amplification method based on cycleGAN |
| CN114065874A (en) * | 2021-11-30 | 2022-02-18 | 河北省科学院应用数学研究所 | Medicine glass bottle appearance defect detection model training method and device and terminal equipment |
-
2022
- 2022-03-10 CN CN202210237047.4A patent/CN114638294A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020042741A1 (en) * | 2018-08-27 | 2020-03-05 | 北京百度网讯科技有限公司 | Battery detection method and device |
| CN110807359A (en) * | 2019-09-19 | 2020-02-18 | 江苏濠汉信息技术有限公司 | Pole tower defect inspection method based on model deployed in unmanned aerial vehicle |
| CN111047576A (en) * | 2019-12-12 | 2020-04-21 | 珠海博明视觉科技有限公司 | Surface defect sample generation tool |
| CN112446429A (en) * | 2020-11-27 | 2021-03-05 | 广东电网有限责任公司肇庆供电局 | CGAN (Carrier grade Access network) -based routing inspection image data small sample expansion method |
| CN112906769A (en) * | 2021-02-04 | 2021-06-04 | 国网河南省电力公司电力科学研究院 | Power transmission and transformation equipment image defect sample amplification method based on cycleGAN |
| CN114065874A (en) * | 2021-11-30 | 2022-02-18 | 河北省科学院应用数学研究所 | Medicine glass bottle appearance defect detection model training method and device and terminal equipment |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115346109A (en) * | 2022-08-02 | 2022-11-15 | 北京新岳纵横科技有限公司 | IOU (input/output Unit) strategy based enhanced sample generation method |
| CN115330803A (en) * | 2022-10-17 | 2022-11-11 | 菲特（天津）检测技术有限公司 | Surface defect data enhancement method and device, electronic equipment and storage medium |
| CN115330803B (en) * | 2022-10-17 | 2023-01-24 | 菲特（天津）检测技术有限公司 | Surface defect data enhancement method and device, electronic equipment and storage medium |
| CN115908402A (en) * | 2022-12-30 | 2023-04-04 | 胜科纳米(苏州)股份有限公司 | Defect analysis method and device, electronic equipment and storage medium |
| CN115908402B (en) * | 2022-12-30 | 2023-10-03 | 胜科纳米(苏州)股份有限公司 | Defect analysis method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110060237B (en) | Fault detection method, device, equipment and system | |
| CN114638294A (en) | Data enhancement method and device, terminal equipment and storage medium | |
| CN109886928B (en) | Target cell marking method, device, storage medium and terminal equipment | |
| CN112348765A (en) | Data enhancement method and device, computer readable storage medium and terminal equipment | |
| CN108090486B (en) | Image processing method and device in billiard game | |
| CN110443212B (en) | Positive sample acquisition method, device, equipment and storage medium for target detection | |
| CN109840883B (en) | Method and device for training object recognition neural network and computing equipment | |
| CN110570442A (en) | Contour detection method under complex background, terminal device and storage medium | |
| US20230214989A1 (en) | Defect detection method, electronic device and readable storage medium | |
| CN111290684B (en) | Image display method, image display device and terminal equipment | |
| CN112396050B (en) | Image processing method, device and storage medium | |
| CN111191582A (en) | Three-dimensional target detection method, detection device, terminal device and computer-readable storage medium | |
| CN113609984A (en) | Pointer instrument reading identification method and device and electronic equipment | |
| CN112559341A (en) | Picture testing method, device, equipment and storage medium | |
| CN112348778A (en) | Object identification method and device, terminal equipment and storage medium | |
| CN111680704A (en) | Automatic and rapid extraction method and device for newly-increased human active plaque of ocean red line | |
| CN112634201A (en) | Target detection method and device and electronic equipment | |
| CN112634235A (en) | Product image boundary detection method and electronic equipment | |
| CN112419207A (en) | Image correction method, device and system | |
| CN112330597A (en) | Image difference detection method and device and computer equipment | |
| CN110110697B (en) | Multi-fingerprint segmentation extraction method, system, device and medium based on direction correction | |
| CN109389628B (en) | Image registration method, apparatus and storage medium | |
| CN114821274A (en) | Method and device for identifying state of split and combined indicator | |
| CN114359048A (en) | Image data enhancement method and device, terminal equipment and storage medium | |
| CN113269752A (en) | Image detection method, device terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220617 |