CN113192061A - LED package appearance detection image extraction method and device, electronic equipment and storage medium - Google Patents
LED package appearance detection image extraction method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113192061A CN113192061A CN202110571748.7A CN202110571748A CN113192061A CN 113192061 A CN113192061 A CN 113192061A CN 202110571748 A CN202110571748 A CN 202110571748A CN 113192061 A CN113192061 A CN 113192061A
- Authority
- CN
- China
- Prior art keywords
- point
- image
- material sheet
- single material
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 89
- 238000003860 storage Methods 0.000 title claims abstract description 7
- 238000000605 extraction Methods 0.000 title claims description 12
- 239000000463 material Substances 0.000 claims abstract description 80
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000012937 correction Methods 0.000 claims abstract description 16
- 230000005856 abnormality Effects 0.000 claims abstract description 13
- 238000004806 packaging method and process Methods 0.000 claims abstract description 9
- 238000005520 cutting process Methods 0.000 claims abstract description 8
- 238000012216 screening Methods 0.000 claims abstract description 8
- 230000002159 abnormal effect Effects 0.000 claims abstract description 7
- 239000002245 particle Substances 0.000 claims abstract description 4
- 238000012545 processing Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 6
- 230000000295 complement effect Effects 0.000 claims description 4
- 230000001502 supplementing effect Effects 0.000 claims description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims 1
- 230000007547 defect Effects 0.000 abstract description 15
- 238000007689 inspection Methods 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 239000003292 glue Substances 0.000 description 4
- 238000012858 packaging process Methods 0.000 description 4
- 238000003708 edge detection Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000002989 correction material Substances 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30121—CRT, LCD or plasma display
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image detection, in particular to a method and a device for extracting an LED packaging appearance detection image, electronic equipment and a storage medium; the method comprises the steps of preliminarily detecting the position of a single material sheet in image data by adopting a target detection method, and rotationally correcting an image according to the detected position of the single material sheet; detecting the position of the single material sheet after the rotation correction again by a target detection method; then screening out abnormal point positions and point positions with more deviations through abnormality detection; filling the centers of the vacant positions or seriously damaged material sheets by a point set matching method; finally, calculating the global coordinate of the single material sheet, and extracting the single material sheet picture through fixed cutting size; the invention avoids the situations that the detection of the center position of a single particle slice becomes unstable, the deviation from the center position is more, and even the detection cannot be carried out under the condition that the defects exist in the existing method; the accuracy and the stability of extracting the LED packaging appearance detection image are improved.
Description
Technical Field
The invention relates to the technical field of image detection, in particular to a method and a device for extracting an LED package appearance detection image, electronic equipment and a storage medium.
Background
An LED (Light-emitting diode) is widely used in the fields of indication, display, and the like due to its advantages of long life, low power consumption, and the like. Reliability, stability and high light extraction are factors that must be considered in replacing existing illumination sources with LEDs. The packaging process is one of the main factors influencing the function of the LED, and the key procedures of the packaging process comprise frame mounting, pressure welding and packaging. Due to the packaging process, a plurality of defects exist in the LED packaging process. Wherein the apparent surface defects are typically accomplished by visual inspection and manual inspection. The traditional vision is difficult to solve the situation of complex lines, the missing detection and false detection rate is high, and the detection is unstable; the manual detection usually subjectively judges the flaw defect, and once the detection time is too long, the situations of fatigue, false detection, missing detection and the like occur, so that the working efficiency and the accuracy rate are reduced. The deep learning obtains good effect on feature extraction and positioning, can be more accurate to the defect detection field, and has better stability. The extraction of the single material sheet is an important step of deep learning in defect detection application, and the extraction of all single small materials can be accurately found and has important significance for defect detection.
Currently, the center position of the material sheet can be found through threshold segmentation and template matching, but the accuracy and the stability have inherent defects.
Disclosure of Invention
The invention aims to provide an extraction method of an LED package appearance detection image, which can stably and accurately extract a picture of a single material sheet in the LED package appearance detection image.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for extracting an LED package appearance detection image comprises the following steps:
preliminarily detecting the position of a single material sheet in the image data by adopting a target detection method, and rotationally correcting the image according to the detected position of the single material sheet;
detecting the position of the single material sheet after the rotation correction again by a target detection method;
then screening out abnormal point positions and point positions with more deviations through abnormality detection;
filling the centers of the vacant positions or seriously damaged material sheets by a point set matching method;
and finally, calculating the global coordinate of the single material sheet, and extracting the single material sheet picture through the fixed cutting size.
In a further technical scheme, before the position of a single material sheet is preliminarily detected, image data is subjected to scaling processing so as to accelerate the image processing speed.
In a further technical scheme, the target detection method comprises the steps of training a model capable of identifying a material sheet by using a deep learning network, and detecting a zoomed image to identify the central position of a single material sheet;
specifically, certain original data is collected firstly, and then image enhancement and filtering processing are carried out on target data; then, carrying out data annotation by using a material sheet in the software picture, simultaneously labeling an object which is easy to be confused, cutting and zooming the image during labeling, and labeling the processed picture; and finally, performing data training to obtain a model capable of stably detecting a target circumscribed rectangle, wherein the model uses a YoloV5 network structure, and a loss function used in the training is the sum of a regression frame error and a class error, wherein the corresponding formulas of the loss function OD _ loss, the regression frame error IOU _ loss and the class error CLS _ loss are shown in formulas (1), (2) and (3):
OD_loss＝IOU_loss+CLS_loss (1)
CLS_loss＝-αt(1-pt)γlog(pt)-(1-αt)(pt)γlog(1-pt) (2)
in a further technical scheme, the rotation correction is to use the coordinates of the central point of a single material sheet detected by the target detection method, calculate the deflection angle of the whole image by calculating the angle between the position of an adjacent near point and the horizontal presentation, and perform rotation correction on the image through affine transformation.
In a further technical scheme, in the abnormality detection step, points outside the ROI area in the image are screened, then all point locations within the ROI are sorted according to the abscissa and the ordinate respectively, the point locations are considered to be in the same row when the ordinate between the two points is smaller than 1/4 row spacing, the point locations are considered to be in the same column when the abscissa between the two points is smaller than 1/4 column spacing, the coordinates in the same row and the same column are averaged respectively, and the point locations deviating from 1/10 single-particle sheet width in the same row are removed.
In a further technical scheme, the point set matching method comprises the steps of point set manufacturing, matching according to a point set and bit complementing;
the point set is manufactured by searching for a complete picture to ensure that all central points can be detected through a target, arranging all point positions of the material sheet according to a first-row and later-row ordering mode, and recording all the central points in a point set file;
when matching and bit complementing are carried out according to the point set, the point set in the point set file is loaded firstly, the point set template is moved to the position same as the detected central point by comparing the first row and the first column coordinates of the point set template and the detected central point, and the moving formula is shown as follows (x)template,ytemplate) Is the coordinate of the point set template, (x)pos,ypos) Detecting all detected coordinates in the picture; dxIs the distance of the abscissa of the template from the detected center point, DyThe vertical coordinate distance between the template and the detected central point.
Sequencing the detected point sets and the point set template in the same front-to-back mode, traversing the point set template and the detected point sets, comparing the point set template with the detected point sets, if the distance between a point on the point set template and the detected point is less than that of two tablets, screening out, determining that the point is missed, and filling the centers of the missed tablets in the point set template;
finally, correcting the center point of the complementary position through the detected position of the center point of the material sheet, (x)i,yi) (x) center coordinates of the undetected webi-1,yi),(xi+1,yi),(xi,yi-1),(xi,yi+1) The coordinates of the center of the web near the center of the undetected web are corrected by the formula:
the invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the method for extracting the LED package appearance detection image.
The invention also provides a computer readable storage medium, on which a computer program is stored, wherein the computer program is executed by a processor to implement the above-mentioned method for extracting an image of LED package appearance inspection.
Compared with the prior art, the invention has the following technical effects:
according to the method for extracting the LED package appearance detection image, the position of a single material sheet is identified by adopting a target detection method, so that the problems that the detection of the central position of the single material sheet is not stable enough, the deviation of the central position is more, and even the detection cannot be carried out under the condition that the defects exist, such as multiple glue, glue overflow or serious damage and the like in the LED package process, and the detection omission of the defects of the single material sheet is finally caused under the condition that a threshold value segmentation method, an edge detection separation method or a template matching method is adopted in the prior art are solved; the accuracy and the stability of extracting the LED packaging appearance detection image are improved.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Drawings
Fig. 1 is a flowchart illustrating an extraction method of an LED package appearance inspection image according to an embodiment of the present invention;
FIG. 2 shows a cropped scaled view of a training size for a target detection image;
FIG. 3 is a network structure diagram of YoloV5 in the present invention;
FIG. 4 shows an inspection raw image for the appearance of a certain LED package;
FIG. 5 is a schematic view showing the center position of a single web of FIG. 4 as detected by an object detection method;
FIG. 6 shows the image of FIG. 5 after the center point is corrected by rotation;
fig. 7 is a schematic diagram showing the position of a single web finally extracted from the original image for the appearance inspection of the LED package in fig. 4.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further clarified by combining the specific drawings.
The prior art provides a plurality of methods for specific application of image segmentation, the center position of a material sheet can be found through threshold segmentation, edge detection segmentation and template matching, but the accuracy and the stability have inherent defects. The basic principle of the traditional threshold segmentation method is that 1) the background and the foreground of a gray image are separated through a threshold, so that a target is found, and in application, when defects exist in a material sheet, such as multiple glue, glue overflow, serious damage and the like, the defects are easily influenced by noise and brightness, the detection of the center of a single material sheet becomes unstable, the center is deviated more, and even the defects cannot be detected, so that the defect detection omission of a single chip is finally caused; 2) the basic principle of the edge detection segmentation method is to determine detection boundaries of different regions according to drastic changes of pixel values, images of the same region often have the same color and texture distribution, so that the edges of a material sheet can be detected, but a large amount of edge information exists in an actual material sheet, the edges of the material sheet and a background cannot be well distinguished, and therefore material sheet position information cannot be accurately positioned. Meanwhile, the traditional algorithm needs multiple detection, so that the image preprocessing is complex and the time consumption is long. 3) The basic principle of the template matching method is that a template picture is subjected to sliding window on a target picture, pixel characteristic points on the template picture and the target picture are compared, if the similarity is within a certain range, the sliding window area is determined to be a target area, and a circumscribed rectangle and a center of the target area are output.
In view of this, the conventional image segmentation method is not well applicable to image segmentation of LEDs, and the technical idea of the present invention is to detect as many points as possible by using a deep learning target detection method, appropriately correct an image, screen detected abnormal central points by abnormality detection, and complement undetected points by point set matching, so that all small tablet images can be accurately and stably extracted.
Specifically, as shown in fig. 1, the present invention provides a method for extracting an image of LED package appearance detection, where the method includes the following steps:
s1: zooming the obtained LED package appearance detection image to accelerate the image processing speed;
the technical personnel in the field know that the picture shot by the industrial camera usually has very high definition and resolution, if the original picture is directly processed, the time is very long, and a large amount of computing resources are occupied, therefore, in the method provided by the invention, the obtained LED packaging appearance detection image is preferably subjected to zooming processing, the size of the image is reduced, the occupation of the computing resources is reduced, and the image processing speed is accelerated;
s2: preliminarily detecting the position of a single material sheet in the image data by adopting a target detection method, and rotationally correcting the image according to the detected position of the single material sheet;
s3: detecting the position of the single material sheet after the rotation correction again by a target detection method;
s4: screening out abnormal point positions and point positions with more deviations through abnormality detection;
s5: filling the centers of the vacant positions or seriously damaged material sheets by a point set matching method;
s6: and finally, calculating the global coordinate of the single material sheet, and extracting the single material sheet picture through the fixed cutting size.
According to the method provided by the invention, the target detection method comprises the steps of training a model capable of identifying a material sheet by using a deep learning network, and detecting an image after zooming to identify the central position of a single material sheet;
specifically, certain original data is collected firstly, and then image enhancement and filtering processing are carried out on target data; then, carrying out data annotation by using a material sheet in the software picture, simultaneously labeling an object which is easy to be confused, cutting and zooming the image during labeling, and labeling the processed picture;
specifically, as shown in fig. 2, a schematic diagram of cropping and scaling of a training size of a target detection image is shown, and by performing cropping and scaling processing on the image, labeling cost and training time are effectively reduced.
Finally, data training is carried out to obtain a model capable of stably detecting a target circumscribed rectangle, the model uses a YoloV5 network structure, specifically, as shown in fig. 3, the network structure diagram of YoloV5 in the invention is shown, a loss function used in training is the sum of a regression frame error and a class error, wherein the corresponding formulas of the loss function OD _ loss, the regression frame error IOU _ loss and the class error CLS _ loss are formulas (1), (2) and (3):
OD_loss＝IOU_loss+CLS_loss (1)
CLS_loss＝-αt(1-pt)γlog(pt)-(1-αt)(pt)γlog(1-pt) (2)
the target detection method provided by the invention is used for detecting the central position of the single material sheet, so that the central position of the single material sheet is more accurately detected, and the detection stability is better. When the target detection model is used, different networks, such as YoloV3, PP-Yolo and the like can be used to achieve the same effect, and the positions of the single material sheets in different scenes can be found.
According to the method provided by the invention, the rotation correction is to use the center point coordinates of a single material sheet detected by the target detection method, calculate the deflection angle of the whole picture by calculating the angle between the adjacent near point position and the horizontal presentation, and perform rotation correction on the image through affine transformation.
Specifically, (x)1,y1),(x2,y2)...(xn,yn) The calculation formula of the rotation correction angle of the coordinate of the center point of the material sheet detected by the target detection method is as follows:
wherein theta is the rotation correction angle, and n is the number of detected small materials.
The method for acquiring the deflection angle uses the position relation of the centers of all material sheets, calculates the angle relative to the coordinate axis from the center point to the coordinate information of the center point apart from the center point by 3 material sheets and stores the angle, and the 3 sigma criterion screens the data and then averages the data to acquire the relatively accurate rotation correction angle. According to the invention, all the detection input pictures have the same angle and visual effect by rotating the correction material sheet.
According to the method provided by the invention, in the abnormality detection step, points outside the ROI area in the image are screened, then all point locations within the ROI are sorted according to the abscissa and the ordinate respectively, the point locations are considered to be in the same row when the ordinate between the two points is smaller than 1/4 row spacing, the point locations are considered to be in the same column when the abscissa between the two points is smaller than 1/4 column spacing, the coordinates in the same row and the same column are averaged respectively, and the point locations deviating from 1/10 single-particle sheet width in the same row are removed. According to the method and the device, the required point positions are screened out through anomaly detection, the point positions with anomaly detection are eliminated, and the matching precision and stability of the subsequent point sets are improved.
According to the method provided by the invention, the point set matching method comprises the steps of point set manufacturing, matching according to the point set and bit supplementing;
the point set is manufactured by searching for a complete picture to ensure that all central points can be detected through a target, arranging all point positions of the material sheet according to a first-row and later-row ordering mode, and recording all the central points in a point set file;
when matching and bit complementing are carried out according to the point set, the point set in the point set file is loaded firstly, the point set template is moved to the position same as the detected central point by comparing the first row and the first column coordinates of the point set template and the detected central point, and the moving formula is shown as follows (x)template,ytemplate) Is the coordinate of the point set template, (x)pos,ypos) Detecting all detected coordinates in the picture; dxIs the distance of the abscissa of the template from the detected center point, DyThe vertical coordinate distance between the template and the detected central point.
Sequencing the detected point sets and the point set template in the same front-to-back mode, traversing the point set template and the detected point sets, comparing the point set template with the detected point sets, if the distance between a point on the point set template and the detected point is less than that of two tablets, screening out, determining that the point is missed, and filling the centers of the missed tablets in the point set template;
finally, correcting the center point of the complementary position through the detected position of the center point of the material sheet, (x)i,yi) (x) center coordinates of the undetected webi-1,yi),(xi+1,yi),(xi,yi-1),(xi,yi+1) The coordinates of the center of the web near the center of the undetected web are corrected by the formula:
in order to further show the extraction method of the LED package appearance detection image, a picture of a part of an LED chip is selected for display. The original picture in fig. 4 shows, the center position of the material sheet preliminarily detected by the target detection method is shown in fig. 5, it can be seen that the center position detected by the target detection method is relatively accurate and stable, and fig. 6 shows the result of the image re-detection after the rotation correction, which ensures the uniformity of the image formation of the single material sheet. Finally, the final result obtained by point set matching is shown in fig. 7, and image extraction can be performed on a plurality of web positions on a single picture.
The invention also provides an extraction device of the LED packaging appearance detection image, which comprises an image data acquisition module, an image detection module, an abnormality detection module, a bit-filling correction module and an output module; the image data acquisition module is used for acquiring an LED package appearance detection image and zooming the image to accelerate the image processing speed; the image detection module is used for preliminarily detecting the position of a single material sheet to obtain the coordinates of the center point of the single material sheet; the abnormality detection module is used for carrying out abnormality detection on the acquired coordinates of the center point of the single material sheet and eliminating the position of the abnormal single material sheet; the position-supplementing correction module is used for supplementing all the single material pieces on the LED packaging appearance detection image and correcting the coordinates; the output module is used for calculating the global coordinates of the single material pieces and extracting all the single material piece pictures through cutting.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the method for extracting the LED package appearance detection image.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described method for extracting an LED package appearance inspection image.
The foregoing shows and describes the general principles, essential features, and inventive features of this invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (9)
1. A method for extracting an LED package appearance detection image is characterized by comprising the following steps:
preliminarily detecting the position of a single material sheet in the image data by adopting a target detection method, and rotationally correcting the image according to the detected position of the single material sheet;
detecting the position of the single material sheet after the rotation correction again by a target detection method;
then screening out abnormal point positions and point positions with more deviations through abnormality detection;
filling the centers of the vacant positions or seriously damaged material sheets by a point set matching method;
and finally, calculating the global coordinate of the single material sheet, and extracting the single material sheet picture through the fixed cutting size.
2. A method according to claim 1, wherein the image data is scaled to increase the image processing speed prior to preliminary detection of individual sheet positions.
3. The method according to claim 1, wherein the target detection method comprises training a model capable of identifying a material sheet by using a deep learning network, and detecting the scaled image to identify the center position of a single material sheet;
specifically, certain original data is collected firstly, and then image enhancement and filtering processing are carried out on target data; then, using software to label data of the material sheets in the pictures, labeling confusable targets, cutting and zooming the images during labeling, and labeling the processed pictures; and finally, performing data training to obtain a model capable of stably detecting a target circumscribed rectangle, wherein the model uses a YoloV5 network structure, and a loss function used in the training is the sum of a regression frame error and a class error, wherein the corresponding formulas of the loss function OD _ loss, the regression frame error IOU _ loss and the class error CLS _ loss are shown in formulas (1), (2) and (3):
OD_loss＝IOU_loss+CLS_loss (1)
CLS_loss＝-αt(1-pt)γlog(pt)-(1-αt)(pt)γlog(1-pt) (2)
4. the method according to claim 1, wherein the rotation correction is to use the coordinates of the center point of a single material sheet detected by the target detection method, calculate the deflection angle of the whole image by calculating the angle between the position of the adjacent near point and the horizontal, and perform rotation correction on the image through affine transformation.
5. The method according to claim 1, wherein the abnormality detecting step includes the steps of screening out points outside the ROI area in the image, sorting all point locations within the ROI according to an abscissa and an ordinate, determining that the point locations are in the same row when the ordinate between the two points is smaller than 1/4 line spacing, determining that the point locations are in the same column when the abscissa between the two points is smaller than 1/4 column spacing, averaging the coordinates in the same row and the same column, and removing the point locations deviating from 1/10 single-particle width in the same row.
6. The method of claim 1, wherein the point set matching method comprises making a point set, matching and bit complementing according to the point set;
the point set is manufactured by searching for a complete picture to ensure that all central points can be detected through a target, arranging all point positions of the material sheet according to a first-row and later-row ordering mode, and recording all the central points in a point set file;
when matching and bit complementing are carried out according to the point set, the point set in the point set file is loaded firstly, the point set template is moved to the position same as the detected central point by comparing the first row and the first column coordinates of the point set template and the detected central point, and the moving formula is shown as follows (x)template,ytemplate) Is the coordinate of the point set template, (x)pos,ypos) Detecting all detected coordinates in the picture; dxIs the distance of the abscissa of the template from the detected center point, DyThe vertical coordinate distance between the template and the detected central point is taken as the distance between the template and the detected central point;
sequencing the detected point sets and the point set template in the same front-to-back mode, traversing the point set template and the detected point sets, comparing the point set template with the detected point sets, if the distance between a point on the point set template and the detected point is less than that of two tablets, screening out, determining that the point is missed, and filling the centers of the missed tablets in the point set template;
finally, correcting the center point of the complementary position through the detected position of the center point of the material sheet, (x)i,yi) As the coordinates of the center of the web that is not detected,(xi-1,yi),(xi+1,yi),(xi,yi-1),(xi,yi+1) The coordinates of the center of the web near the center of the undetected web are corrected by the formula:
7. an extraction device for an LED package appearance detection image is characterized by comprising:
the image data acquisition module is used for acquiring an LED packaging appearance detection image and zooming the image to accelerate the image processing speed;
the image detection module is used for preliminarily detecting the position of a single material sheet to obtain the center point coordinates of the single material sheet;
the abnormality detection module is used for carrying out abnormality detection on the acquired coordinates of the center point of the single material sheet and eliminating the position of the abnormal single material sheet;
the position supplementing and correcting module is used for supplementing and correcting all single material pieces on the LED packaging appearance detection image and correcting coordinates;
and the output module is used for calculating the global coordinates of the single material pieces and extracting all the single material piece pictures through clipping.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1-6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110571748.7A CN113192061B (en) | 2021-05-25 | 2021-05-25 | Extraction method and device of LED package appearance detection image, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110571748.7A CN113192061B (en) | 2021-05-25 | 2021-05-25 | Extraction method and device of LED package appearance detection image, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113192061A true CN113192061A (en) | 2021-07-30 |
| CN113192061B CN113192061B (en) | 2023-08-29 |
Family
ID=76984927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110571748.7A Active CN113192061B (en) | 2021-05-25 | 2021-05-25 | Extraction method and device of LED package appearance detection image, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113192061B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463272A (en) * | 2021-12-31 | 2022-05-10 | 江西省通讯终端产业技术研究院有限公司 | LED (light emitting diode) massive sorting method, system and equipment based on machine vision and hyperspectral imaging technology |
| CN116777906A (en) * | 2023-08-17 | 2023-09-19 | 常州微亿智造科技有限公司 | Abnormality detection method and abnormality detection device in industrial detection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| CN108776140A (en) * | 2018-06-08 | 2018-11-09 | 福建新大陆电脑股份有限公司 | A kind of printed matter flaw detection method and system based on machine vision |
| CN111161243A (en) * | 2019-12-30 | 2020-05-15 | 华南理工大学 | Industrial product surface defect detection method based on sample enhancement |
| CN112053318A (en) * | 2020-07-20 | 2020-12-08 | 清华大学 | Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning |
-
2021
- 2021-05-25 CN CN202110571748.7A patent/CN113192061B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| CN108776140A (en) * | 2018-06-08 | 2018-11-09 | 福建新大陆电脑股份有限公司 | A kind of printed matter flaw detection method and system based on machine vision |
| CN111161243A (en) * | 2019-12-30 | 2020-05-15 | 华南理工大学 | Industrial product surface defect detection method based on sample enhancement |
| CN112053318A (en) * | 2020-07-20 | 2020-12-08 | 清华大学 | Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463272A (en) * | 2021-12-31 | 2022-05-10 | 江西省通讯终端产业技术研究院有限公司 | LED (light emitting diode) massive sorting method, system and equipment based on machine vision and hyperspectral imaging technology |
| CN116777906A (en) * | 2023-08-17 | 2023-09-19 | 常州微亿智造科技有限公司 | Abnormality detection method and abnormality detection device in industrial detection |
| CN116777906B (en) * | 2023-08-17 | 2023-11-14 | 常州微亿智造科技有限公司 | Abnormality detection method and abnormality detection device in industrial detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113192061B (en) | 2023-08-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111179225B (en) | Test paper surface texture defect detection method based on gray gradient clustering | |
| CN111161243B (en) | Industrial product surface defect detection method based on sample enhancement | |
| WO2021168733A1 (en) | Defect detection method and apparatus for defect image, and computer-readable storage medium | |
| CN109472271B (en) | Printed circuit board image contour extraction method and device | |
| CN109767445B (en) | High-precision PCB defect intelligent detection method | |
| JP2018522293A (en) | Visual identification method of multi-type BGA chip based on linear clustering | |
| WO2021109697A1 (en) | Character segmentation method and apparatus, and computer-readable storage medium | |
| CN113706490B (en) | Wafer defect detection method | |
| CN115131348B (en) | Method and system for detecting textile surface defects | |
| CN113298769B (en) | FPC flexible flat cable appearance defect detection method, system and medium | |
| CN110047063B (en) | Material drop detection method, device, equipment and storage medium | |
| CN113192061A (en) | LED package appearance detection image extraction method and device, electronic equipment and storage medium | |
| CN113034464A (en) | Method for detecting defects of liquid crystal display in real time under multiple backgrounds | |
| CN110570422A (en) | Capsule defect visual detection method based on matrix analysis | |
| CN112308854A (en) | Automatic detection method and system for chip surface flaws and electronic equipment | |
| CN110781913A (en) | Zipper cloth belt defect detection method | |
| CN118501177A (en) | Appearance defect detection method and system for formed foil | |
| CN113112396B (en) | Method for detecting conductive particles | |
| CN114708234A (en) | Method and device for identifying number of detonators on automatic bayonet coding all-in-one machine | |
| CN117237747B (en) | Hardware defect classification and identification method based on artificial intelligence | |
| CN112419225B (en) | SOP type chip detection method and system based on pin segmentation | |
| CN110516725B (en) | Machine vision-based wood board stripe spacing and color detection method | |
| CN108898584B (en) | Image analysis-based full-automatic veneered capacitor welding polarity discrimination method | |
| CN112069974A (en) | Image recognition method and system for recognizing defects of components | |
| CN114581447B (en) | Conveying belt deviation identification method and device based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |