CN117538658A - Artificial intelligence fault positioning method and device based on infrared spectrum and thermal imaging - Google Patents
Artificial intelligence fault positioning method and device based on infrared spectrum and thermal imaging Download PDFInfo
- Publication number
- CN117538658A CN117538658A CN202311530095.3A CN202311530095A CN117538658A CN 117538658 A CN117538658 A CN 117538658A CN 202311530095 A CN202311530095 A CN 202311530095A CN 117538658 A CN117538658 A CN 117538658A
- Authority
- CN
- China
- Prior art keywords
- area
- fault
- infrared spectrum
- grinding
- thermal imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000002329 infrared spectrum Methods 0.000 title claims abstract description 76
- 238000001931 thermography Methods 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 16
- 238000000227 grinding Methods 0.000 claims abstract description 77
- 238000001514 detection method Methods 0.000 claims abstract description 24
- 230000002159 abnormal effect Effects 0.000 claims abstract description 18
- 238000001228 spectrum Methods 0.000 claims abstract description 17
- 238000007599 discharging Methods 0.000 claims abstract description 7
- 239000000428 dust Substances 0.000 claims description 23
- 239000000523 sample Substances 0.000 claims description 21
- 238000010586 diagram Methods 0.000 claims description 19
- 238000013527 convolutional neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 9
- 244000137852 Petrea volubilis Species 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 7
- 238000012512 characterization method Methods 0.000 claims description 6
- 238000005498 polishing Methods 0.000 claims description 6
- 230000003595 spectral effect Effects 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000010884 ion-beam technique Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 239000000463 material Substances 0.000 claims 2
- 238000012546 transfer Methods 0.000 claims 2
- 230000004807 localization Effects 0.000 claims 1
- 238000003701 mechanical milling Methods 0.000 claims 1
- 238000000992 sputter etching Methods 0.000 claims 1
- 238000009792 diffusion process Methods 0.000 description 5
- 238000004566 IR spectroscopy Methods 0.000 description 3
- 230000009102 absorption Effects 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 2
- 238000007517 polishing process Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000003331 infrared imaging Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N25/00—Investigating or analyzing materials by the use of thermal means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
- G01R31/28—Testing of electronic circuits, e.g. by signal tracer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
- G01R31/28—Testing of electronic circuits, e.g. by signal tracer
- G01R31/2801—Testing of printed circuits, backplanes, motherboards, hybrid circuits or carriers for multichip packages [MCP]
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
- G01R31/28—Testing of electronic circuits, e.g. by signal tracer
- G01R31/2801—Testing of printed circuits, backplanes, motherboards, hybrid circuits or carriers for multichip packages [MCP]
- G01R31/2806—Apparatus therefor, e.g. test stations, drivers, analysers, conveyors
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Pathology (AREA)
- Immunology (AREA)
- Biochemistry (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- General Engineering & Computer Science (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Computer Hardware Design (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Testing Or Measuring Of Semiconductors Or The Like (AREA)
Abstract
The invention relates to an artificial intelligence fault positioning method, namely a device, based on infrared spectrum and infrared thermal imaging, comprising the following steps: s1, constructing a fault positioning device, wherein the fault positioning device comprises an upper computer, a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area which are sequentially distributed near a conveyor belt in the conveying direction; s2, the IC carrier plate enters from a feeding area and is transmitted to an infrared spectrum detection area to electrify the carrier plate, and after the infrared spectrum is scanned, the infrared spectrum is uploaded to an upper computer in real time to carry out spectrum anomaly identification; s3, the IC carrier plate is transmitted into a grinding area from an infrared spectrum detection area, and the upper computer only grinds the carrier plate in the identified abnormal infrared spectrum area; s4, the ground fault IC carrier plate is transferred into a thermal imaging area for artificial intelligent identification, and fault positioning is completed. The invention realizes the further precision of the grinding area, the identification of suspicious faults and the accurate positioning of the fault area after grinding.
Description
Technical Field
The invention relates to a fault positioning method and a fault positioning device, in particular to an artificial intelligence fault positioning method and a fault positioning device based on infrared spectrum and infrared thermal imaging of a fault IC carrier plate, belonging to the field of intelligent detection of electronic products.
Background
The prior art generally predicts suspected fault areas through impedance test results, but the detection is complicated, because the impedance test involves testing a plurality of components, and therefore, a rough fault range needs to be analyzed by combining a layout, and the positioning is often inaccurate, especially for complex circuits. A major feature of IC component failure is abnormal heating, resulting in at least one heat source on the IC board. Thus spectrally producing anomalies in the near and infrared bands.
Therefore, infrared imaging becomes a means for accurately determining the occurrence position of the IC carrier plate in the prior art. But the suspected faulty areas need to be ground, which creates a risk of damaging the non-faulty areas. Although appropriate, mature grinding strategies can overcome this drawback. However, the time cost of research and development is obviously large, but the advantage of the research and development is obviously low, and the accurate position distribution of the heat source is displayed to the maximum extent. Because the rate of thermal diffusion may be such that the imaging rate does not follow if not ground, the heat source may not be precisely determined to a reduced extent.
Therefore, how to utilize the advantages and disadvantages of the prior art and to improve the accuracy and the recognition efficiency of fault location as a whole becomes a problem to be solved urgently.
Disclosure of Invention
In view of the analysis problems in the prior art, the invention considers the following points that firstly, the impedance test is replaced by the spectrum technology, so that a suspected fault area is obtained rapidly, and secondly, the fault position is accurately identified by using the continuously optimized intelligent algorithm by utilizing the difference between the artificial mark and the artificial intelligent prediction.
Based on the above consideration, one aspect of the present invention provides an artificial intelligence fault locating method based on infrared spectrum and infrared thermal imaging, comprising the steps of:
s1, constructing a fault positioning device, wherein the fault positioning device comprises an upper computer, and a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area which are sequentially distributed near a conveyor belt in the conveying direction;
s2, the IC carrier plate enters from a feeding area and is transmitted to an infrared spectrum detection area by a conveyor belt, the carrier plate is electrified, the infrared spectrum is scanned in a preset time starting and stopping period, and the infrared spectrum is uploaded to an upper computer in real time to perform spectrum anomaly identification;
s3, the IC carrier plate is transmitted into a grinding area from an infrared spectrum detection area, and the upper computer only carries out a multi-head grinding device for controlling the grinding area on the carrier plate in the identified abnormal infrared spectrum area so as to grind the abnormal infrared spectrum area;
s4, transmitting the ground fault IC carrier plate into a thermal imaging area, transmitting a thermal imaging image of the electrified fault IC carrier plate into an upper computer, performing artificial intelligent recognition, and completing fault positioning.
The infrared spectrum detection area in step S2 includes an infrared spectrum scanner and an energizing probe, which are disposed in a first camera bellows including an inlet and an outlet, and the energizing probe is used to lap the probe with the IC carrier board transported in place, so as to complete energizing. Preferably, the preset time starting and stopping period is 3-20s after power-on to 30s-3min after power-on.
The multi-head grinding device in the step S3 comprises a two-dimensional moving frame and multi-head grinding heads arranged on a lifting frame at the top of the two-dimensional moving frame, wherein each grinding head is respectively provided with sand paper with different numbers.
Preferably, the upper computer judges whether the abrasive paper needs to be replaced or not according to the working time of each grinding head on the multi-head grinding head, and sends out replacement early warning in the period before the abrasive paper needs to be replaced.
Preferably, the grinding area is provided with a dust suction device controlled by an upper computer, and ground objects on the surface of the fault IC carrier plate in the grinding process and after the grinding is finished are sucked.
Therefore, preferably, an IC board limiting device controlled by the upper computer is further arranged in the grinding area, so that the faulty IC carrier board can be stopped from being conveyed when passing, and the position movement of the faulty IC carrier board caused by dust collection is limited, and a dust collection opening of the dust collection device is arranged right above the IC carrier board and can move.
Optionally, the limiting device is a pair of limiting clamps sequentially arranged in the conveying direction.
The thermal imaging zone in step S4 includes a thermal imager placed in a second camera that also has an outlet and an inlet.
When the fault positioning is identified as abnormal, the abnormal fault IC carrier plate is pushed away from the conveyor belt through the sorting pushing device in the blanking area, further subsequent characterization analysis is carried out, and the normal IC carrier plate is collected and stored manually or by a manipulator.
The method for identifying the spectrum anomaly in the step S2 comprises the following steps:
p1 ranges the band of the spectrum from 4000 cm to 400cm -1 Dividing the spectrum intensity in each wave band into a plurality of wave bands according to a preset step length, taking an arithmetic average value of the intensities at two end points of the wave band, normalizing the arithmetic average value in all the wave bands, and mapping the arithmetic average value into RGB color pixels;
p2, arranging the mapped pixels into a test rectangular chart from short to long according to the arithmetic mean value of the wavelengths, or from long to short, and when the test rectangular chart is empty with the first camera bellows, differentiating the scanned background infrared spectrum by the background rectangular chart formed by the steps P1 and P2 to form a differential chart;
and P3, constructing a convolutional neural network, and acquiring a differential graph formed by a plurality of IC carrier plates with different fault conditions (namely faults at different positions and non-faults) correspondingly, wherein the differential graph is divided into a training set and a verification set, and the convolutional neural network is optimized through training of the training set so as to finish the acquisition of a convolutional neural network model C for spectrum anomaly identification.
Preferably, the step size is 0.5cm -1 -1cm -1 。
Preferably, the convolutional neural network is a convolutional neural network ResNet with a residual mechanism.
The method for artificial intelligence recognition in the step S4 comprises the following steps:
q1 obtains thermal imaging diagrams corresponding to a plurality of different fault IC carrier boards, and manually marks a fault area a by comparing the structure diagram of the IC carrier boards with the thermal imaging diagrams,
q2, pseudo-colorizing each thermal imaging graph to form a temperature distribution pseudo-colorizing graph, setting temperature thresholds of all areas on the IC carrier plate, respectively binarizing corresponding areas on the temperature distribution pseudo-colorizing graph, and identifying a contour area p of a fault area by utilizing edge detection; q3 inputting the contour region p into the RoiAlign layer, obtaining a prediction frame p 'through a full-connection layer FC, calculating contour error loss (such as average gap value of p' and a 'in a preset direction) by using a smooth (p', a ') function by an artificial frame a' of the fault region a, adjusting the parameters of the RoiAlign layer by back propagation, carrying out frame regression to correct the prediction frame, enabling the contour error to be minimum, and stopping training to obtain a fault positioning model D;
and Q4, inputting the thermal imaging diagram to be detected processed in the step Q2 into the fault positioning model D, and outputting a prediction result.
It can be appreciated that the prediction frame and the artificial mark are close, so that the finally trained model D has the marking habit of a person, thereby more embodying the experience of the person, further accurately reflecting the accurate range of the actual heat source without the need of a heat diffusion range, and further accurately positioning the range.
In step S4, the step of completing the fault location includes registering the prediction result to the fault IC carrier board map and/or the image map of the ground fault IC carrier board.
In the above method, preferably, the current of the energization is 10mA or less.
Is pushed off the conveyor belt, and further performs subsequent characterization analysis specifically including: and performing longitudinal section sectioning treatment on the leakage fault position according to the registration result to expose a specific fault position, and then performing characterization analysis on the specific fault position by using a microscope.
Preferably, the longitudinal section sectioning treatment is performed by adopting a focused ion beam, ion grinding or mechanical grinding mode.
It is another object of the present invention to provide an artificial intelligence fault locating device based on infrared spectroscopy and infrared thermal imaging, comprising: the upper computer is sequentially distributed in the conveying direction in a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area near the conveying belt, wherein,
the infrared spectrum detection area comprises an infrared spectrum scanner and an energizing probe which are arranged in a first camera bellows comprising an inlet and an outlet, wherein the energizing probe is used for overlapping the probe with an IC carrier plate transported in place so as to complete energizing;
the multi-head grinding device comprises a two-dimensional moving frame and multi-head grinding heads arranged on a lifting frame at the top of the two-dimensional moving frame, wherein each grinding head is respectively provided with sand paper with different numbers;
the thermal imaging area comprises a thermal imager placed in a second camera bellows;
the upper computer carries out spectrum anomaly identification according to the received real-time infrared spectrum, only carries out a multi-head grinding device for controlling a grinding area on the carrier plate in the identified region of the anomaly infrared spectrum so as to grind the region of the anomaly infrared spectrum, carries out artificial intelligent identification according to a thermal imaging diagram of the electrified fault IC carrier plate, and completes fault positioning. Preferably, the infrared spectrum scanner is reflective.
Preferably, the upper computer judges whether the abrasive paper needs to be replaced or not according to the working time of each grinding head on the multi-head grinding head, and sends out replacement early warning in the period before the abrasive paper needs to be replaced.
Preferably, the grinding area is provided with a dust suction device controlled by an upper computer, and ground objects on the surface of the fault IC carrier plate in the grinding process and after the grinding is finished are sucked.
Therefore, preferably, an IC board limiting device controlled by the upper computer is further arranged in the grinding area, so that the faulty IC carrier board can be stopped from being conveyed when passing, and the position movement of the faulty IC carrier board caused by dust collection is limited, and a dust collection opening of the dust collection device is arranged right above the IC carrier board and can move.
Optionally, the limiting device is a pair of limiting clamps sequentially arranged in the conveying direction.
Advantageous effects
1. Locating the suspected fault source through infrared spectrum, further refining the grinding area,
2. and identifying suspicious faults and accurately positioning the fault areas after grinding through the artificial intelligent model.
Drawings
Figure 1 a flow chart of a fault location method of the invention of embodiment 1 based on artificial intelligence of infrared spectroscopy and infrared thermal imaging,
FIG. 2 is a schematic diagram showing the configuration of a fault locating device constructed in step S1 of the artificial intelligence fault locating method based on infrared spectrum and infrared thermal imaging in embodiment 1 of the present invention,
FIG. 3 is a schematic diagram showing the configuration of the infrared spectrum and the energized probes in the first camera bellows,
figure 4 is a schematic diagram of a lifting frame structure arranged on a two-dimensional moving frame X pair of the grinding area,
fig. 5 shows a schematic view of the configuration of the limit clamp, i.e. the dust collection device, arranged in the v region of the grinding area,
figure 6 is a flowchart of a method for spectral anomaly identification in step S2 of example 2 of the present invention,
FIG. 7 is a flowchart of the method of artificial intelligence recognition in step S4 of step S2 of embodiment 2 of the present invention.
Detailed Description
Example 1
The embodiment provides a specific fault positioning method of artificial intelligence based on infrared spectrum and infrared thermal imaging, as shown in fig. 1, comprising the following steps:
s1, constructing a fault positioning device, wherein the fault positioning device comprises an upper computer, and a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area which are sequentially distributed near a conveyor belt in the conveying direction;
s2, the IC carrier plate enters from a feeding area and is transmitted to an infrared spectrum detection area by a conveyor belt, the carrier plate is electrified, the infrared spectrum is scanned in a preset time starting and stopping period, and the infrared spectrum is uploaded to an upper computer in real time to perform spectrum anomaly identification;
s3, the IC carrier plate is transmitted into a grinding area from an infrared spectrum detection area, and the upper computer only carries out a multi-head grinding device for controlling the grinding area on the carrier plate in the identified abnormal infrared spectrum area so as to grind the abnormal infrared spectrum area;
s4, transmitting the ground fault IC carrier plate into a thermal imaging area, transmitting a thermal imaging image of the electrified fault IC carrier plate into an upper computer, performing artificial intelligent recognition, and completing fault positioning.
As shown in fig. 2, the fault positioning device comprises a manipulator in a loading area, and the upper computer is used for controlling and grabbing the positioned IC plate in the bearing plate. The manipulator is arranged at one part on two sides of the conveyor belt, and the IC carrier plates are alternately arranged on the conveyor belt.
First enters the infrared spectrum detection zone and enters the first camera bellows through a first camera bellows entrance (not shown in fig. 2). As shown in fig. 3, in the first camera bellows, the energizing probes on both sides of the conveyor belt are provided, and the probes are lapped on the IC carrier board and are connected to a power supply (not shown in fig. 3), thereby achieving energization. A reflective infrared spectrum scanner is disposed above the conveyor belt. Two-dimensional scanning is achieved by controlling the position of the scan (e.g., with another two-dimensional moving gantry). And the scan position with the abnormal infrared spectrum was recorded. The scanning position can be determined by determining the position on the IC carrier board to be scanned at a certain moment through coordinate positioning in the scanning process of the upper control infrared spectrum scanner.
Wherein, the electrified probe realizes lap joint and disconnection through simple one-dimensional linear motion of lowering and lifting. The IC carrier is positioned to a fixed location, specifically by a position sensor, at which time the conveyor belt pauses the conveyance. The upper computer commands the electrified probe to be put down and just switch on the power supply. After 5s, triggering an infrared spectrum scanner to scan, completing scanning within 1min, and commanding the electrified probe to lift up and disconnect to cut off the power.
And outputting an infrared spectrum detection area at the outlet of the first camera bellows, wherein the upper computer records the IC carrier plate with the abnormal infrared spectrum. Second, only the areas of the IC carrier plate having the anomalous infrared spectrum are polished as the next station passes through the polishing zone.
Specifically, the upper computer controls the two-dimensional moving frame of the grinding area to be reset in advance to move in the numbering sequence of a plurality of areas (not shown in the figures) preset on the IC carrier plate, so that the normal area is skipped, and the grinding head is suspended in the area with abnormal infrared spectrum.
Specifically, as shown in fig. 2 and 4, the two-dimensional moving frame has a Y-moving pair (Y-direction is shown in fig. 2), an X-moving pair (X-direction is shown in fig. 4), and a lifting frame mounted on the X-moving pair, the lifting frame including a vertical rail extending longitudinally, i.e., in a Z-direction (as shown in fig. 4) and capable of reciprocating with the X-pair, a lateral slider provided on the vertical rail, and a T-shaped frame provided at the bottom of one end of the lateral slider. Two sides of the T-shaped frame are respectively provided with a grinding head to form a multi-head grinding head. One of the polishing heads carries coarse sand paper, and the other carries fine sand paper.
The multi-head grinding head suspended in the infrared spectrum abnormal region is provided with an upper computer for controlling the running position, and the pre-set regions are sequentially subjected to rough grinding and fine grinding procedures. As shown in fig. 2, a pair of limit jigs are provided in the conveyor region indicated by the letter V in the grinding region, in order in the conveying direction. As shown in fig. 5, when the IC carrier moves to replace the V-zone, the upper computer controls the limit clamp to clamp the ground IC carrier, and controls the dust suction device arranged right above the limit clamp, the dust suction port of the dust suction device moves right above the IC carrier, and the ground objects are sucked away.
Then, the thermal imaging area is entered, and the thermal imaging area enters from the inlet (the outlet is not shown in fig. 2) of the second camera bellows, and the upper computer controls the thermal imager to open so as to perform thermal imaging on the ground fault IC board. After completion, referring again to fig. 2, the second camera bellows is transported by the conveyor belt out of the outlet into the blanking area. For the fault IC carrier plate, the fault IC carrier plate is withdrawn through a sorting pushing device arranged in the area and falls into an abnormal collecting area (shown as 'abnormal' in figure 2) beside the conveyor belt; and normal IC carriers are collected and stored by a robot (the robot in the blanking area is not shown in fig. 2).
Example 2
This embodiment describes an embodiment of the method for spectral anomaly identification in step S2 and the method for artificial intelligence identification in step S4 in embodiment 1.
As shown in fig. 6, the former includes the steps of:
p1 ranges the band of the spectrum from 4000 cm to 400cm -1 (the present embodiment intercepts a part of the range thereof as an exemplary illustration) according to a preset 0.5cm -1 Step size, dividing into a plurality of wave bands. The figure shows the initial band t0 and a certain intermediate band tx (x is between 0 and the natural number n) of a strong absorption peak, and the final band tn.
The spectral intensities within each band are averaged mathematically over the intensities at the two ends of the band. In fig. 6, the intensities at the two end points of the band are given by way of example for the t0 and tx bands, in this example the absorptions a11, a12 and Ax1, ax2. And the arithmetic mean over all bands is normalized and mapped to RGB color pixels.
The pixels corresponding to t0, tx, tn are shown in fig. 6.
P2, arranging the mapped pixels into a test rectangular chart from the right to the left and from the top to the bottom according to the sequence from the short to the long of the arithmetic mean value of the wavelengths, and differentiating the rectangular chart with a background infrared spectrum scanned when the box environment is empty by the background rectangular chart formed by the steps P1 and P2 to form a differential chart;
p3, constructing a convolutional neural network, dividing a differential graph formed by a plurality of IC carrier plates with different fault conditions into a training set and a verification set, and training and optimizing the convolutional neural network ResNet with a residual error mechanism through the training set to finish obtaining a convolutional neural network model C for identifying spectrum anomalies.
As shown in fig. 7, the latter includes the steps of:
q1 obtains thermal imaging diagrams corresponding to a plurality of different fault IC carrier boards, and manually marks a fault area a by comparing the structure diagram of the IC carrier boards with the thermal imaging diagrams,
q2, pseudo-colorizing each thermal imaging graph to form a temperature distribution pseudo-colorizing graph, setting temperature thresholds of all areas on the IC carrier plate, respectively binarizing corresponding areas on the temperature distribution pseudo-colorizing graph, and identifying a contour area p of a fault area by utilizing edge detection;
in fig. 7, R1, R2 and two areas are marked, and these two areas may also be preset areas in embodiment 1. The temperature thresholds are denoted T1, T2, respectively. The circled area in the structure of the IC carrier plate in FIG. 7 is a rear view corresponding to the outline of a'. The layer of the heat diffusion area can be quickly known through the IC carrier plate structure diagram, so that the heat source, namely the heat diffusion position, is accurately marked.
And Q3, inputting the contour region p into the RoiAlign layer, obtaining a prediction frame p 'through the full-connection layer FC, calculating contour error loss by utilizing a smooth (p', a ') function by using an artificial frame a' of the fault region a, adjusting RoiAlign layer parameters by back propagation, carrying out frame regression to correct the prediction frame, enabling the contour error to be minimum, stopping training, and obtaining a fault positioning model D. I.e. when the smoothfunction value is still decreasing, training is continued until the region is stable and no longer decreases, and the step Q4 is entered.
And Q4, inputting the thermal imaging diagram to be detected processed in the step Q2 into the fault positioning model D, and outputting a prediction result.
It can be appreciated that the prediction frame and the artificial mark are close, so that the finally trained model D has the marking habit of a person, thereby more embodying the experience of the person, further accurately reflecting the accurate range of the actual heat source without the need of a heat diffusion range, and further accurately positioning the range.
In step S4, the step of completing the fault location includes registering the prediction result to the morphology photograph of the ground fault IC carrier board.
Example 3
This embodiment will explain an artificial intelligence fault locating device based on infrared spectroscopy and infrared thermal imaging. As shown in fig. 2, it includes: the upper computer is sequentially distributed in the conveying direction in a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area near the conveying belt, wherein,
the infrared spectrum detection area comprises an infrared spectrum scanner and a power-on probe which are arranged in a first camera bellows comprising an inlet and an outlet, wherein the power-on probe (shown in figure 3) is used for overlapping the probe with an IC carrier transported in place so as to complete power-on;
the multi-head grinding device comprises a two-dimensional moving frame and multi-head grinding heads arranged on a lifting frame at the top of the two-dimensional moving frame, wherein each grinding head is respectively provided with sand paper with different numbers;
the thermal imaging zone includes a thermal imager (not shown in fig. 2) placed in a second camera;
the upper computer carries out spectrum anomaly identification according to the received real-time infrared spectrum, only carries out a multi-head grinding device for controlling a grinding area on the carrier plate in the identified region of the anomaly infrared spectrum so as to grind the region of the anomaly infrared spectrum, carries out artificial intelligent identification according to a thermal imaging diagram of the electrified fault IC carrier plate, and completes fault positioning. The infrared spectrum scanner is reflective (fig. 3).
The upper computer judges whether the abrasive paper needs to be replaced according to the working time of each grinding head on the multi-head grinding head, and sends out replacement early warning in a period before the abrasive paper needs to be replaced.
As shown in fig. 5, the polishing area is provided with a dust suction device controlled by an upper computer, and the polished objects on the surface of the faulty IC carrier plate are sucked during the polishing process and after the polishing process is completed. The grinding area is also provided with an IC board limiting device controlled by the upper computer so as to stop conveying when the fault IC carrier board passes by and limit the position movement of the fault IC carrier board caused by dust collection, and a dust collection port of the dust collection device is arranged right above the IC carrier board and can move. The limiting device is a pair of limiting clamps which are sequentially arranged in the conveying direction.
Claims (24)
1. The fault positioning method of artificial intelligence based on infrared spectrum and infrared thermal imaging is characterized by comprising the following steps: s1, constructing a fault positioning device, wherein the fault positioning device comprises an upper computer, and a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area which are sequentially distributed near a conveyor belt in the conveying direction;
s2, the IC carrier plate enters from a feeding area and is transmitted to an infrared spectrum detection area by a conveyor belt, the carrier plate is electrified, the infrared spectrum is scanned in a preset time starting and stopping period, and the infrared spectrum is uploaded to an upper computer in real time to perform spectrum anomaly identification;
s3, the IC carrier plate is transmitted into a grinding area from an infrared spectrum detection area, and the upper computer only carries out a multi-head grinding device for controlling the grinding area on the carrier plate in the identified abnormal infrared spectrum area so as to grind the abnormal infrared spectrum area;
s4, transmitting the ground fault IC carrier plate into a thermal imaging area, transmitting a thermal imaging image of the electrified fault IC carrier plate into an upper computer, performing artificial intelligent recognition, and completing fault positioning.
2. The method of claim 1, wherein the infrared spectrum detection zone in step S2 comprises an infrared spectrum scanner and a powered probe disposed in a first camera box comprising an inlet and an outlet, the powered probe being configured to overlap the probe with an IC carrier board transported in place to complete the powering.
3. The method of claim 2, wherein the predetermined time start-stop period is from 3-20s after power-up to 30s-3min after power-up.
4. The method according to claim 1 or 2, wherein the multi-head grinding apparatus in step S3 comprises a two-dimensional moving frame, and a multi-head grinding head provided on a top lift of the two-dimensional moving frame, wherein each grinding head carries a different number of sandpaper thereon.
5. The method of claim 4, wherein the host computer determines whether the sandpaper needs to be replaced according to the working time of each grinding head on the multi-head grinding head, and sends out a replacement early warning in a period before the replacement is required.
6. The method of claim 5, wherein the polishing zone has a vacuum device controlled by a host computer to suck down material from the surface of the faulty IC carrier during and after polishing.
7. The method of claim 4 or 5, wherein an IC board limiting device controlled by a host computer is further provided in the polishing area to stop the transfer of the faulty IC carrier board while passing, and to limit the positional movement thereof due to dust collection, and a dust collection port of the dust collection device is disposed so as to be movable directly above the IC carrier board.
8. The method of claim 7, wherein the stop device is a pair of stop clamps disposed sequentially in the conveying direction.
9. The method of claim 8, wherein the thermal imaging zone in step S4 comprises a thermal imager placed in a second camera that also has an outlet and an inlet.
10. The method of any one of claims 1-3,5,6,8 or 9, wherein when the fault location is identified as abnormal, the abnormal faulty IC carrier is pushed off the conveyor belt by a sorting pushing device in the blanking area, and further subsequent characterization analysis is performed, and the normal IC carrier is collected and stored manually or by a manipulator.
11. The method according to claim 10, wherein the method of spectral anomaly identification in step S2 comprises the steps of:
p1 ranges the band of the spectrum from 4000 cm to 400cm -1 Dividing the spectrum intensity in each wave band into a plurality of wave bands according to a preset step length, taking an arithmetic average value of the intensities at two end points of the wave band, normalizing the arithmetic average value in all the wave bands, and mapping the arithmetic average value into RGB color pixels;
p2, arranging the mapped pixels into a test rectangular chart from short to long according to the arithmetic mean value of the wavelengths, or from long to short, and when the test rectangular chart is empty with the first camera bellows, differentiating the scanned background infrared spectrum by the background rectangular chart formed by the steps P1 and P2 to form a differential chart;
and P3, constructing a convolutional neural network, dividing a differential graph formed by a plurality of IC carrier plates with different fault conditions into a training set and a verification set, and training and optimizing the convolutional neural network through the training set to finish obtaining a convolutional neural network model C for identifying spectrum anomalies.
12. The method of claim 11, wherein the step size is 0.5cm -1 -1cm -1 。
13. The method of claim 12, wherein the convolutional neural network is a convolutional neural network res net with a residual mechanism.
14. The method of claim 13, wherein the method of artificial intelligence recognition in step S4 comprises: q1 obtains thermal imaging diagrams corresponding to a plurality of different fault IC carrier boards, and manually marks a fault area a by comparing the structure diagram of the IC carrier boards with the thermal imaging diagrams,
q2, pseudo-colorizing each thermal imaging graph to form a temperature distribution pseudo-colorizing graph, setting temperature thresholds of all areas on the IC carrier plate, respectively binarizing corresponding areas on the temperature distribution pseudo-colorizing graph, and identifying a contour area p of a fault area by utilizing edge detection; q3, inputting the contour region p into the RoiAlign layer, obtaining a prediction frame p 'through the full-connection layer FC, calculating contour error loss by utilizing a smooth (p', a ') function by using an artificial frame a' of the fault region a, adjusting RoiAlign layer parameters by back propagation, carrying out frame regression to correct the prediction frame, enabling the contour error to tend to be minimum, and stopping training to obtain a fault positioning model D;
and Q4, inputting the thermal imaging diagram to be detected processed in the step Q2 into the fault positioning model D, and outputting a prediction result.
15. The method according to claim 14, wherein in step S4, the performing fault localization comprises registering the prediction result to the faulty IC carrier map and/or the image map of the faulty IC carrier after lapping.
16. The method of any one of claims 1-3,5,6,8,9, 12-15, wherein the energized current is 10mA or less.
17. The method according to any one of claims 11-15, wherein the pushing off from the conveyor belt, further performing a subsequent characterization analysis, in particular, comprises: and performing longitudinal section sectioning treatment on the leakage fault position according to the registration result to expose a specific fault position, and then performing characterization analysis on the specific fault position by using a microscope.
18. The method of claim 17, wherein the longitudinal section sectioning is performed by focused ion beam, ion milling, or mechanical milling.
19. An artificial intelligence fault locating device based on infrared spectrum and infrared thermal imaging, comprising: the upper computer is sequentially distributed in the conveying direction in a feeding area, an infrared spectrum detection area, a grinding area, a thermal imaging area and a discharging area near the conveying belt, wherein,
the infrared spectrum detection area comprises an infrared spectrum scanner and an energizing probe which are arranged in a first camera bellows comprising an inlet and an outlet, wherein the energizing probe is used for overlapping the probe with an IC carrier plate transported in place so as to complete energizing;
the multi-head grinding device comprises a two-dimensional moving frame and multi-head grinding heads arranged on a lifting frame at the top of the two-dimensional moving frame, wherein each grinding head is respectively provided with sand paper with different numbers;
the thermal imaging area comprises a thermal imager placed in a second camera bellows;
the upper computer carries out spectrum anomaly identification according to the received real-time infrared spectrum, only carries out a multi-head grinding device for controlling a grinding area on the carrier plate in the identified region of the anomaly infrared spectrum so as to grind the region of the anomaly infrared spectrum, carries out artificial intelligent identification according to a thermal imaging diagram of the electrified fault IC carrier plate, and completes fault positioning.
20. The fault locating device of claim 19, wherein the infrared spectrum scanner is reflective.
21. The fault locating device according to claim 20, wherein the upper computer judges whether the sand paper needs to be replaced according to the working time of each grinding head on the multi-head grinding head, and sends out a replacement early warning in a period before the sand paper needs to be replaced.
22. The fault location device of claim 21, wherein the grinding area has a dust suction device controlled by a host computer to suck ground material from the surface of the faulty IC carrier during and after grinding.
23. The fault location device of claim 22, wherein an IC board limiting device controlled by the host computer is further provided in the grinding area to stop the transfer of the faulty IC carrier board while passing, and to limit the positional movement thereof due to dust suction, and a dust suction port of the dust suction device is disposed to be movable directly above the IC carrier board.
24. The fault locating device of claim 23, wherein the limiting device is a pair of limiting clamps arranged in sequence in the conveying direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311530095.3A CN117538658A (en) | 2023-11-16 | 2023-11-16 | Artificial intelligence fault positioning method and device based on infrared spectrum and thermal imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311530095.3A CN117538658A (en) | 2023-11-16 | 2023-11-16 | Artificial intelligence fault positioning method and device based on infrared spectrum and thermal imaging |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117538658A true CN117538658A (en) | 2024-02-09 |
Family
ID=89785579
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311530095.3A Pending CN117538658A (en) | 2023-11-16 | 2023-11-16 | Artificial intelligence fault positioning method and device based on infrared spectrum and thermal imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117538658A (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140226150A1 (en) * | 2013-02-12 | 2014-08-14 | Zygo Corporation | Surface topography interferometer with surface color |
| US9417309B1 (en) * | 2013-11-27 | 2016-08-16 | Xilinx, Inc. | Test structure and method for calibrating three-dimensional thermography fault isolation tool |
| US20160314573A1 (en) * | 2015-04-27 | 2016-10-27 | Thermoteknix Systems Limited | Conveyor Belt Monitoring System and Method |
| CN109781268A (en) * | 2019-03-16 | 2019-05-21 | 福州大学 | Keypoint part temperature monitoring system in a kind of switchgear based on the infrared thermovision technology of low cost |
| CN110334661A (en) * | 2019-07-09 | 2019-10-15 | 国网江苏省电力有限公司扬州供电分公司 | Infrared power transmission and transformation abnormal heating point target detecting method based on deep learning |
| CN110502997A (en) * | 2019-07-23 | 2019-11-26 | 平安科技（深圳）有限公司 | Method and relevant device are surveyed in agricultural insurance based on artificial intelligence |
| CN111630356A (en) * | 2017-12-01 | 2020-09-04 | 兰斯大学 | Method for characterizing a sample using a neural network |
| CN111751002A (en) * | 2020-05-15 | 2020-10-09 | 国网浙江省电力有限公司嘉兴供电公司 | Intelligent charged equipment fault diagnosis method based on infrared thermal imaging |
| CN111798407A (en) * | 2020-05-15 | 2020-10-20 | 国网浙江省电力有限公司嘉兴供电公司 | Electrified equipment fault diagnosis method based on neural network model |
| CN111986172A (en) * | 2020-08-18 | 2020-11-24 | 华北电力科学研究院有限责任公司 | Infrared image fault detection method and device for power equipment |
| CN112381784A (en) * | 2020-11-12 | 2021-02-19 | 国网浙江省电力有限公司信息通信分公司 | Equipment detecting system based on multispectral image |
| WO2021117013A1 (en) * | 2019-12-13 | 2021-06-17 | Hearthcore S.A.P.I. De C.V. | Determination of medical condition risk using thermographic images |
| CN113466740A (en) * | 2021-07-07 | 2021-10-01 | 深圳市美信咨询有限公司 | Fault positioning method and fault positioning device |
| CN114037916A (en) * | 2021-10-26 | 2022-02-11 | 中国石油化工股份有限公司 | Visual fault identification and positioning method for high-pressure air cooler of petrochemical device |
| WO2022229413A1 (en) * | 2021-04-30 | 2022-11-03 | Universidad De Sevilla | Method for detecting and quantifying viruses in fluid samples by means of digital processing of hyperspectral images of diffuse optical reflectance obtained in the visible and near infrared ranges |
| CN116206220A (en) * | 2022-12-22 | 2023-06-02 | 万互智能通信科技研究院(南京)有限公司 | Automatic target identification analysis platform for infrared imaging |
| CN116310274A (en) * | 2023-02-27 | 2023-06-23 | 深圳供电局有限公司 | State evaluation method for power transmission and transformation equipment |
-
2023
- 2023-11-16 CN CN202311530095.3A patent/CN117538658A/en active Pending
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140226150A1 (en) * | 2013-02-12 | 2014-08-14 | Zygo Corporation | Surface topography interferometer with surface color |
| US9417309B1 (en) * | 2013-11-27 | 2016-08-16 | Xilinx, Inc. | Test structure and method for calibrating three-dimensional thermography fault isolation tool |
| US20160314573A1 (en) * | 2015-04-27 | 2016-10-27 | Thermoteknix Systems Limited | Conveyor Belt Monitoring System and Method |
| US20200333185A1 (en) * | 2017-12-01 | 2020-10-22 | Universite De Reims Champagne Ardenne | Method for characterising samples using neural networks |
| CN111630356A (en) * | 2017-12-01 | 2020-09-04 | 兰斯大学 | Method for characterizing a sample using a neural network |
| CN109781268A (en) * | 2019-03-16 | 2019-05-21 | 福州大学 | Keypoint part temperature monitoring system in a kind of switchgear based on the infrared thermovision technology of low cost |
| CN110334661A (en) * | 2019-07-09 | 2019-10-15 | 国网江苏省电力有限公司扬州供电分公司 | Infrared power transmission and transformation abnormal heating point target detecting method based on deep learning |
| CN110502997A (en) * | 2019-07-23 | 2019-11-26 | 平安科技（深圳）有限公司 | Method and relevant device are surveyed in agricultural insurance based on artificial intelligence |
| WO2021117013A1 (en) * | 2019-12-13 | 2021-06-17 | Hearthcore S.A.P.I. De C.V. | Determination of medical condition risk using thermographic images |
| CN111798407A (en) * | 2020-05-15 | 2020-10-20 | 国网浙江省电力有限公司嘉兴供电公司 | Electrified equipment fault diagnosis method based on neural network model |
| CN111751002A (en) * | 2020-05-15 | 2020-10-09 | 国网浙江省电力有限公司嘉兴供电公司 | Intelligent charged equipment fault diagnosis method based on infrared thermal imaging |
| CN111986172A (en) * | 2020-08-18 | 2020-11-24 | 华北电力科学研究院有限责任公司 | Infrared image fault detection method and device for power equipment |
| CN112381784A (en) * | 2020-11-12 | 2021-02-19 | 国网浙江省电力有限公司信息通信分公司 | Equipment detecting system based on multispectral image |
| WO2022229413A1 (en) * | 2021-04-30 | 2022-11-03 | Universidad De Sevilla | Method for detecting and quantifying viruses in fluid samples by means of digital processing of hyperspectral images of diffuse optical reflectance obtained in the visible and near infrared ranges |
| CN113466740A (en) * | 2021-07-07 | 2021-10-01 | 深圳市美信咨询有限公司 | Fault positioning method and fault positioning device |
| CN114037916A (en) * | 2021-10-26 | 2022-02-11 | 中国石油化工股份有限公司 | Visual fault identification and positioning method for high-pressure air cooler of petrochemical device |
| CN116206220A (en) * | 2022-12-22 | 2023-06-02 | 万互智能通信科技研究院(南京)有限公司 | Automatic target identification analysis platform for infrared imaging |
| CN116310274A (en) * | 2023-02-27 | 2023-06-23 | 深圳供电局有限公司 | State evaluation method for power transmission and transformation equipment |
Non-Patent Citations (4)
| Title |
|---|
| 刘云鹏等: "基于深度学习的输变电设备异常发热点红外图片目标检测方法", 南方电网技术, vol. 13, no. 2, 20 February 2019 (2019-02-20), pages 27 - 33 * |
| 李祉豪: "红外热成像技术对高压电气设备的诊断与分析", 实验室研究与探索, vol. 36, no. 4, 15 April 2017 (2017-04-15), pages 37 - 40 * |
| 覃泓宁等: "融合光谱特征和超像素的遥感建筑物分级提取", 国外电子测量技术, vol. 42, no. 3, 15 March 2023 (2023-03-15), pages 182 - 188 * |
| 陈丽萍等: "基于不同决策树的面向对象林区遥感影像分类比较", 应用生态学报, vol. 29, no. 12, 11 October 2018 (2018-10-11), pages 3995 - 4003 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10449655B2 (en) | Work polishing method and work polishing apparatus | |
| EP1995553B1 (en) | System and method for identifying a feature of a workpiece | |
| KR102607918B1 (en) | Glass sheet acquisition and positioning system | |
| KR102091165B1 (en) | Apparatus for inspecting tunnel crack | |
| KR101069881B1 (en) | Gantry and Apparatus for Forming Curved Surface of Plate Having the Same | |
| CN105787407A (en) | Method and system for automatically identifying express delivery barcode integrating machine vision and laser scanning | |
| JPH04341835A (en) | Skiving apparatus for reclaimed tire | |
| CN112198161A (en) | PVC gloves real-time detection system based on machine vision | |
| CN114705693B (en) | AI visual inspection device and system for shell structure defect detection | |
| CN207982931U (en) | Cutter based on CCD galls damage on-line measuring device | |
| CN111307812A (en) | Welding spot appearance detection method based on machine vision | |
| CN117538658A (en) | Artificial intelligence fault positioning method and device based on infrared spectrum and thermal imaging | |
| CN108422321A (en) | The polishing pad image detecting system and method for chemical mechanical grinding | |
| CN108627519A (en) | Automatic industrial vision defect-detecting equipment, flaw detection component and system | |
| JP2007333491A (en) | Visual insepction device of sheet member | |
| CN206038551U (en) | Zero component part testing device based on machine vision | |
| CN208514344U (en) | The polishing pad image detecting system of chemical mechanical grinding | |
| CN107194302A (en) | A kind of planar array type bar code scanner | |
| CN116448780A (en) | Chip defect detection device, method and equipment | |
| TWM450052U (en) | Automatic identification mark machine and apparatus for defective | |
| CN116078704A (en) | Appearance detection equipment based on CCD | |
| JPH09192983A (en) | Tool attrition amount measuring method | |
| KR101608628B1 (en) | Steel plate blanking system and method | |
| KR20210015687A (en) | Laser processing apparatus | |
| RU2494180C1 (en) | Unit for automated performance of preparatory-cutting operations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |