CN117392211B - BGA element rapid identification positioning method and system and storage medium - Google Patents
BGA element rapid identification positioning method and system and storage medium Download PDFInfo
- Publication number
- CN117392211B CN117392211B CN202311683975.4A CN202311683975A CN117392211B CN 117392211 B CN117392211 B CN 117392211B CN 202311683975 A CN202311683975 A CN 202311683975A CN 117392211 B CN117392211 B CN 117392211B
- Authority
- CN
- China
- Prior art keywords
- point set
- center
- template
- positioning result
- bga
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000003860 storage Methods 0.000 title claims abstract description 8
- 238000009826 distribution Methods 0.000 claims description 22
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 15
- 238000004422 calculation algorithm Methods 0.000 description 14
- 239000011159 matrix material Substances 0.000 description 13
- 230000009466 transformation Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 12
- 238000013519 translation Methods 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 101000717877 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) 40S ribosomal protein S11-A Proteins 0.000 description 2
- 101000717881 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) 40S ribosomal protein S11-B Proteins 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
Abstract
The invention relates to the technical field of component identification and positioning, and particularly discloses a method, a system and a storage medium for quickly identifying and positioning a BGA component, wherein the method extracts a center point set of an electrode terminal in an image of the BGA component to be detected, and utilizes the characteristic that electrodes of the BGA component are uniformly distributed in a grid array to quickly and roughly position the BGA component; the coarse positioning result is further used as an initial value for registration, so that the convergence speed can be increased, the iteration times can be reduced, the probability of the iteration falling into a local extremum can be reduced, the running speed is increased, and the accuracy is increased.
Description
Technical Field
The invention relates to the technical field of component identification and positioning, in particular to a method and a system for quickly identifying and positioning a BGA component and a storage medium.
Background
With the continuous progress of machine vision technology and motion control technology, the market puts higher and higher demands on the mounting accuracy and speed of the chip mounter, so that the time consumption and precision demands on the component recognition process of the chip mounter are also higher and higher.
The traditional BGA element detection method mainly uses a spherical grid array electrode as a characteristic to carry out rectangular fitting, template matching or point set registration to obtain the element center position and rotation angle.
The method for detecting the pose of the BGA element by utilizing the rectangular fitting mode has the advantages of high detection speed and simplicity in operation, but also has the defect that the identification stability and accuracy cannot be ensured. The method takes the center of the electrode terminal of the outer ring of the BGA element as a characteristic point, and obtains the pose estimation of the whole element by utilizing a straight line fitting or rectangular fitting mode, and the implementation process is simple and does not involve complex operation, so that the detection speed is high. However, the fitting process only adopts the center of the outer ring electrode terminal as the characteristic point, and the number of the characteristic points is small, so that the stability and the accuracy of the fitting result are not high. In addition, when the BGA component is smaller, the number of the outer ring terminals is small, and the method cannot be applied; and in the case of irregular distribution of the electrode terminals of the BGA device, the outer ring terminals may not be formed in a complete rectangle, and this method is also not applicable.
Although the template matching method well solves the problems of detection accuracy and stability, the detection process is time-consuming and the demand for computer resources is increased. The traditional template matching method can only perform position scanning and does not support the change of the rotation angle. The improved method mostly adopts a plurality of rotary templates to carry out repeated scanning so as to obtain angle information, so that the time complexity of an algorithm is changed from original O (m) to O (m) n p), the time consumption is increased in multiple, the demand on computer resources is increased, and the problem of insufficient calculation accuracy of angles exists. In addition, if an actual image is used as a template in actual detection, the element in the template image cannot be ensured to be in a 0 ° state, and a deviation exists in the rotation angle in the detection result. If the virtual template is adopted, the detection precision and accuracy are sacrificed to a certain extent, and the risks of missing detection and false detection exist. And template matching methods are often difficult to cope with the complex situations where image scaling exists. Therefore, the template matching method can not meet the requirements of the chip mounter on the detection speed and accuracy of the BGA element.
The method of registration using feature point sets, although to some extent improves the inapplicability of conventional template matching algorithms to rotation and scaling. However, the following problems still exist in the practical application of the conventional point set registration method: the point set registration process needs continuous iterative circulation, when the feature points are more, the method has large calculation amount, consumes serious time under the condition of not carrying out certain optimization, is difficult to meet the real-time requirement of the chip mounter, and has limitation in application occasions. Therefore, the requirement of high precision and high speed of the chip mounter cannot be met by simply using the traditional point set matching method.
In summary, the existing method cannot quickly and accurately realize pose detection of the BGA device.
Disclosure of Invention
The invention aims to overcome the problems in the prior art and provides a method and a system for quickly identifying and positioning a BGA element and a storage medium, which can quickly and accurately realize the pose positioning of the BGA element.
In order to achieve the above object, a first aspect of the present invention provides a method for quickly identifying and positioning a BGA device, comprising the steps of:
extracting the outline of an electrode terminal in the BGA element image, and acquiring the center coordinate of the electrode terminal as an actual terminal center point set;
the minimum circumscribed rectangle search based on the actual terminal center point set distribution grid array can completely contain candidate positions and candidate angles of all electrode terminal centers, and the average value of all candidate positions and the average value of candidate angles are used as element coarse positioning results;
calculating theoretical center coordinates of the electrode terminals of the BGA element as a template point set;
and carrying out point set registration on the template point set and the actual terminal center point set by taking the rough positioning result as an initial value to obtain a fine positioning result.
A second aspect of the present invention provides a BGA component rapid identification and positioning system, comprising:
the electrode terminal center point set extraction module is used for extracting the outline of the electrode terminal in the BGA element image and obtaining the center coordinate of the electrode terminal as an actual terminal center point set;
the coarse positioning module is used for searching the minimum circumscribed rectangle of the distributed grid array based on the actual terminal center point set, can completely contain candidate positions and candidate angles of all electrode terminal centers, and takes the average value of all candidate positions and the average value of candidate angles as an element coarse positioning result;
the template point set module is used for calculating theoretical center coordinates of the electrode terminals of the BGA element to be used as a template point set;
and the fine positioning module is used for carrying out point set registration on the template point set and the actual terminal center point set by taking the coarse positioning result as an initial value to obtain a fine positioning result.
A third aspect of the present invention provides a computer storage medium comprising:
a memory having a computer program stored thereon;
and the processor is used for executing the computer program in the memory to realize the steps of the quick identification and positioning method for the BGA component.
According to the technical scheme, the electrode terminal center point set in the image of the BGA element to be detected is extracted, the BGA element is quickly and roughly positioned by utilizing the characteristic that the electrodes of the BGA element are uniformly distributed in the grid array, and the method is small in calculated amount, simple to operate and capable of quickly obtaining the approximate pose information of the BGA element; further, the coarse positioning result is used as an initial value for registration, so that the convergence speed can be increased, the iteration times can be reduced, the probability of the iteration falling into a local extremum can be reduced, the running speed is increased, and the accuracy is increased; and correcting parameters of the template point set by using the first registration result, reconstructing the template point set by using the corrected parameters, enabling the template to be more in line with the actual situation, and performing second registration by using the first registration result as an initial value and the reconstructed template point set, so that the final result can be ensured to be more accurate. Therefore, the technical scheme of the invention has the advantages of strong anti-interference capability, high detection speed, high positioning precision and the like.
Drawings
FIG. 1 is a raw image taken of a BGA component in some embodiments of the present disclosure;
FIG. 2 is a flow diagram of identifying a position fix in accordance with some embodiments of the present disclosure;
FIG. 3 is an external profile of an electrode terminal extracted and screened in accordance with some embodiments of the present disclosure;
FIG. 4 is a coarse positioning flow chart of some embodiments of the present disclosure;
FIG. 5 is a coarse positioning result graph of some embodiments of the present disclosure;
FIG. 6 is a diagram of creating a template point set distribution map in accordance with some embodiments of the present disclosure;
FIG. 7 is an ICP iterative process point-to-point distribution for some embodiments of the present disclosure;
FIG. 8 is a schematic diagram of positioning results for some embodiments of the present disclosure;
FIG. 9 is an original image of BGA device 1 in a test example of the present invention;
fig. 10 is an original image of BGA device 2 in a test example of the present invention.
Detailed Description
The following describes the detailed implementation of the embodiments of the present invention with reference to the drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the invention, are not intended to limit the invention.
Fig. 1 shows an original image of a BGA device with electrode terminals for identification exhibiting circular bright spots under lateral light irradiation, and all terminals being distributed in a grid array. The simplest pose estimation method is to adopt a fitting method, but when elements are smaller, the number of feature points available for fitting is smaller, and the stability and accuracy of a fitting result cannot be guaranteed. If the template matching method is adopted, although the problems of accuracy and stability of the detection result can be improved, after the scaling factors of the angle, the horizontal direction and the vertical direction are considered, the consumption of time and computer resources by an algorithm is overlarge, and the requirement of quick positioning of the chip mounter cannot be well met. If the traditional feature point set registration mode is adopted, the requirement of template matching on resources can be improved to a certain extent, but the problem that stable feature points are difficult to obtain in practical application, a large number of iterations are required in the registration process, time consumption is serious when the feature points are more, and the like is solved, and the requirements of a chip mounter on high precision and high efficiency are difficult to meet at the same time. In this embodiment, the element shown in fig. 1 is taken as an object, and in order to solve the problem of rapid and accurate positioning of the BGA element, a first aspect of the embodiment of the present invention provides a method for rapid identifying and positioning of the BGA element, as shown in fig. 2, including the following steps:
(1) Extracting the outer contour of an electrode terminal in the BGA element image, and acquiring the center coordinate of the electrode terminal as an actual terminal center point set;
the extraction of the electrode terminal outer contour and electrode terminal center in the BGA component image is shown in fig. 3, and the subsequent steps are performed based on the center point set of all the electrode terminal contours in fig. 3.
(2) The minimum circumscribed rectangle search based on the actual terminal center point set distribution grid array can completely contain candidate positions and candidate angles of all electrode terminal centers, and the average value of all candidate positions and the average value of candidate angles are taken as element coarse positioning results, and the coarse positioning results are shown in fig. 5.
(3) Calculating theoretical center coordinates of the electrode terminals of the BGA element as a template point set;
calculating the pixel level size of the element parameter of the element to be detected by using the given element parameter, the camera pixel size and the element coarse positioning angle obtained in the step (2), wherein the pixel level size comprises a horizontal terminal interval, a vertical terminal interval and an external dimension;
illustratively, the center of the theoretical external dimension of the element is taken as an origin, the horizontal right direction is taken as a horizontal positive direction, and the vertical downward direction is taken as a vertical positive direction. And calculating the relative coordinate positions of the centers of all electrode terminals of the BGA element to be detected according to the relative positions of the theoretical distribution positions of the given element terminals relative to the centers of the theoretical appearance of the element and the pixel size of the camera, and taking the coordinate positions of the centers of all electrode terminals as a final template point set, wherein the creation result is shown in figure 6.
(4) And carrying out point set registration on the template point set and the actual terminal center point set by taking the rough positioning result as an initial value to obtain a fine positioning result.
Further, step (2) uses the BGA element electrode distribution rule (BGA element electrode terminals are distributed in a grid array), uses the minimum circumscribed rectangle of the theoretical electrode distribution grid to search candidate positions and candidate angles which can completely contain all electrode centers, and uses the average value of all candidate positions and the average value of candidate angles as the result of element coarse positioning. As shown in fig. 4, the method specifically comprises the following steps:
s1, determining reference position and angle search parameters: using the coordinate mean value of the actual terminal center point set of the BGA element as rough definitionReference position P for bit position search 0 Presetting an angle search range SearchR and an angle search step size stepR;
the component sucking deflection angle in the production process of the chip mounter is not more than +/-45 degrees, and the component sucking deflection angle is exemplified by taking +/-45 degrees as an angle searching range (SearchR); coarse positioning has low requirements on angle precision, so that in the example of the invention, 1 degree is taken as an angle searching step (StepR), and the current searching angle is taken as midr= -45 degrees. As shown in fig. 4, i is a count indicating whether or not the actual terminal center point set is within the minimum circumscribed rectangle, and can be regarded as the electrode terminal number.
S2, calculating the minimum circumscribed rectangle sizes (FW, FH), X and Y direction search ranges (SearchX, searchY) and search step sizes (stepX, stepY) of a central coordinate distribution grid array under a search angle midR;
further, the minimum circumscribed rectangle size #FW、FH) The calculation formula is as follows:
，
the X and Y direction search ranges (SearchX, searchY) are calculated as follows:
，
the search step (stepX, stepY) is calculated as follows:
，
FW and FH respectively represent the length and width of the minimum circumscribed rectangle pixel level, and the unit pixel;SearchX、SearchYthe search range size in the X, Y direction under the current angle is represented by a unit pixel;StepX、StepYrespectively representing the search step length in the X, Y direction under the current angle and the unit pixel;ScaleX、ScaleYrepresenting the dimensions of the picture elements identifying the camera in the transverse and longitudinal directions, respectively, i.e. the actual length of a pixel in the horizontal and vertical directions, singleBits μm/pixel;W mr 、H mr representing the theoretical length and width values of the minimum circumscribed rectangle respectively, and the unit mm;PitchN、PitchErepresenting the center distance between two adjacent electrode terminals in the horizontal and vertical directions of the BGA element respectively, and the unit is mm; d represents the diameter of the electrode terminal in mm. Wherein,
W mr =(BumpNumN-1)*PitchN+D
H mr =(BumpNumE-1)*PitchE+D
wherein,BumpNumN、BumpNumErepresenting the maximum number of terminals that can exist in the horizontal and vertical directions of the BGA component to be tested, i.e., the number of rows and columns of the electrode distribution grid, respectively.
S3, taking the central coordinate of the current X direction to obtain midX=P 0 .x-SearchX/2；P 0 X represents the reference position P 0 Is the X-direction coordinate value of SearchX, which represents the X-direction search range;
s4, taking the central coordinate of the current Y direction to be midY=P 0 .y-SearchY/2； P 0 Y respectively represent reference positions P 0 Is the Y-direction coordinate value of (2), searchY represents the Y-direction search range;
s5, calculating two center straight lines of the minimum circumscribed rectangle of the grid array under the search angle midR according to the current X, Y direction search position (midX, midY); the calculation formula is as follows:
if midr=0, the two center straight lines are respectively: x=midx, y=midy;
if midr+.0, the two center straight lines are respectively: y=k 1 x+b 1 ，y=K 2 x+b 2 Wherein, the method comprises the steps of, wherein,
K 1 =tan(midR)，K 2 =-1/K 1 ，b 1 =midY-K 1 *midX，b 2 =midY-K 2 *midX。
s6, counting the number of the actual terminal center point set in the minimum circumscribed rectangle, stopping counting when the electrode terminal center coordinates are not in the current minimum circumscribed rectangle, and entering a step S7; when the center coordinates of the electrode terminals are all in the current minimum circumscribed rectangle, adding midY into a Y-direction candidate set Ysets;
further, the manner of judging whether the center of the current electrode terminal is inside the minimum external moment of the current grid array is as follows:
calculating actual center coordinates of the current electrode terminalrealBallcenterX，realBallcenterY) Distance to the two center lines of the smallest external moment of the current grid array.
If midr=0,
d
1
=|midY-realBallcenterY|，d
2
=|midX-realBallcenterX|
if midR is not equal to 0,
wherein d 1 ，d 2 Respectively represent the actual center of the current electrode terminalrealBallcenterX， realBallcenterY) Distance to the center line in the horizontal direction and the center line in the vertical direction of the current theoretical grid array.
Reuse d 1 ，d 2 And respectively determining whether the actual center of the current electrode terminal is in the minimum external moment of the current grid array according to the relation between FW and FH. When d 1 <FH/2 and d 2 <At FW/2, the actual center of the current electrode terminal is considered to be within the theoretical grid array.
S7, taking the center coordinate of the current Y direction as midY=midY+stepY, if midY is less than or equal to P 0 y+SearchY/2, repeating the steps S5-S6, otherwise executing S8;
s8, if the number of the Ysets is increased, adding midX into the X-direction candidate set Xsets, otherwise, directly executing the step S9;
s9, taking the central coordinate midX=midX+stepX of the current X direction, if midX is less than or equal to P 0 x+SearchX/2, repeating the steps S4-S8, otherwise executing the step S10;
s10, if the number of the Xsets is increased, adding midR into the candidate angle set, otherwise, directly executing the step S11;
s11, taking the current angle midR=midR+stepR, if the midR is less than or equal to the maximum value of the preset search angle range, repeating the steps S2-S10, otherwise, executing the step S12;
s12, obtaining an angle estimation value (coarseR) in coarse positioning by using an angle average value in the candidate angle set, and obtaining a central position estimation value coordinate (CoarseX, coarseY) in coarse positioning by using a position average value in the candidate position sets Xsets and Ysets, wherein the search result is shown in FIG. 5.
The coarse positioning mode can also obtain more accurate center and angle estimation for BGA elements with empty points or interference points in the grid array, and compared with the traditional method of directly obtaining center coordinates by using a point set average value mode, the accuracy of center estimation is higher, and the stability is stronger; for the conditions of smaller BGA element, less outer ring terminals or irregular distribution of electrode terminals of the BGA element, compared with a mode of estimating pose information by using straight line fitting or external torque, the accuracy and stability of center and angle estimation are higher.
Further, in step (4), the performing point set-to-point registration on the template point set and the actual terminal center point set with the coarse positioning result as an initial value, to obtain a fine positioning result includes the following steps:
s (4) -1, carrying out point set-to-point registration on the template point set and the actual terminal center point set by adopting an ICP registration method by taking a coarse positioning result as an initial value to obtain a first fine positioning result;
the specific steps for obtaining the first accurate positioning result by registering the first ICP point set include:
1) And initializing parameters. The maximum iteration cycle number is set to be 50, the maximum average distance is 1.414Pixel, the precision is 0.001Pixel, the actual terminal center point set of the BGA element to be detected obtained in the step (1) is taken as a target point cloud { Q }, and the template point set constructed in the step (3) is taken as a source point cloud { P };
2) And transforming the source point cloud. If the loop is executed for the first time, the steps are utilized(2) Rotating and translating the source point cloud { P } to obtain a transformation point cloud { with a medium-coarse positioning resultP¢ }; if the first execution cycle is not performed, the rotation translation matrix obtained in the last cycle is utilized to carry out rotation translation on the source point cloud, and a transformation point cloud { is obtainedP¢}；
3) And searching adjacent point pairs. Searching the source point cloud transformed point cloud { in the target point cloud { Q } by using the KD treeP¢ each point p i Nearest neighbor point q of (2) j . And eliminating the point pairs with the distance larger than the theoretical terminal spacing. For a plurality of points p i With the same point q j In the pairing situation, only one group closest to the pairing situation is reserved;
4) And calculating a distance error. And calculating the average distance between adjacent point pairs, stopping iteration if the average distance is smaller than the set maximum average distance or the difference between the average distance between the adjacent point pairs of the current iteration and the average distance between the adjacent point pairs of the last iteration is smaller than the precision, and outputting the currently adopted transformation matrix. Otherwise, continuing iteration;
5) Minimizing distance errors. And calculating a rotation transformation matrix Rmatrix between adjacent point pair sets by utilizing SVD decomposition, and obtaining a distance transformation matrix Tmatrix by utilizing the rotation transformation matrix Rmatrix and the mass center between the adjacent point pair sets. And (2) returning to the step (2) if the current iteration number is smaller than the maximum iteration number, and continuing iteration. On the negative side, stopping iteration;
6) And outputting a first ICP result. And obtaining pose information of the first accurate positioning according to the rotation matrix Rmaxtrix and the translation matrix Tpatrix which are iteratively output by ICP.
Since the coarse positioning result is used as the initial value, the iteration times in the first ICP registration step are very few, in the embodiment of the invention, only 3-5 times are needed, the iteration process is performed with reference to fig. 7, from left to right, the actual terminal point set distribution and the template point set distribution condition after transformation and the average deviation size after each iteration are sequentially performed, and as can be seen from fig. 7, the iteration is stopped only by 3 times of iteration, and since the coarse positioning result is used as the initial value, the average distance error is reduced from 1.942Pixel to 0.238Pixel after the first iteration, the registration speed is very fast, and the resource consumption is low. In addition, the increased screening of the matching point pairs in the improved flow also further improves the convergence rate.
S (4) -2, correcting the terminal spacing in the element parameters by using the first fine positioning result, the actual terminal center point set and the template point set, and reconstructing a new template point set;
further, step S (4) -2 includes the following processes:
template terminal pitch correction: obtaining the corresponding relation between the template point set and the actual terminal center point set based on the first fine positioning result, obtaining the geometric relation of each point in the actual terminal center point set by using the geometric relation of each point in the template point set and the corresponding relation, and finally calculating the average value of the pixel length of the electrode terminal spacing of the BGA element in the horizontal and vertical directions by using the geometric relation of each point in the actual terminal center point set, instead of using the given element size and the camera pixel size to calculate the spacing of each point in the template point set, and recording asRemixPitchN pixel 、RemixPitchE pixel ；
Reconstructing a template point set: according toRemixPitchN pixel 、RemixPitchE pixel The relative position coordinates of the electrode terminals of the BGA component with respect to the center of the component are recalculated as a new set of template points. The center of the outline of the pixel level is taken as an origin, the horizontal right is taken as a horizontal positive direction, and the vertical downward is taken as a vertical positive direction. With corrected template terminal spacingRemixPitchN pixel 、RemixPitchE pixel And the relative position relation of the centers of the terminals is calculated again, and the relative coordinate positions of the centers of the electrode terminals of the BGA element to be detected are taken as a new template point set.
The invention uses the registration result of the first ICP point set and the pixel length of the terminal spacing of the BGA component to be detected in the template point set parameter in the horizontal and vertical directions of the actual terminal center point setPitchN pixel 、PitchE pixel And correcting, and reconstructing a template point set by using the corrected parameters.In the step (3), element parameters adopted in the construction of the template point set are calibration parameters, the element parameters are different from actual parameters, in addition, angle information obtained by coarse positioning is often not accurate enough, so that a certain deviation exists between the template constructed in the step (3) and the actual condition, and the deviation possibly affects the final registration result. Therefore, the technical scheme of the invention corrects the template point set after the first ICP point set registration, so that the template point set is more in line with the actual situation, and the accuracy and reliability of the output result are ensured.
S (4) -3, using the first fine positioning result as an initial value, and performing point set-to-point registration on the new template point set and the actual terminal center point set by adopting an ICP registration method to obtain the fine positioning result, wherein the fine positioning result is shown in FIG. 8.
The purpose of the second ICP registration is to obtain more accurate pose information, iteration times can be greatly reduced by using the first ICP registration result as an initial value, and more accurate registration results can be obtained by using a reconstructed template point set which is more close to the center of the BGA element terminal to be detected and is truly distributed as a source point cloud. The method specifically comprises the following steps:
1) And initializing parameters. The maximum number of iterative cycles is exemplarily set to 50 in this step; the maximum average distance is 0.707pixel; the precision is 0.001pixel; taking the actual center coordinate point set of the electrode terminal of the BGA element to be detected obtained in the step (1) as a target point cloud { Q }, and taking the template point set created in the step (3) as a source point cloud { P };
2) And transforming the source point cloud. If the first cycle is executed, in the step, rotation and translation transformation are performed on the source point cloud { P } by using the first ICP point set registration result, the center of the point cloud { P } is translated to the center coordinate position of the first ICP point set registration result, and then the translated point cloud is rotated by taking the translated center as the origin and the first ICP point set registration output angle as the rotation angle, so as to obtain a transformed point cloud {P'-a }; if the cycle is not executed for the first time, the rotation translation matrix obtained in the previous cycle is utilized to carry out rotation translation on the source point cloud { P } to obtain a transformation point cloud {P'}；
3) Transform point cloud {P'Neighboring point pair search in { Q } and target point cloud. Searching a transformation point cloud { in a target point cloud { Q } by using a KD treeP'Each point p in } i Nearest neighbor q of (2) j . Wherein for distances greater than the theoretical terminal pitch pixel lengthRemixPitchN pixel AndRemixPitchE pixel the point pair of the maximum value is eliminated, and a plurality of points p are obtained i With the same point q j In the pairing situation, only one group closest to the pairing situation is reserved;
4) And calculating a distance error. Calculating the average distance between the adjacent point pairs for the plurality of adjacent point pairs obtained in the step 3), and stopping iteration if the average distance is smaller than the set maximum average distance or the difference between the average distance of the current iteration and the average distance of the last iteration is smaller than the precision, and executing the step 6); otherwise, executing the step 5);
5) Minimizing distance errors. And (3) for the plurality of adjacent point pairs obtained in the step (3), calculating a rotation transformation matrix Rmatrix between two point sets corresponding to the adjacent point pairs by utilizing SVD decomposition, and obtaining a distance transformation matrix Tmatrix by utilizing the rotation transformation matrix Rmatrix and the mass center between the two point sets corresponding to the adjacent point pairs. And (2) returning to the step (2) if the current iteration number is smaller than the maximum iteration number, and continuing iteration. No side, stopping iteration, and executing step 6);
6) And outputting ICP results. And obtaining final accurate positioning pose information according to a rotation matrix Rmaxtrix and a translation matrix Tpatrix which are output by the last iteration of ICP, wherein the effect is as shown in FIG. 8.
Because the first ICP registration result is taken as an initial value, the iterative times of the second ICP registration process are very few, and only 2-3 iterations are needed in the embodiment of the invention, so that the whole time consumption is not greatly influenced. In addition, the corrected template point set distribution situation is more fit with the actual point set distribution situation, so that the accuracy of the second ICP registration process is higher, and the registration result is more accurate.
The method utilizes the BGA element electrode distribution rule, searches possible candidate positions and candidate angles by using the minimum circumscribed rectangle of the electrode terminal distribution grid array, and takes the average value of all candidate positions and candidate angles as a coarse positioning result; constructing a template feature point set by using the given nominal size of the element and the camera scale; and finally, performing two times of point set registration by using an improved ICP point set registration algorithm, wherein the first time of registration uses a coarse positioning result as an initial value, the distance between the template point sets is corrected after the first time of registration is completed, and the second time of registration uses the first time of registration result as an initial value, so that the accurate pose information of the element is finally obtained.
Based on the same inventive concept, a second aspect of the embodiment of the present invention provides a BGA device rapid identification and positioning system, including:
the electrode terminal center point set extraction module is used for extracting the outline of the electrode terminal in the BGA element image and obtaining the center coordinate of the electrode terminal as an actual terminal center point set;
the coarse positioning module is used for searching the minimum circumscribed rectangle of the distributed grid array based on the actual terminal center point set, can completely contain candidate positions and candidate angles of all electrode terminal centers, and takes the average value of all candidate positions and the average value of candidate angles as an element coarse positioning result;
the template point set module is used for calculating theoretical center coordinates of the electrode terminals of the BGA element to be used as a template point set;
and the fine positioning module is used for carrying out point set registration on the template point set and the actual terminal center point set by taking the coarse positioning result as an initial value to obtain a fine positioning result.
Based on the same inventive concept, a third aspect of an embodiment of the present invention provides a computer storage medium, comprising:
a memory having a computer program stored thereon;
and the processor is used for executing the computer program in the memory to realize the steps of the quick identification and positioning method for the BGA component.
In summary, the technical scheme of the invention utilizes the characteristic that the electrode terminals of the BGA element all show grid array distribution to roughly position the BGA element, and the method has simple operation and small calculated amount and can quickly acquire the approximate pose information of the BGA element. In the invention, the two ICP point set registration processes are adopted in the fine positioning stage, wherein the coarse positioning result is used as an initial value in the first ICP registration, so that the iteration times are greatly reduced, the iteration is prevented from sinking into a local extremum, the running speed is improved, and the accuracy is increased; and before the second ICP registration, the template is corrected and reconstructed by using the first ICP registration result, so that the template is ensured to be more in line with the actual distribution condition. The second ICP registration takes the first ICP registration result as an initial value, so that the iteration times are greatly reduced, and the reconstructed template point set is taken as a source point set, so that the accuracy and reliability of the identification result are ensured. In the other two ICP registration processes, the traditional ICP flow is improved, the functions of screening out abnormal points and screening out one-to-many conditions are added, the possibility that iteration falls into a local extremum is further reduced, and the convergence speed is increased.
Therefore, the technical scheme provided by the invention has the advantages of high detection speed, high positioning accuracy, strong anti-interference capability and the like.
Test example:
and (3) verifying and experimental analyzing the reliability and the effectiveness of the BGA element rapid and accurate positioning and identifying method.
In order to verify the reliability and effectiveness of the method provided by the invention, the following verification experiment is designed:
the experiment uses BGA component images shot by YAMAHA brand YS12 equipment as sample images and uses the identification result of YAMAHA brand YS12 equipment as a standard value. The sample image is identified by utilizing the algorithm flow provided by the invention, and the difference between the identification result and the standard value of the algorithm provided by the invention is used as the evaluation value of the experimental result.
The experiment was performed with two different sizes of BGA components. As shown in fig. 9, the BGA device 1 has an external dimension of 4mm by 4mm, a terminal arrangement of 6*6 grid, a nominal terminal diameter of 0.3mm, and a center-to-center spacing of 0.5mm in both horizontal and vertical directions. As shown in fig. 10, the BGA device 2 has an external dimension of 10.5mm by 10.5mm, a terminal arrangement of 15 x 15 grids, no terminals at the middle part of the grids, a nominal terminal diameter of 0.3mm, and a center-to-center spacing of 0.65mm between the horizontal and vertical terminals. During the experiment, 50 images were acquired for each BGA device, with BGA device 1 image size 512pixel x 245pixel and BGA device 2 image size 2048pixel x 563pixel. The 50 images of BGA component 1 and the 50 images of BGA component 2 were identified using the algorithm proposed by the present invention.
The statistics of the experimental test results are shown in table 1 and table 2 below. As can be seen from table 1, for BGA element 1, the deviation of the center coordinates is no more than 0.10 pixels at maximum and the angular deviation is no more than 0.08 ° at maximum, compared with the standard result. As can be seen from table 2, for BGA component 2, the deviation of the center coordinates is no more than 0.11 pixels at maximum and the angular deviation is no more than 0.02 ° at maximum, compared with the standard result. The algorithm recognition result provided by the invention has high accuracy. From tables 1 and 2, it can be found that the standard deviation of the center coordinate deviation and the angle deviation of the algorithm for the BGA element 1 is not more than 0.03, and the standard deviation of the center coordinate deviation and the angle deviation of the algorithm for the BGA element 2 is not more than 0.04, which shows that the stability of the algorithm identification result is strong. In addition, as can be seen from tables 1 and 2, the average time consumption of the algorithm for identifying the small-sized BGA element 1 is only 14.308ms, the average time consumption for identifying the medium-sized and large-sized BGA element 2 is only 64.154ms, and the algorithm provided by the invention has the advantages of high identification speed and good instantaneity.
TABLE 1 statistical table of identification results for BGA device 1
Table 2 BGA device 2 identification result statistics table
In table 1, Δx, Δy, Δr in table 2 represent the differences between the center coordinate X, center coordinate Y, and rotation angle R of the algorithm recognition result and the reference standard value, respectively.
The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto. Within the scope of the technical idea of the invention, a number of simple variants of the technical solution of the invention are possible, including the combination of the individual specific technical features in any suitable way. The various possible combinations of the invention are not described in detail in order to avoid unnecessary repetition. Such simple variations and combinations are likewise to be regarded as being within the scope of the present disclosure.
Claims (9)
1. A method for quickly identifying and positioning a BGA component is characterized by comprising the following steps:
extracting the outline of an electrode terminal in the BGA element image, and acquiring the center coordinate of the electrode terminal as an actual terminal center point set;
the minimum circumscribed rectangle search based on the actual terminal center point set distribution grid array can completely contain candidate positions and candidate angles of all electrode terminal centers, and the average value of all candidate positions and the average value of candidate angles are used as element coarse positioning results;
calculating theoretical center coordinates of the electrode terminals of the BGA element as a template point set;
and carrying out point set-to-point registration on the template point set and the actual terminal center point set by taking the rough positioning result as an initial value to obtain a fine positioning result, wherein the method comprises the following steps of: taking the coarse positioning result as an initial value, carrying out point set registration on the template point set and the actual terminal center point set by adopting an ICP registration method to obtain a first fine positioning result; correcting the terminal spacing in the element parameters by using the first fine positioning result, the actual terminal center point set and the template point set, and reconstructing a new template point set; and carrying out point set registration on the new template point set and the actual terminal center point set by using the first fine positioning result as an initial value through an ICP registration method to obtain the fine positioning result.
2. The method according to claim 1, wherein the minimum circumscribed rectangle search based on the actual terminal center point set distribution grid array can completely contain candidate positions and candidate angles of all electrode terminal centers, and taking the average value of all candidate positions and the average value of candidate angles as the element coarse positioning result specifically comprises the following steps:
s1, taking the coordinate mean value of the actual terminal center point set of the BGA element as a reference position P for coarse positioning position searching 0 ；
S2, taking a current search angle midR= -SearchR/2, wherein SearchR is a preset angle search range;
s3, calculating the minimum circumscribed rectangular size, the X and Y direction search range and the search step length of the central coordinate distribution grid array under midR;
s4, scanning images from a starting position (P0.x-SearchX/2, P0.y-SearchY/2) according to the searching ranges in the X and Y directions and the searching step length, counting the quantity of the actual terminal center point set in the minimum circumscribed rectangle under the current searching position (midX, midY), adding the midY into a Y-direction candidate set Ysets if the electrode terminal center coordinates are all in the current minimum circumscribed rectangle, and adding the midX into an X-direction candidate set Xsets; wherein P is 0 .x、P 0 Y respectively represent reference positions P 0 X, Y direction coordinate values of SearchX, searchY represent the X and Y direction search ranges, respectively;
s5, if the number of elements in the sets Ysets and Xsets in the step S4 is increased, adding midR into the candidate angle set angles, otherwise, executing the step S6;
s6, taking the current angle midR=midR+stepR, wherein stepR is a preset angle searching step length, if midR is less than or equal to SearchR/2, repeating the steps S3-S5, otherwise, executing the step S7;
s7, obtaining an angle estimation value in the coarse positioning by using an angle average value in the candidate angle set angles, and obtaining a central position estimation value coordinate in the coarse positioning by using a position average value in the candidate position sets Xsets and Ysets.
3. The method of claim 2, wherein the minimum bounding rectangle size calculation formula is as follows:
the calculation formula of the X and Y direction search range is as follows:
the search step size calculation formula is as follows:
FW and FH respectively represent the length and width of the minimum circumscribed rectangle Pixel level, and the unit Pixel; scaleX, scaleY represents the Pixel size in μm/Pixel for identifying the camera landscape and portrait, respectively; w (W) mr 、H mr Representing the theoretical length and width values of the minimum circumscribed rectangle respectively, and the unit mm; pitchN, pitchE represents the center distance between two adjacent electrode terminals in the horizontal and vertical directions of the BGA device, respectively, in mm; d represents the diameter of the electrode terminal in mm; stepX and stepY represent the X and Y direction search steps, respectively.
4. A method according to claim 2 or 3, characterized in that counting the number of the smallest circumscribed rectangles of the actual terminal center point set in the current search position (midX, midY) is in particular:
the two center straight lines of the minimum circumscribed rectangle of the grid array under the current search angle midR and the current position (midX, midY) are calculated as follows:
if midr=0, the two center straight lines are respectively: x=midx, y=midy;
if midr+.0, the two center straight lines are respectively: y=k 1 x+b 1 ，y＝K 2 x+b 2 Wherein K is 1 ＝tan(midR)，K 2 ＝-1/K 1 ，b 1 ＝midY-K 1 *midX，b 2 ＝midY-K 2 *midX；
Circularly traversing each point in the center point set of the actual terminal, and judging the distances d between the center of the current electrode terminal and the two center lines respectively 1 、d 2 Relation with FW, FH, when d 1 <FH/2 and d 2 <FW/2, then the current electrode terminal center is in the minimum circumscribed rectangle; FW and FH respectively represent the length and width of the minimum circumscribed rectangle pixel level;
and counting the number of the electrode terminal centers in the minimum circumscribed rectangle after the traversing is completed.
5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,
if midr=0,
d 1 ＝|midY-realBallcenterY|，d 2 ＝|midX-realBallcenterX|
if midR is not equal to 0,
where, (realBallcenterX, realBallcenterY) represents the current electrode terminal center coordinates.
6. The method of claim 5, wherein the terminal pitch in the component parameters is corrected using the first fine positioning result, the actual terminal center point set, and the template point set, and a new template point set is reconstructed:
obtaining the corresponding relation between the template point set and the actual terminal center point set based on the first fine positioning result, obtaining the geometric relation of each point in the actual terminal center point set by using the geometric relation of each point in the template point set and the corresponding relation, and finally calculating the average value of the pixel length of the electrode terminal spacing of the BGA element in the horizontal and vertical directions by using the geometric relation of each point in the actual terminal center point set, instead of the distance of each point in the template point set calculated by using the given element size and the camera pixel size, and recording as RemixPITCHN pixel 、RemixPitchE pixel ；
According to RemixPITCHN pixel 、RemixPitchE pixel The relative position coordinates of the electrode terminals of the BGA component with respect to the center of the component are recalculated as a new set of template points.
7. The method according to claim 5, wherein the ICP registration method includes a neighbor point-to-search constraint, specifically: and eliminating the point pairs with the distance larger than the theoretical electrode terminal spacing, and only reserving a group with the nearest distance for the case that a plurality of points are paired with the same point.
8. A BGA component rapid identification and positioning system comprising:
the electrode terminal center point set extraction module is used for extracting the outline of the electrode terminal in the BGA element image and obtaining the center coordinate of the electrode terminal as an actual terminal center point set;
the coarse positioning module is used for searching the minimum circumscribed rectangle of the distributed grid array based on the actual terminal center point set, can completely contain candidate positions and candidate angles of all electrode terminal centers, and takes the average value of all candidate positions and the average value of candidate angles as an element coarse positioning result;
the template point set module is used for calculating theoretical center coordinates of the electrode terminals of the BGA element to be used as a template point set;
the fine positioning module is used for carrying out point set-to-point registration on the template point set and the actual terminal center point set by taking the coarse positioning result as an initial value to obtain a fine positioning result, and comprises the following steps: taking the coarse positioning result as an initial value, carrying out point set registration on the template point set and the actual terminal center point set by adopting an ICP registration method to obtain a first fine positioning result; correcting the terminal spacing in the element parameters by using the first fine positioning result, the actual terminal center point set and the template point set, and reconstructing a new template point set; and carrying out point set registration on the new template point set and the actual terminal center point set by using the first fine positioning result as an initial value through an ICP registration method to obtain the fine positioning result.
9. A computer storage medium, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311683975.4A CN117392211B (en) | 2023-12-11 | 2023-12-11 | BGA element rapid identification positioning method and system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311683975.4A CN117392211B (en) | 2023-12-11 | 2023-12-11 | BGA element rapid identification positioning method and system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117392211A CN117392211A (en) | 2024-01-12 |
| CN117392211B true CN117392211B (en) | 2024-03-12 |
Family
ID=89468668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311683975.4A Active CN117392211B (en) | 2023-12-11 | 2023-12-11 | BGA element rapid identification positioning method and system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117392211B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007169716A (en) * | 2005-12-22 | 2007-07-05 | Dainippon Printing Co Ltd | Metal mask, and method and device for metal mask position alignment |
| CN101839690A (en) * | 2010-04-13 | 2010-09-22 | 河海大学常州校区 | Visual inspection method for chip electronic component position error based on edge fitting |
| CN104981105A (en) * | 2015-07-09 | 2015-10-14 | 广东工业大学 | Detecting and error-correcting method capable of rapidly and accurately obtaining element center and deflection angle |
| CN105809705A (en) * | 2016-03-30 | 2016-07-27 | 广东工业大学 | Patch element positioning identification method based on minimum enclosing rectangle |
| CN106484194A (en) * | 2015-08-26 | 2017-03-08 | 上海精研电子科技有限公司 | A kind of infrared touch panel multipoint positioning method |
| CN108830902A (en) * | 2018-04-19 | 2018-11-16 | 江南大学 | A kind of workpiece identification at random and localization method based on points cloud processing |
| CN112184715A (en) * | 2020-11-10 | 2021-01-05 | 武汉工程大学 | Method for calculating theoretical center of welding spot of BGA image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109360203B (en) * | 2018-10-30 | 2021-12-03 | 京东方科技集团股份有限公司 | Image registration method, image registration device and storage medium |
-
2023
- 2023-12-11 CN CN202311683975.4A patent/CN117392211B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007169716A (en) * | 2005-12-22 | 2007-07-05 | Dainippon Printing Co Ltd | Metal mask, and method and device for metal mask position alignment |
| CN101839690A (en) * | 2010-04-13 | 2010-09-22 | 河海大学常州校区 | Visual inspection method for chip electronic component position error based on edge fitting |
| CN104981105A (en) * | 2015-07-09 | 2015-10-14 | 广东工业大学 | Detecting and error-correcting method capable of rapidly and accurately obtaining element center and deflection angle |
| CN106484194A (en) * | 2015-08-26 | 2017-03-08 | 上海精研电子科技有限公司 | A kind of infrared touch panel multipoint positioning method |
| CN105809705A (en) * | 2016-03-30 | 2016-07-27 | 广东工业大学 | Patch element positioning identification method based on minimum enclosing rectangle |
| CN108830902A (en) * | 2018-04-19 | 2018-11-16 | 江南大学 | A kind of workpiece identification at random and localization method based on points cloud processing |
| CN112184715A (en) * | 2020-11-10 | 2021-01-05 | 武汉工程大学 | Method for calculating theoretical center of welding spot of BGA image |
Non-Patent Citations (1)
| Title |
|---|
| 圆形标志中心子像素定位方法的研究与实现;黄桂平;《武汉大学学报(信息科学版)》;第30卷(第05期);第388-391页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117392211A (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111260731B (en) | Self-adaptive detection method for checkerboard sub-pixel level corner points | |
| Spreeuwers | Fast and accurate 3D face recognition: using registration to an intrinsic coordinate system and fusion of multiple region classifiers | |
| CN111627075B (en) | Camera external parameter calibration method, system, terminal and medium based on aruco code | |
| CN111223133A (en) | Registration method of heterogeneous images | |
| CN116433737A (en) | Method and device for registering laser radar point cloud and image and intelligent terminal | |
| CN105701776B (en) | A kind of lens distortion antidote and system for automatic optics inspection | |
| CN116188544A (en) | Point cloud registration method combining edge features | |
| US20220148153A1 (en) | System and method for extracting and measuring shapes of objects having curved surfaces with a vision system | |
| CN114612412A (en) | Processing method of three-dimensional point cloud data, application of processing method, electronic device and storage medium | |
| CN117029733B (en) | Bolt loosening detection method, system and device based on computer vision | |
| Xie et al. | A4lidartag: Depth-based fiducial marker for extrinsic calibration of solid-state lidar and camera | |
| CN113705564A (en) | Pointer type instrument identification reading method | |
| CN117392211B (en) | BGA element rapid identification positioning method and system and storage medium | |
| CN117372487A (en) | Image registration method, device, computer equipment and storage medium | |
| CN112102415A (en) | Depth camera external parameter calibration method, device and equipment based on calibration ball | |
| CN111445513A (en) | Plant canopy volume obtaining method and device based on depth image, computer equipment and storage medium | |
| CN113840135B (en) | Color cast detection method, device, equipment and storage medium | |
| CN114897990A (en) | Camera distortion calibration method and system based on neural network and storage medium | |
| CN113627548A (en) | Planar workpiece template matching method, device, medium and computer equipment | |
| CN111815519A (en) | Rotating target detection method, device, system and readable medium | |
| Ping et al. | Verification of turning insert specifications through three-dimensional vision system | |
| CN111950659B (en) | Double-layer license plate image processing method and device, electronic equipment and storage medium | |
| CN111462242B (en) | Rectangular pin chip positioning method based on improved deformable component model | |
| CN117115242B (en) | Identification method of mark point, computer storage medium and terminal equipment | |
| CN110874850A (en) | Real-time unilateral grid feature registration method oriented to target positioning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |