CN114943778A - Reference plane determining method, detecting method, device, equipment and storage medium - Google Patents
Reference plane determining method, detecting method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114943778A CN114943778A CN202210881034.0A CN202210881034A CN114943778A CN 114943778 A CN114943778 A CN 114943778A CN 202210881034 A CN202210881034 A CN 202210881034A CN 114943778 A CN114943778 A CN 114943778A
- Authority
- CN
- China
- Prior art keywords
- pixel
- depth map
- pixel points
- pixel point
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 238000001514 detection method Methods 0.000 claims abstract description 53
- 239000013598 vector Substances 0.000 claims abstract description 53
- 238000005070 sampling Methods 0.000 claims description 38
- 238000004590 computer program Methods 0.000 claims description 17
- 238000010586 diagram Methods 0.000 claims description 14
- 230000007547 defect Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 abstract description 10
- 239000003086 colorant Substances 0.000 abstract description 9
- 230000003287 optical effect Effects 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- RVCKCEDKBVEEHL-UHFFFAOYSA-N 2,3,4,5,6-pentachlorobenzyl alcohol Chemical compound OCC1=C(Cl)C(Cl)=C(Cl)C(Cl)=C1Cl RVCKCEDKBVEEHL-UHFFFAOYSA-N 0.000 description 5
- 238000000354 decomposition reaction Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000003990 capacitor Substances 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- ATJFFYVFTNAWJD-UHFFFAOYSA-N Tin Chemical compound [Sn] ATJFFYVFTNAWJD-UHFFFAOYSA-N 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000011889 copper foil Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007650 screen-printing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000005476 soldering Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a reference surface determining method, a detection method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a three-dimensional coordinate of each pixel point in a depth map of the board card to be detected, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane; clustering pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets; the target point set is determined from the plurality of pixel points in the set, the reference surface can be automatically determined according to the three-dimensional coordinate fitting reference surface of the pixel points in the target point set, the color interference of the depth map is avoided, the bottom plate area is not required to be marked manually, the problem that the color extraction method and the bottom plate area are not marked manually are inaccurate is solved, the method is suitable for board cards with different colors, the extracted reference surface is high in precision, and the universality is high.
Description
Technical Field
The present invention relates to the field of visual image processing technologies, and in particular, to a method, a device, an apparatus, and a storage medium for determining a reference plane.
Background
In the automatic optical detection process of the PCBA board, a depth image of the PCBA board can be collected through a 3D imaging system, so that defect detection can be carried out based on the depth image.
Each pixel point in the depth image acquired in the automatic optical detection process comprises three-dimensional coordinates (x, y, z), wherein z is the height of a component on the PCBA board card corresponding to the pixel point relative to a calibration plane, and the height of the component relative to a bottom plate area of the PCBA board card is detected in defect detection, so that the bottom plate area needs to be determined from the depth image to serve as a reference surface, and the reference surface serves as a reference to correct the z value of each pixel point in the depth image. Currently, determining the floor area from the depth map includes a color extraction method and a manual marking method, wherein the color extraction method is to identify the floor area based on the color of the floor area, and the manual marking method is to manually mark the floor area from the depth map by experience.
The above method for determining the floor area has the following problems:
(1) when the PCBA board card has interference of different colors of objects such as silk screen printing, copper foil, soldering tin and the like, the color of the bottom plate area cannot be accurately identified by the color extraction method, so that the bottom plate area is inaccurately identified;
(2) manual marking is too dependent on human experience and the manually marked local area cannot represent the entire floor area.
Disclosure of Invention
The invention provides a reference surface determining method, a detecting method, a device, equipment and a storage medium, and aims to solve the problem that in the prior art, a base plate area cannot be accurately extracted as a reference surface through a color extraction method and manual marking, so that the correction precision of a depth map is poor.
In a first aspect, the present invention provides a method for determining a reference plane, including:
acquiring a three-dimensional coordinate of each pixel point in a depth map of a board to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane;
clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
determining a target point set from the plurality of pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
and fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
In a second aspect, the present invention provides a detection method for detecting defects of a board to be detected, including:
acquiring an initial depth map of a board card to be tested, wherein each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
determining a reference plane based on the initial depth map;
calculating a distance according to the initial three-dimensional coordinate and the datum plane;
replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
determining a detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determination method of the first aspect.
In a third aspect, the present invention provides a reference plane determining apparatus, including:
the plane fitting module is used for obtaining the three-dimensional coordinate of each pixel point in the depth map of the board card to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane;
the clustering module is used for clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
the target point set determining module is used for determining a target point set from the pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
and the datum plane fitting module is used for fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
In a fourth aspect, the present invention provides a detection apparatus for detecting defects of a board to be detected, including:
the initial depth map acquisition module is used for acquiring an initial depth map of the board card to be detected, and each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
a reference plane determining module for determining a reference plane based on the initial depth map;
the distance calculation module is used for calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference plane;
the depth map updating module is used for replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
the detection result generation module is used for determining the detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determining method of the first aspect.
In a fifth aspect, the present invention provides a board detecting apparatus, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the method for determining a reference plane according to the first aspect of the present invention and/or the method for detecting according to the second aspect of the present invention.
In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a processor to implement the method for determining a reference plane according to the first aspect of the present invention and/or the method for detecting according to the second aspect of the present invention when executed.
The embodiment of the invention obtains the normal vector of the local plane after fitting the local plane for the pixel points according to the three-dimensional coordinates of the pixel points in the depth map, then clusters the pixel points according to the three-dimensional coordinates and the normal vector of the pixel points to obtain a plurality of pixel point sets, further determines the target point set of the bottom plate area of the board card to be tested from the plurality of pixel point sets, and fits the reference plane by adopting the three-dimensional coordinates of the pixel points in the target point sets, thereby realizing that the bottom plate area can be determined through the depth map, and the reference plane can be fitted by adopting the three-dimensional coordinates of the pixel points in the bottom plate area without being interfered by colors in the depth map and being marked manually, solving the problems of inaccurate bottom plate area extraction by a color extraction method and manual marking, being applicable to the board cards with different colors, having high precision of the extracted reference plane and having high precision for correcting the depth map based on the reference plane, the universality is strong.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for determining a datum plane according to an embodiment of the present invention;
fig. 2A is a flowchart of a method for determining a reference plane according to a second embodiment of the present invention;
FIG. 2B is a schematic diagram of downsampling a depth map according to an embodiment of the present invention;
FIG. 2C is a diagram illustrating a neighborhood in an embodiment of the present invention;
FIG. 2D is a schematic diagram of generating sub-regions in a downsampled depth map in accordance with an embodiment of the present invention;
FIG. 2E is a schematic diagram of an initial mask map of an embodiment of the present invention;
fig. 3 is a flowchart of a detection method according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a reference plane determining apparatus according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a detecting device according to a fifth embodiment of the present invention;
fig. 6 is a schematic structural diagram of a board card detection device according to a sixth embodiment of the present invention.
Detailed Description
In order to make the technical solution of the present invention better understood, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not a whole embodiment. All other embodiments, which can be obtained by a person of ordinary skill in the art without any creative effort based on the embodiments of the present invention, shall fall within the protection scope of the present invention.
Example one
Fig. 1 is a flowchart of a reference plane determining method according to an embodiment of the present invention, where this embodiment is applicable to extracting a reference plane of a board to be detected to correct a depth map through the reference plane, and the method may be implemented by a reference plane determining device, which may be implemented in hardware and/or software, and the reference plane determining device may be configured in a board detection apparatus. As shown in fig. 1, the reference plane determining method includes:
s101, obtaining a three-dimensional coordinate of each pixel point in a depth map of the board card to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane.
The board card to be detected can be a circuit board which needs to detect the height from the top of electronic components such as a resistor, a capacitor, an inductor and a chip on the board card to the bottom plate of the board card, the depth map can be an image obtained by placing the board card to be detected on an optical detection machine and extracting the depth after a camera on the optical detection machine takes a picture, each pixel point in the image comprises a three-dimensional coordinate, the three-dimensional coordinate is a coordinate of the position of the pixel point corresponding to the board card to be detected relative to a calibration coordinate system, and in one example, the calibration coordinate system can be the center of a workbench of the optical detection machine.
For each pixel point on the depth map, a neighborhood of the pixel point can be obtained, the three-dimensional coordinate of each pixel point is read from the depth map, a local plane is fitted by adopting the three-dimensional coordinates of a plurality of pixel points in the neighborhood, the local plane enables the sum of the distances from the plurality of pixel points in the neighborhood to the local plane to be minimum, namely the pixel points in the neighborhood are on the plane as far as possible, and the normal vector of the local plane can be obtained after the local plane is fitted.
S102, clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets.
In this embodiment, after each pixel point is fitted to a local plane to obtain a normal vector of the local plane, the pixel points may be clustered by a region growing clustering algorithm to obtain a plurality of pixel point sets, each pixel point set is a set of pixel points with similar characteristics, in one example, a pixel point may be randomly selected as a seed point, euclidean distances between other pixel points in a preset neighborhood and the seed point, normal vector included angles, and the like are calculated, when both the euclidean distances and the normal vector included angles of two pixel points are smaller than corresponding preset differences, it is indicated that the positions of the two pixel points corresponding to a board card to be tested belong to the same electronic component, the seed point and the pixel point may be added to one pixel point set, then a pixel point is randomly selected from the pixel point set as a seed point, the above steps are repeated, and the pixel point set is output until no more pixel points are added to the pixel point set, and repeating the steps for the rest pixel points to finally obtain a plurality of pixel point sets.
Of course, besides the region growing clustering algorithm, the pixel points may also be clustered by using clustering algorithms such as K-Means (K-Means) clustering, mean shift clustering, density-based clustering (DBSCAN), and the like, and the algorithm for clustering the pixel points is not limited in this embodiment.
S103, determining a target point set from the plurality of pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested.
In an optional embodiment, for each pixel point set, the point set attribute data of the pixel point set may be obtained, the confidence of each pixel point set is calculated through the point set attribute data, and when the confidence is greater than a preset threshold, the target point set may be determined to be the set of the pixel points corresponding to the backplane area.
Illustratively, the perimeter and the area of the outline of the bottom plate region are generally larger than those of the electronic component, and the number of pixels occupied by the bottom plate region is large, so that the perimeter and the area of the circumscribed rectangle of the pixel point set and the number of pixels included in the pixel point set can be obtained as point set attribute data, then different weights are assigned to the perimeter, the area and the number of pixels of the circumscribed rectangle to perform weighted summation to obtain a confidence, and when the confidence is larger than a preset threshold, the pixel point set is determined to be a set of pixels corresponding to the bottom plate region, namely, a target point set.
And S104, fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
In this embodiment, the three-dimensional coordinates of the pixel points in the target point set can be extracted from the depth map, a plane is fitted through the three-dimensional coordinates, the method for specifically fitting the plane is the same as that for fitting the local plane in S101, the plane obtained through fitting is a reference plane, the reference plane can be used to replace the calibration plane to re-correct the depth coordinate values in the three-dimensional coordinates of each pixel point, finally, an updated depth map is obtained, and the board card to be detected is detected through the updated depth map.
According to the embodiment of the invention, after the local plane is fitted for the pixel points according to the three-dimensional coordinates of the pixel points in the depth map, the normal vector of the local plane is obtained, then the pixel points are clustered according to the three-dimensional coordinates and the normal vector of the pixel points to obtain a plurality of pixel point sets, the target point set is further determined from the plurality of pixel point sets, the base plate area can be determined through the depth map according to the three-dimensional coordinates of the pixel points in the base plate area, the base plate area is fitted by adopting the three-dimensional coordinates of the pixel points in the base plate area, the color interference in the depth map is avoided, the base plate area is not required to be marked manually, the problem that the base plate area is not accurately extracted by a color extraction method and manual marking is solved, the method can be suitable for board cards with different colors, the accuracy of the extracted base plate is high, the accuracy of depth map correction is high based on the base plate, and the universality is strong.
Example two
Fig. 2A is a flowchart of a reference plane determining method provided in the second embodiment of the present invention, and the second embodiment of the present invention performs optimization based on the first embodiment, as shown in fig. 2A, the reference plane determining method includes:
s201, downsampling the depth map of the board card to be tested to obtain a downsampled depth map.
As shown in fig. 2B, an example of downsampling a depth map is shown, where the depth map includes a plurality of pixel points, each pixel point includes a three-dimensional coordinate, and the three-dimensional coordinate is a coordinate of a position of the pixel point on the board to be tested with respect to a calibration coordinate system, which may be a center of a workbench of the optical inspection machine in one example.
In an alternative embodiment, during the down-sampling, the down-sampling base N _ unit and the unit down-sampling point N _ down _ unit may be set first, and then the down-sampling target pixel point N _ target may be calculated by the following formula:
N_target = (N_input / N_unit ) × N_down_unit
in the above formula, N _ input is the total number of pixels of the depth map, N _ unit is the downsampling base number, which may be exemplarily set to 2000, N _ down _ unit is the unit downsampling point number, which may be set to 1000 plus 1500, and specifically may be determined according to the total number of pixels of the depth map.
After the number of downsampling target pixel points N _ target is calculated through the above formula, sampling step lengths in the X direction and the Y direction may be calculated, and in one example, if the sampling step lengths in the X direction and the Y direction are equal, the sampling step length in the X direction may be equal to the sampling step length in the X direction n The sampling step in the Y direction may be equal to Y n /M, wherein X n The number of pixels in a line in the X direction in the depth map, Y n The number of a column of pixel points in the Y direction in the depth map is M, and the number of the down-sampling target pixel points N _ target is a value obtained by opening the square of the number of the down-sampling target pixel points N _ target.
Of course, in practical application, the sampling step size in the X direction and the sampling step size in the Y direction may also be directly set, and the sampling manner of the depth map is not limited in this embodiment.
As shown in fig. 2B, the black-filled pixel points are sampling points, and after sampling, a downsampled depth map as shown in fig. 2C can be obtained, and as can be seen from fig. 2C, the number of the pixel points included in the downsampled depth map is smaller than the number of the pixel points of the depth map before downsampling in fig. 2B, which can reduce the data amount of subsequent processing and improve the speed of determining the reference plane.
S202, aiming at each pixel point in the down-sampling depth map, determining a neighborhood of the pixel point.
In an optional embodiment, each pixel point in the downsampled depth map may be used as a center, and a region within a preset radius range is determined as a neighborhood of the pixel point, or K neighboring pixel points of the pixel point are selected to form a neighborhood of the pixel point.
As shown in fig. 2C, in an example, for the pixel a, a circular region B with the pixel a as a center may be selected as a neighborhood, and for the pixel C, 8 pixels adjacent to the pixel C may be selected to form a neighborhood D of the pixel C.
In another embodiment, the down-sampling depth map has an ordering, that is, each pixel in the down-sampling depth map may be marked with a serial number as an index, so that each pixel may retrieve neighboring pixels through the ordering to form a neighborhood, and thus, the neighboring pixels of each pixel may be quickly found through the ordering of the depth map to form a neighborhood.
Of course, in practical applications, other ways may be used by those skilled in the art to determine the neighborhood of a pixel point.
S203, obtaining the three-dimensional coordinates of the pixel points in the neighborhood, and fitting the local plane according to the three-dimensional coordinates to obtain the normal vector of the local plane.
In an optional embodiment, for each neighborhood, the three-dimensional coordinates of each pixel point in the neighborhood can be read from the down-sampling depth map, and then a local plane is fitted to the three-dimensional coordinates of the pixel points in the neighborhood by using a least square method to obtain a normal vector of the local plane, which specifically includes the following steps:
establishing a local plane equation:
aX + bY + cZ - d = 0
according to the principle of the least square method, the fitted local plane is to be optimal, and actually, the sum of squares of distances from the space points corresponding to the three-dimensional coordinates associated with each pixel point in the neighborhood to the local plane is minimized, namely, the solution:
in the above formula, d i Is a point p in the neighborhood i =（x i ，y i ，z i ) Distance to local plane, i.e. d i =| ax i + by i + z i + d | to minimize the sum of squares e of the distances, the solution can be performed by SVD matrix decomposition, which includes the following steps:
and (3) solving a center of mass m = (x, y, z) of the neighborhood, wherein the center of mass m is the center point of all points in the neighborhood, namely:
n is the number of points contained in the neighborhood, and there are:
assuming a matrix:
the purpose of fitting the local plane is that all points in the neighborhood are on the local plane, and in practical cases, some points are out of the local plane, and the purpose of fitting is that the sum of distances from the local plane to all points is minimum, that is, the objective function of fitting optimization is:
min||AX||
the constraint condition is | | X | =1, and Singular Value Decomposition (Singular Value Decomposition) can be performed on the matrix a:
in the singular value decomposition, U is an m × m matrix, D is an m × n matrix, D is all 0 except for elements on a main diagonal, each element on the main diagonal is called a singular value, V is an m × n matrix, and U and V are unitary matrices.
From the decomposition result, eigenvalues and eigenvectors corresponding to the matrix a can be obtained, the minimum value of e is the minimum eigenvalue of the matrix a, the corresponding eigenvectors are the plane parameters a, b, c, and the normal vector n = (a, b, c) can be obtained.
And determining a neighborhood for each pixel point in the down-sampling depth map, and fitting a local plane for the neighborhood to obtain a normal vector of the local plane.
And S204, clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets.
In an optional embodiment, for pixel points in a down-sampling depth map, one pixel point can be randomly selected as a seed point, other pixel points in a preset neighborhood are determined, the Euclidean distance between the other pixel points and the seed point is calculated by adopting a three-dimensional coordinate aiming at each other pixel point, the normal vector of the other pixel points and the normal vector of the seed point are adopted to calculate the vector included angle, when the Euclidean distance is smaller than a preset distance threshold value and the normal vector included angle is smaller than the preset included angle, the other pixel points and the seed point are added into a pixel point set, then one pixel point is randomly selected from the pixel point set to serve as the seed point, the step of determining the other pixel points in the preset neighborhood is returned, and the pixel point set is output until no new pixel point exists in the pixel point set; and randomly selecting one pixel point from the rest pixel points as a seed point, returning to the step of determining other pixel points in the preset neighborhood until all the pixel points are added into the pixel point set, and obtaining a plurality of pixel point sets.
The clustering process is a region growing clustering algorithm, and the specific process is as follows:
s1, randomly selecting a pixel point from the down-sampling depth map as a seed point, setting a neighborhood of a search radius r, traversing the down-sampling depth map, searching other pixel points of which the normal vector included angles and Euclidean distances meet a threshold value in the neighborhood of the radius r, and adding the pixel point and the seed point into a pixel point set U1 in the same category;
s2, randomly selecting a pixel point from the pixel point set U1 as a new seed point, and repeating the step S1;
and S3, sequentially iterating until no more pixel points are added to the pixel point set U1, and outputting the iteration to obtain a pixel point set U1.
S4, repeating S1 to S3 for other pixels except for the pixel point set U1, and sequentially outputting pixel point sets U2, U3, … … and Un;
and S5, if the number of the residual pixel points is less than the preset number or the iteration times exceeds the preset times, stopping iteration and outputting all pixel point sets.
In the embodiment, for each pixel point in the down-sampling image, similar pixel points are searched by the distance and the normal vector smaller than the preset threshold, and the pixel points belonging to the same plane can be searched.
Certainly, in practical applications, besides the clustering method, clustering algorithms such as K-Means (K-Means) clustering, mean shift clustering, density-based clustering (DBSCAN), and the like may be used to cluster the pixel points, and the algorithm for clustering the pixel points is not limited in this embodiment.
S205, point set attribute data of a plurality of pixel point sets are obtained.
For each pixel point set, the number of the pixel points included in the pixel point set is different, the perimeter and the area of the minimum circumscribed rectangle of all the pixel points included in each pixel point set and the number of the pixel points can be used as the attribute data of the pixel point set, and of course, the attribute data of the pixel point set can also include other data, for example, the average value of the depth coordinates in the three-dimensional coordinates associated with the pixel points and the like can also be included.
And S206, calculating the confidence coefficient of the pixel point set by adopting the point set attribute data.
Specifically, each point set attribute data of the pixel point set may be given a weight, and then the confidence of the pixel point set is obtained through weighted summation.
And S207, determining the pixel point set with the confidence coefficient larger than a preset threshold value as a target point set.
Generally speaking, on a board card to be tested, the areas of the tops of components such as resistors, capacitors, inductors, chips and the like are usually smaller, the number of pixels included in a bottom plate region corresponding to a down-sampling depth map is smaller, the perimeter and the area of an external rectangle thereof are smaller, the area of the bottom plate region is usually larger, the number of pixels included in the bottom plate region corresponding to the down-sampling depth map is larger, a pixel point set belonging to the bottom plate region can be screened out through confidence, specifically, a confidence threshold can be set, when the confidence is larger than the confidence threshold, the pixel point set is described as the pixel point set of the bottom plate region, the pixel point set is taken as a target point set, it is described that the number of the target point set is more than one, in addition, the confidence threshold can be determined according to the bottom plate region exposed on the board card to be tested, the density of electronic components, and the top area of the electronic components, different confidence threshold values can be set for different board cards to be tested, so that the target point set belonging to the bottom board area can be determined through the confidence threshold values.
And S208, generating a mask diagram of the bottom plate area of the board card to be detected by adopting the target point set.
In an optional embodiment, for each target point set, generating a sub-region containing pixel points in the target point set in a down-sampling depth map, summing a plurality of sub-regions to obtain an initial mask region, filling a preset color in the initial mask region to obtain an initial mask map, and up-sampling the initial mask map to obtain a mask map with the size same as that of the depth map, so as to serve as the mask map of the backplane region of the board to be tested.
In an example, since each target point set includes a plurality of pixel points, a minimum circumscribed polygon including the plurality of pixel points may be generated, where the minimum circumscribed polygon is a sub-region including the pixel points in the target point set, as shown in fig. 2D, the sub-regions a1, a2, and A3 are generated by the plurality of target point sets, the sub-regions a1, a2, and A3 are merged to obtain an initial mask region, the initial mask region is filled with a preset color to obtain an initial mask map as shown in fig. 2E, as can be seen from fig. 2D to fig. 2E, the sub-region a1 is communicated with the sub-region a2, and the initial mask region a12 is obtained after merging, where the size of the initial mask map is the same as that of the downsampled depth map, and the mask region is filled with white, and of course, other colors may also be filled.
The initial mask map is generated based on the downsampling depth map, and the initial mask map can be amplified through upsampling, for example, the initial mask map is amplified to a mask map with the same size as the depth map, the mask map is the mask map of the bottom plate area of the board to be tested, namely, the white filled area in the mask map of the bottom plate area of the board to be tested is the bottom plate area.
S209, determining a bottom plate area from the depth map according to the mask map, and fitting a reference plane by adopting the three-dimensional coordinates of the pixel points in the bottom plate area.
In an example, a mask map may be superimposed on the depth map, since the mask map and the depth map have the same size, a region on the mask map, which is filled with white color, corresponding to the region in the depth map is a bottom plate region, three-dimensional coordinates of all pixel points of the bottom plate region may be read out, and a plane is fitted through the read three-dimensional coordinates of the bottom plate region, where the plane obtained by fitting is a reference plane, and a specific fitting process may refer to S203, which is not described in detail herein.
In the embodiment, after the depth map is downsampled to obtain a downsampled depth map, the neighborhood of each pixel point in the downsampled depth map is determined and a local plane is fitted for the neighborhood to obtain a normal vector, clustering is further carried out according to the three-dimensional coordinates and the normal vector of the pixel points in the downsampled depth map to obtain a plurality of pixel point sets, calculating the confidence coefficient of each pixel point set according to the point set attribute data of the pixel point sets, taking the pixel point sets with the confidence coefficient larger than the threshold value as the target point sets of the bottom plate area, generating a sub-region containing pixel points of the target point set in the depth map through the target point set, filling preset colors in the sub-region to obtain an initial mask map, sampling the initial mask image to obtain a mask image of a bottom plate area of the board card to be tested, superposing the mask image into a depth image, and reading a three-dimensional coordinate fitting plane of pixel points in the depth map covered by the mask area as a reference plane. The bottom plate area can be determined through the depth map, the reference surface is fitted by adopting the three-dimensional coordinates of the pixel points in the bottom plate area, color interference in the depth map is avoided, the bottom plate area does not need to be marked manually, the problem that the color extraction method and the manual marking extraction method are inaccurate in the bottom plate area is solved, the method is suitable for board cards with different colors, the extracted reference surface is high in precision, and the universality is high.
Furthermore, the depth map is downsampled firstly, the mask map with the same size as the depth map is obtained by upsampling after the mask map is obtained, the number of pixel points needing to be processed is reduced through downsampling, and the speed of determining the reference plane is improved.
Furthermore, a pixel point set of similar pixel points is clustered and determined through distance and normal vectors, confidence is calculated through attribute data of the pixel point set, the pixel point set with the confidence being larger than a preset threshold is determined as a target point set of the bottom plate area to determine the bottom plate area, the color of the bottom plate area is not affected, the bottom plate area can be accurately determined, the method is suitable for bottom plates with various colors, and the universality is high.
EXAMPLE III
Fig. 3 is a flowchart of a detection method according to a third embodiment of the present invention, where the method is applicable to a situation where a depth map is corrected to detect a defect of a board, and the method may be executed by a detection device, where the detection device may be implemented in a form of hardware and/or software, and the detection device may be configured in a board detection device. As shown in fig. 3, the detection method includes:
s301, obtaining an initial depth map of the board card to be tested, wherein each pixel point in the initial depth map comprises an initial three-dimensional coordinate.
The board card to be tested may be a circuit board that needs to detect a height from a top of an electronic component to a bottom board of the board card, in an example, the initial depth map may be a depth map in which a calibration plane is used as a measurement reference, in the initial depth map, a three-dimensional coordinate value of each pixel point is based on the calibration plane, for example, a center point of the calibration plane is used as an origin of a coordinate system, in the three-dimensional coordinates (x, y, z) of each pixel point, x and y represent a position of the electronic component on the board card corresponding to the pixel point, and z represents a distance from the top of the electronic component to the calibration plane, generally, the calibration plane is a surface of a workbench of the board card testing apparatus, and may be a plane on which the board card to be tested is placed, for example. After the board card detection equipment acquires images of the board card through the camera, the distance from the top of each electronic element on the board card to a calibration plane can be calculated through calibration parameters of the camera to obtain a z coordinate, and an x coordinate and a y coordinate are determined through an imaging relation, so that each pixel point and a three-dimensional coordinate are associated to obtain an initial depth map.
And S302, determining a reference plane based on the initial depth map.
In the embodiment of the invention, for the pixel points in the initial depth map, the pixel points are fitted to the local plane according to the three-dimensional coordinates to obtain the normal vector of the local plane, the pixel points are clustered according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets, the target point sets are determined from the plurality of pixel point sets, and the three-dimensional coordinates of the pixel points in the target point sets are adopted to fit the reference plane.
And S303, calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference plane.
Specifically, the distance from the spatial point to the plane may be calculated, and in this embodiment, the equation of the reference plane obtained in S302 is assumed to be:
Ax+By+Cz+D=0
the initial three-dimensional coordinates of the pixel point are (x 0, y0, z 0), then the distance is calculated as follows:
the distance d is a distance from a position of the pixel point on the board card to be tested to the reference plane, that is, a distance from the position of the pixel point on the board card to be tested to the bottom plate area of the board card to be tested.
And S304, replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map.
After the distance is obtained by calculation, the distance of each pixel point may be used to replace the depth coordinate value z in the initial depth map, so as to obtain a target depth map, that is, the depth coordinate value in the three-dimensional coordinate associated with each pixel point in the target depth map is the distance from the component on the board corresponding to the pixel point to the reference plane, that is, the height from the top of the component to the bottom board area, that is, for the target depth map, the three-dimensional coordinate of each pixel point is (x 0, y0, d 0).
S305, determining the detection result of the board card to be detected by adopting the target depth map and a preset standard depth map.
The standard depth map may be a depth map which is manufactured in advance, the bottom plate area of a qualified board card is used as a reference surface, the distance from the top of a component to the bottom plate area is used as a coordinate z value, after the target depth map is obtained, the coordinate d0 value of each pixel point in the target depth map is read, the coordinate z value of the corresponding pixel point in the standard depth map is read, d0 and z are compared, an error is calculated, if the error is within a preset range, the height of the top of the component corresponding to the pixel point to the bottom plate area is qualified, otherwise, the height of the top of the component corresponding to the pixel point to the bottom plate area is unqualified, and a detection result of each pixel point is generated to serve as a detection result of the board card to be detected.
In this embodiment, after obtaining the initial depth map of the board card to be detected, the reference plane is determined by the reference plane determining method in the first embodiment or the second embodiment, the distance is further calculated according to the initial three-dimensional coordinates of the pixel points and the reference plane, the distance is used to replace the depth coordinate value in the initial three-dimensional coordinates to obtain the target depth map, and the target depth map and the preset standard depth map are used to determine the detection result of the board card to be detected.
Example four
Fig. 4 is a schematic structural diagram of a reference plane determining apparatus according to a fourth embodiment of the present invention. As shown in fig. 4, the reference plane determining apparatus includes:
the plane fitting module 401 is configured to obtain a three-dimensional coordinate of each pixel point in a depth map of the board card to be tested, and fit a local plane for the pixel point according to the three-dimensional coordinate to obtain a normal vector of the local plane;
a clustering module 402, configured to cluster the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
a target point set determining module 403, configured to determine a target point set from the multiple pixel point sets, where the target point set is a pixel point set in a bottom plate area of the board to be tested;
and a reference plane fitting module 404, configured to fit a reference plane according to the three-dimensional coordinates of the pixel points in the target point set.
Optionally, the plane fitting module 401 includes:
the down-sampling module is used for down-sampling the depth map of the board card to be detected to obtain a down-sampling depth map;
a neighborhood determination module, configured to determine, for each pixel point in the downsampled depth map, a neighborhood of the pixel point;
and the fitting module is used for acquiring the three-dimensional coordinates of the pixel points in the neighborhood and fitting a local plane according to the three-dimensional coordinates to obtain a normal vector of the local plane.
Optionally, the neighborhood determination module includes:
the neighborhood determining unit is used for determining a region in a preset radius range as a neighborhood of the pixel point by taking each pixel point in the sampling depth map as a center; or; and selecting K adjacent pixel points of the pixel points to form a neighborhood of the pixel points.
Optionally, the fitting module comprises:
and the fitting unit is used for acquiring the three-dimensional coordinates of each pixel point in each neighborhood, and fitting the three-dimensional coordinates of the pixel points in the neighborhood to a local plane by adopting a least square method to obtain a normal vector of the local plane.
Optionally, the clustering module 402 includes:
the seed point selection unit is used for randomly selecting one pixel point as a seed point and determining other pixel points in a preset range;
the distance sum normal vector difference value calculating unit is used for calculating Euclidean distances between the other pixel points and the seed point by adopting the three-dimensional coordinates and calculating normal vector included angles by adopting normal vectors of the other pixel points and the seed point;
the point set generating unit is used for adding the other pixel points and the seed point into a pixel point set when the Euclidean distance is smaller than a preset distance threshold and the normal vector included angle is smaller than a preset included angle;
the first seed point updating unit is used for randomly selecting a pixel point from the pixel point set as a seed point, returning to the seed point selecting unit and outputting the pixel point set until no new pixel point is added in the pixel point set;
and the second seed point updating unit is used for randomly selecting one pixel point from the rest pixel points as a seed point, returning to the step of the seed point selecting unit until all the pixel points are added into the pixel point set to obtain a plurality of pixel point sets.
Optionally, the target point set determining module 403 includes:
a point set attribute data acquisition unit for acquiring point set attribute data of a plurality of pixel point sets;
a confidence coefficient calculation unit for calculating the confidence coefficient of the pixel point set by using the point set attribute data;
the target point determining and calculating unit is used for determining the pixel point set with the confidence coefficient larger than a preset threshold value as a target point set;
optionally, the reference plane fitting module 404 includes:
the mask image generating module is used for generating a mask image of the bottom plate area of the board card to be detected by adopting the target point set;
and the fitting module is used for determining a bottom plate area from the depth map according to the mask map and fitting a reference plane by adopting the three-dimensional coordinates of the pixel points in the bottom plate area.
Optionally, the mask map generating module includes:
a sub-region generating unit, configured to generate, for each target point set, a sub-region including pixel points in the target point set in the downsampled depth map;
the initial mask image generating unit is used for solving a union set of a plurality of subregions to obtain an initial mask region and filling a preset color into the initial mask region to obtain an initial mask image;
and the up-sampling unit is used for up-sampling the initial mask image to obtain a mask image with the same size as the depth image so as to be used as the mask image of the bottom plate area of the board card to be detected.
The reference plane determining device provided by the embodiment of the invention can execute the reference plane determining method provided by the first embodiment and the second embodiment of the invention, and has corresponding functional modules and beneficial effects of the executing method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a detecting device according to a fifth embodiment of the present invention. The detection device is used for detecting the defects of the board, and as shown in fig. 5, the detection device comprises:
the initial depth map acquiring module 501 is configured to acquire an initial depth map of a board to be tested, where each pixel point in the initial depth map includes an initial three-dimensional coordinate;
a reference plane determining module 502 for determining a reference plane based on the initial depth map;
a distance calculating module 503, configured to calculate a distance according to the initial three-dimensional coordinates of the pixel point and the reference plane;
a depth map updating module 504, configured to replace the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
a detection result generation module 505, configured to determine a detection result of the board card to be detected by using the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determining method described in the first embodiment or the second embodiment.
The detection device provided by the embodiment of the invention can execute the detection method provided by the third embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
EXAMPLE six
Fig. 6 shows a schematic structural diagram of a board detection device 60 that can be used to implement the embodiment of the present invention. Board detection apparatus is intended to mean an apparatus containing various forms of digital computers, such as, for example, an apparatus containing a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers.
As shown in fig. 6, the board detection apparatus includes at least one processor 61, and a memory communicatively connected to the at least one processor 61, such as a Read Only Memory (ROM) 62, a Random Access Memory (RAM) 63, and the like, where the memory stores a computer program executable by the at least one processor, and the processor 61 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 62 or the computer program loaded from the storage unit 68 into the Random Access Memory (RAM) 63. In the RAM 63, various programs and data necessary for the operation of the board detection apparatus 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other by a bus 64. An input/output (I/O) interface 65 is also connected to bus 64.
A plurality of components in the board detection apparatus 60 are connected to the I/O interface 65, including: an input unit 66 such as a keyboard, a mouse, a camera that acquires a depth image, and the like; an output unit 67 such as various types of displays, speakers, and the like; a storage unit 68 such as a magnetic disk, optical disk, or the like; and a communication unit 69 such as a network card, modem, wireless communication transceiver, etc. The communication unit 69 allows the board detection device 60 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the reference plane determining method, and/or the detecting method, may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 68. In some embodiments, part or all of the computer program may be loaded and/or installed onto the board detection apparatus 60 via the ROM 62 and/or the communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the above-described reference plane determining method, and/or the detecting method, may be performed. Alternatively, in other embodiments, the processor 41 may be configured to perform the reference plane determination method, and/or the detection method, in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described herein may be implemented on a board detection apparatus having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the board detection apparatus. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations, and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. A reference plane determining method, comprising:
acquiring a three-dimensional coordinate of each pixel point in a depth map of a board card to be tested, and fitting a local plane for the pixel point according to the three-dimensional coordinate to obtain a normal vector of the local plane;
clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
determining a target point set from the plurality of pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
and fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
2. The method for determining the reference plane according to claim 1, wherein the obtaining of the three-dimensional coordinates of each pixel point in the depth map of the board card to be tested and the fitting of the pixel point to the local plane according to the three-dimensional coordinates to obtain the normal vector of the local plane comprises:
downsampling the depth map of the board card to be tested to obtain a downsampled depth map;
determining a neighborhood of the pixel points for each pixel point in the downsampled depth map;
and acquiring the three-dimensional coordinates of the pixel points in the neighborhood, and fitting a local plane according to the three-dimensional coordinates to obtain a normal vector of the local plane.
3. The reference plane determination method of claim 2, wherein determining, for each pixel point in the downsampled depth map, a neighborhood of the pixel point comprises:
determining a region within a preset radius range as a neighborhood of the pixel point by taking each pixel point in the downsampled depth map as a center;
or;
and selecting K adjacent pixel points of the pixel points to form a neighborhood of the pixel points.
4. The method for determining the reference plane according to claim 2, wherein the obtaining three-dimensional coordinates of the pixel points in the neighborhood and fitting a local plane according to the three-dimensional coordinates to obtain a normal vector of the local plane comprises:
and aiming at each neighborhood, acquiring the three-dimensional coordinates of each pixel point in the neighborhood, and fitting a local plane to the three-dimensional coordinates of the pixel points in the neighborhood by adopting a least square method to obtain a normal vector of the local plane.
5. The method of any one of claims 1 to 4, wherein the clustering the pixel points according to their three-dimensional coordinates and the normal vector to obtain a plurality of pixel point sets comprises:
randomly selecting one pixel point as a seed point, and determining other pixel points in a preset neighborhood;
aiming at each other pixel point, calculating Euclidean distances between the other pixel points and the seed point by adopting the three-dimensional coordinates, and calculating a normal vector included angle by adopting normal vectors of the other pixel points and normal vectors of the seed point;
when the Euclidean distance is smaller than a preset distance threshold value and the normal vector included angle is smaller than a preset included angle, adding the other pixel points and the seed point into a pixel point set;
randomly selecting a pixel point from the pixel point set as a seed point, returning to the step of determining other pixel points in a preset neighborhood, and outputting the pixel point set until no new pixel points exist in the pixel point set;
and randomly selecting one pixel point from the rest pixel points as a seed point, returning to the step of determining other pixel points in the preset neighborhood until all the pixel points are added into the pixel point set, and obtaining a plurality of pixel point sets.
6. The method of determining a reference plane as claimed in any one of claims 2 to 4, wherein said determining a set of target points from a plurality of said sets of pixels comprises:
acquiring point set attribute data of a plurality of pixel point sets;
calculating the confidence of the pixel point set by adopting the point set attribute data;
and determining the pixel point set with the confidence coefficient larger than a preset threshold value as a target point set.
7. The method of claim 6, wherein said fitting a reference plane according to three-dimensional coordinates of pixel points in said set of target points comprises:
generating a mask diagram of a bottom plate area of the board card to be detected by adopting the target point set;
and determining a bottom plate area from the depth map according to the mask map, and fitting a reference plane by adopting the three-dimensional coordinates of the pixel points in the bottom plate area.
8. The method for determining the reference plane as claimed in claim 7, wherein the generating the mask diagram of the bottom plate area of the board to be tested by using the target point set includes:
for each target point set, generating a sub-region containing pixel points in the target point set in the down-sampling depth map;
obtaining an initial mask area by summing a plurality of sub-areas, and filling a preset color in the initial mask area to obtain an initial mask image;
and sampling the initial mask image to obtain a mask image with the same size as the depth image, and taking the mask image as a mask image of a bottom plate area of the board card to be detected.
9. A detection method is characterized in that the method is used for detecting the defects of a board card to be detected, and comprises the following steps:
acquiring an initial depth map of a board card to be tested, wherein each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
determining a reference plane based on the initial depth map;
calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference surface;
replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
determining a detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determination method of any one of claims 1 to 8.
10. A reference plane determining apparatus, comprising:
the plane fitting module is used for obtaining the three-dimensional coordinate of each pixel point in the depth map of the board card to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane;
the clustering module is used for clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
the target point set determining module is used for determining a target point set from the pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
and the datum plane fitting module is used for fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
11. The utility model provides a detection device, its characterized in that for carry out defect detection to the integrated circuit board that awaits measuring, include:
the initial depth map acquisition module is used for acquiring an initial depth map of the board card to be detected, and each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
a reference plane determining module for determining a reference plane based on the initial depth map;
the distance calculation module is used for calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference plane;
the depth map updating module is used for replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
the detection result generation module is used for determining the detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determination method of any one of claims 1 to 8.
12. The utility model provides a board detection equipment which characterized in that, board detection equipment includes:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the datum level determination method of any one of claims 1-8 and/or the detection method of claim 9.
13. A computer-readable storage medium, having stored thereon computer instructions for causing a processor to execute a method of determining a reference plane according to any one of claims 1-8 and/or a method of detecting according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210881034.0A CN114943778B (en) | 2022-07-26 | 2022-07-26 | Reference plane determining method, detecting method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210881034.0A CN114943778B (en) | 2022-07-26 | 2022-07-26 | Reference plane determining method, detecting method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114943778A true CN114943778A (en) | 2022-08-26 |
| CN114943778B CN114943778B (en) | 2023-01-13 |
Family
ID=82910732
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210881034.0A Active CN114943778B (en) | 2022-07-26 | 2022-07-26 | Reference plane determining method, detecting method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114943778B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115854892A (en) * | 2022-12-05 | 2023-03-28 | 上海赫立智能机器有限公司 | Zero plane correction method for PCB detection |
| CN117975429A (en) * | 2024-03-29 | 2024-05-03 | 之江实验室 | Method and device for determining physical properties of black hole jet flow and storage medium |
| CN118052861A (en) * | 2024-02-20 | 2024-05-17 | 上海赫立智能机器有限公司 | Zero plane acquisition method, system, medium and electronic equipment |
| CN118134995A (en) * | 2024-04-30 | 2024-06-04 | 成都飞机工业（集团）有限责任公司 | Method, device, equipment and storage medium for arranging measurement points of long and narrow patches |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180075616A1 (en) * | 2016-03-09 | 2018-03-15 | Boe Technology Group Co., Ltd. | Method and system for determining datum plane |
| CN111402251A (en) * | 2020-04-01 | 2020-07-10 | 苏州苏映视图像软件科技有限公司 | Visual inspection method and system for 3D defect detection |
| CN111932537A (en) * | 2020-10-09 | 2020-11-13 | 腾讯科技（深圳）有限公司 | Object deformation detection method and device, computer equipment and storage medium |
| CN111986299A (en) * | 2019-05-24 | 2020-11-24 | 北京京东尚科信息技术有限公司 | Point cloud data processing method, device, equipment and storage medium |
| CN112200771A (en) * | 2020-09-14 | 2021-01-08 | 浙江大华技术股份有限公司 | Height measuring method, device, equipment and medium |
| CN113102269A (en) * | 2021-04-07 | 2021-07-13 | 沈阳建筑大学 | Prefabricated part finished product quality detection system based on three-dimensional stereoscopic vision and application thereof |
| CN113689420A (en) * | 2021-09-03 | 2021-11-23 | 凌云光技术股份有限公司 | Method and device for detecting welding spot defects of circuit board and detection equipment |
| WO2022001806A1 (en) * | 2020-06-28 | 2022-01-06 | 华为技术有限公司 | Image transformation method and apparatus |
-
2022
- 2022-07-26 CN CN202210881034.0A patent/CN114943778B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180075616A1 (en) * | 2016-03-09 | 2018-03-15 | Boe Technology Group Co., Ltd. | Method and system for determining datum plane |
| CN111986299A (en) * | 2019-05-24 | 2020-11-24 | 北京京东尚科信息技术有限公司 | Point cloud data processing method, device, equipment and storage medium |
| CN111402251A (en) * | 2020-04-01 | 2020-07-10 | 苏州苏映视图像软件科技有限公司 | Visual inspection method and system for 3D defect detection |
| WO2022001806A1 (en) * | 2020-06-28 | 2022-01-06 | 华为技术有限公司 | Image transformation method and apparatus |
| CN112200771A (en) * | 2020-09-14 | 2021-01-08 | 浙江大华技术股份有限公司 | Height measuring method, device, equipment and medium |
| CN111932537A (en) * | 2020-10-09 | 2020-11-13 | 腾讯科技（深圳）有限公司 | Object deformation detection method and device, computer equipment and storage medium |
| CN113102269A (en) * | 2021-04-07 | 2021-07-13 | 沈阳建筑大学 | Prefabricated part finished product quality detection system based on three-dimensional stereoscopic vision and application thereof |
| CN113689420A (en) * | 2021-09-03 | 2021-11-23 | 凌云光技术股份有限公司 | Method and device for detecting welding spot defects of circuit board and detection equipment |
Non-Patent Citations (1)
| Title |
|---|
| 余永维等: "基于点云三维重建的装配件尺寸测量系统", 《组合机床与自动化加工技术》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115854892A (en) * | 2022-12-05 | 2023-03-28 | 上海赫立智能机器有限公司 | Zero plane correction method for PCB detection |
| CN115854892B (en) * | 2022-12-05 | 2024-02-02 | 上海赫立智能机器有限公司 | Zero plane correction method for PCB detection |
| CN118052861A (en) * | 2024-02-20 | 2024-05-17 | 上海赫立智能机器有限公司 | Zero plane acquisition method, system, medium and electronic equipment |
| CN117975429A (en) * | 2024-03-29 | 2024-05-03 | 之江实验室 | Method and device for determining physical properties of black hole jet flow and storage medium |
| CN117975429B (en) * | 2024-03-29 | 2024-05-31 | 之江实验室 | Method and device for determining physical properties of black hole jet flow and storage medium |
| CN118134995A (en) * | 2024-04-30 | 2024-06-04 | 成都飞机工业（集团）有限责任公司 | Method, device, equipment and storage medium for arranging measurement points of long and narrow patches |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114943778B (en) | 2023-01-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114943778B (en) | Reference plane determining method, detecting method, device, equipment and storage medium | |
| US20190122375A1 (en) | Data Processing Method, Apparatus, System and Storage Media | |
| CN109685142A (en) | A kind of image matching method and device | |
| CN113989376B (en) | Method and device for acquiring indoor depth information and readable storage medium | |
| CN114998773B (en) | Characteristic mismatching elimination method and system suitable for aerial image of unmanned aerial vehicle system | |
| CN113706587A (en) | Rapid point cloud registration method, device and equipment based on space grid division | |
| CN114511661A (en) | Image rendering method and device, electronic equipment and storage medium | |
| CN115205380A (en) | Volume estimation method and device, electronic equipment and storage medium | |
| CN115422617A (en) | Frame image size measuring method, device and medium based on CAD | |
| CN113344994B (en) | Image registration method, device, electronic equipment and storage medium | |
| CN108427110A (en) | Distance measuring method, device and electronic equipment | |
| CN117291892A (en) | Board foreign matter detection method, device, equipment and storage medium | |
| CN117372663A (en) | Method, device, equipment and storage medium for supplementing log end face shielding | |
| CN115311624B (en) | Slope displacement monitoring method and device, electronic equipment and storage medium | |
| CN112446895A (en) | Checkerboard angular point automatic extraction method, system, equipment and medium | |
| CN114596362B (en) | High-point camera coordinate calculation method and device, electronic equipment and medium | |
| CN116188447A (en) | Method and device for detecting and positioning printed circuit board, electronic equipment and storage medium | |
| CN114820488A (en) | Sample component analysis method, device, equipment and storage medium | |
| CN114283081A (en) | Depth recovery method based on pyramid acceleration, electronic device and storage medium | |
| CN108073934A (en) | Nearly multiimage detection method and device | |
| CN114943769B (en) | Positioning method, positioning device, electronic equipment and medium | |
| CN111695551A (en) | Dial reading method and device, computer equipment and readable storage medium | |
| CN113496142A (en) | Method and device for measuring volume of logistics piece | |
| CN111428729A (en) | Target detection method and device | |
| CN118097039B (en) | Surface structured light three-dimensional point cloud reconstruction method and system based on discrete projection light |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |