CN114943778B - Reference plane determining method, detecting method, device, equipment and storage medium - Google Patents
Reference plane determining method, detecting method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114943778B CN114943778B CN202210881034.0A CN202210881034A CN114943778B CN 114943778 B CN114943778 B CN 114943778B CN 202210881034 A CN202210881034 A CN 202210881034A CN 114943778 B CN114943778 B CN 114943778B
- Authority
- CN
- China
- Prior art keywords
- depth map
- pixel
- pixel points
- pixel point
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a reference surface determining method, a detecting method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a three-dimensional coordinate of each pixel point in a depth map of the board card to be detected, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane; clustering pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets; the target point set is determined from the plurality of pixel points in the set, the reference surface can be automatically determined according to the three-dimensional coordinate fitting reference surface of the pixel points in the target point set, the color interference of the depth map is avoided, the bottom plate area is not required to be marked manually, the problem that the color extraction method and the bottom plate area are not marked manually are inaccurate is solved, the method is suitable for board cards with different colors, the extracted reference surface is high in precision, and the universality is high.
Description
Technical Field
The present invention relates to the field of visual image processing technologies, and in particular, to a method, a device, an apparatus, and a storage medium for determining a reference plane.
Background
In the automatic optical detection process of the PCBA board, the depth image of the PCBA board can be collected through the 3D imaging system, so that defect detection can be carried out based on the depth image.
Each pixel point in the depth image acquired in the automatic optical detection process comprises three-dimensional coordinates (x, y, z), wherein z is the height of a pixel point corresponding to a device on the PCBA board relative to a calibration plane, and the height of the device relative to a bottom plate area of the PCBA board is detected in defect detection, so that the bottom plate area needs to be determined from the depth image to serve as a reference surface, and the reference surface serves as the reference surface to correct the z value of each pixel point in the depth image. Currently, determining the floor area from the depth map includes a color extraction method and a manual marking method, wherein the color extraction method is to identify the floor area based on the color of the floor area, and the manual marking method is to manually mark the floor area from the depth map by experience.
The above method for determining the floor area has the following problems:
(1) When the PCBA board card has interference of different colors of objects such as silk screen printing, copper foil, soldering tin and the like, the color of the bottom plate area cannot be accurately identified by the color extraction method, so that the bottom plate area is inaccurately identified;
(2) Manual marking is too dependent on human experience and the manually marked local area cannot represent the entire floor area.
Disclosure of Invention
The invention provides a reference surface determining method, a detection device, equipment and a storage medium, and aims to solve the problem that in the prior art, a base plate area cannot be accurately extracted as a reference surface through a color extraction method and manual marking, so that the correction precision of a depth map is poor.
In a first aspect, the present invention provides a method for determining a reference plane, including:
acquiring a three-dimensional coordinate of each pixel point in a depth map of a board card to be tested, and fitting a local plane for the pixel point according to the three-dimensional coordinate to obtain a normal vector of the local plane;
clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
determining a target point set from the plurality of pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
and fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
In a second aspect, the present invention provides a detection method for detecting defects of a board to be detected, including:
acquiring an initial depth map of a board card to be tested, wherein each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
determining a reference plane based on the initial depth map;
calculating a distance according to the initial three-dimensional coordinate and the datum plane;
replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
determining a detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determining method of the first aspect.
In a third aspect, the present invention provides a reference plane determining apparatus, including:
the plane fitting module is used for obtaining a three-dimensional coordinate of each pixel point in a depth map of the board card to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane;
the clustering module is used for clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
the target point set determining module is used for determining a target point set from the pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
and the datum plane fitting module is used for fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
In a fourth aspect, the present invention provides a detection apparatus for detecting defects of a board to be detected, including:
the initial depth map acquisition module is used for acquiring an initial depth map of the board card to be detected, and each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
a reference plane determining module for determining a reference plane based on the initial depth map;
the distance calculation module is used for calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference plane;
the depth map updating module is used for replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
the detection result generation module is used for determining the detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determining method of the first aspect.
In a fifth aspect, the present invention provides a board detecting apparatus, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of determining a reference plane according to the first aspect of the invention and/or the method of detecting according to the second aspect.
In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a processor to implement the method for determining a reference plane according to the first aspect of the present invention and/or the method for detecting according to the second aspect of the present invention when executed.
According to the embodiment of the invention, after the local plane is fitted for the pixel points according to the three-dimensional coordinates of the pixel points in the depth map, the normal vector of the local plane is obtained, then the pixel points are clustered according to the three-dimensional coordinates and the normal vector of the pixel points to obtain a plurality of pixel point sets, the target point set of the bottom plate area of the board card to be tested is further determined from the plurality of pixel point sets, and the three-dimensional coordinates of the pixel points in the target point sets are adopted to fit the reference plane, so that the bottom plate area can be determined through the depth map, the reference plane is fitted by adopting the three-dimensional coordinates of the pixel points in the bottom plate area, the color interference in the depth map is avoided, the bottom plate area does not need to be marked manually, the problem that the bottom plate area is not accurately extracted through a color extraction method and manual marking is solved, the board cards with different colors can be applied, the accuracy of the extracted reference plane is high, the accuracy of depth map correction is high based on the reference plane, and the universality is high.
It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for determining a datum plane according to an embodiment of the present invention;
fig. 2A is a flowchart of a method for determining a reference plane according to a second embodiment of the present invention;
FIG. 2B is a schematic diagram of downsampling a depth map according to an embodiment of the present invention;
FIG. 2C is a diagram illustrating a neighborhood in an embodiment of the present invention;
FIG. 2D is a schematic diagram of generating sub-regions in a downsampled depth map in accordance with an embodiment of the present invention;
FIG. 2E is a schematic illustration of an initial mask map of an embodiment of the present invention;
fig. 3 is a flowchart of a detection method according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a reference plane determining apparatus according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a detecting device according to a fifth embodiment of the present invention;
fig. 6 is a schematic structural diagram of a board card detection device according to a sixth embodiment of the present invention.
Detailed Description
In order to make the technical solution of the present invention better understood, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not a whole embodiment. All other embodiments, which can be obtained by a person of ordinary skill in the art without any creative effort based on the embodiments of the present invention, shall fall within the protection scope of the present invention.
Example one
Fig. 1 is a flowchart of a reference plane determining method according to an embodiment of the present invention, where this embodiment is applicable to a case where a reference plane of a board to be detected is extracted to correct a depth map through the reference plane, and the method may be executed by a reference plane determining device, where the reference plane determining device may be implemented in a form of hardware and/or software, and the reference plane determining device may be configured in a board detection apparatus. As shown in fig. 1, the reference plane determining method includes:
s101, obtaining a three-dimensional coordinate of each pixel point in a depth map of the board card to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane.
The board card to be detected can be a circuit board which needs to detect the height from the top of electronic components such as a resistor, a capacitor, an inductor and a chip on the board card to the bottom plate of the board card, the depth map can be an image obtained by placing the board card to be detected on an optical detection machine and extracting the depth after a camera on the optical detection machine takes a picture, each pixel point in the image comprises a three-dimensional coordinate, the three-dimensional coordinate is a coordinate of the position of the pixel point corresponding to the board card to be detected relative to a calibration coordinate system, and in one example, the calibration coordinate system can be the center of a workbench of the optical detection machine.
For each pixel point on the depth map, a neighborhood of the pixel point can be obtained, the three-dimensional coordinate of each pixel point is read from the depth map, a local plane is fitted by adopting the three-dimensional coordinates of a plurality of pixel points in the neighborhood, the local plane enables the sum of the distances from the plurality of pixel points in the neighborhood to the local plane to be minimum, namely the pixel points in the neighborhood are on the plane as far as possible, and the normal vector of the local plane can be obtained after the local plane is fitted.
S102, clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets.
In this embodiment, after each pixel point is fitted to a local plane to obtain a normal vector of the local plane, the pixel points may be clustered by a region growing clustering algorithm to obtain a plurality of pixel point sets, each pixel point set is a set of pixel points with similar characteristics, in one example, a pixel point may be randomly selected as a seed point, euclidean distances between other pixel points in a preset neighborhood and the seed point, normal vector included angles and the like are calculated, when both the euclidean distances and the normal vector included angles of the two pixel points are smaller than corresponding preset differences, it is described that positions of the two pixel points on a board to be tested belong to the same electronic component, the seed point and the pixel point may be added to one pixel point set, then a pixel point is randomly selected from the pixel point set as a seed point, the above steps are repeated until no pixel point is newly added to the pixel point set, the above steps are repeated for the remaining pixel points, and finally a plurality of pixel point sets are obtained.
Of course, besides the region growing clustering algorithm, the pixel points may also be clustered by using clustering algorithms such as K-Means (K-Means) clustering, mean shift clustering, density-based clustering (DBSCAN), and the like, and the algorithm for clustering the pixel points is not limited in this embodiment.
S103, determining a target point set from the plurality of pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested.
In an optional embodiment, for each pixel point set, the point set attribute data of the pixel point set may be obtained, the confidence of each pixel point set is calculated through the point set attribute data, and when the confidence is greater than a preset threshold, the target point set may be determined to be the set of the pixel points corresponding to the backplane area.
Illustratively, the perimeter and the area of the outline of the bottom plate region are generally larger than those of the electronic component, and the number of pixels occupied by the bottom plate region is large, so that the perimeter and the area of the circumscribed rectangle of the pixel point set and the number of pixels included in the pixel point set can be obtained as point set attribute data, then different weights are assigned to the perimeter, the area and the number of pixels of the circumscribed rectangle to perform weighted summation to obtain a confidence, and when the confidence is larger than a preset threshold, the pixel point set is determined to be a set of pixels corresponding to the bottom plate region, namely, a target point set.
And S104, fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set.
In this embodiment, the three-dimensional coordinates of the pixel points in the target point set can be extracted from the depth map, a plane is fitted through the three-dimensional coordinates, the method for specifically fitting the plane is the same as that for fitting the local plane in S101, the plane obtained through fitting is a reference plane, the reference plane can be used to replace the calibration plane to re-correct the depth coordinate values in the three-dimensional coordinates of each pixel point, finally, an updated depth map is obtained, and the board card to be detected is detected through the updated depth map.
According to the embodiment of the invention, after the local plane is fitted for the pixel points according to the three-dimensional coordinates of the pixel points in the depth map, the normal vector of the local plane is obtained, then the pixel points are clustered according to the three-dimensional coordinates and the normal vector of the pixel points to obtain a plurality of pixel point sets, the target point set is further determined from the plurality of pixel point sets, the base plate area can be determined through the depth map according to the three-dimensional coordinates of the pixel points in the base plate area, the base plate area is fitted by adopting the three-dimensional coordinates of the pixel points in the base plate area, the color interference in the depth map is avoided, the base plate area is not required to be marked manually, the problem that the base plate area is not accurately extracted by a color extraction method and manual marking is solved, the method can be suitable for board cards with different colors, the accuracy of the extracted base plate is high, the accuracy of depth map correction is high based on the base plate, and the universality is strong.
Example two
Fig. 2A is a flowchart of a reference plane determining method provided in the second embodiment of the present invention, and the second embodiment of the present invention performs optimization based on the first embodiment, as shown in fig. 2A, the reference plane determining method includes:
s201, downsampling the depth map of the board card to be tested to obtain a downsampled depth map.
As shown in fig. 2B, an example of downsampling a depth map is shown, where the depth map includes a plurality of pixel points, each pixel point includes a three-dimensional coordinate, and the three-dimensional coordinate is a coordinate of a position of the pixel point on the board to be tested with respect to a calibration coordinate system, which may be a center of a workbench of the optical inspection machine in one example.
In an alternative embodiment, during the down-sampling, the down-sampling base N _ unit and the unit down-sampling point N _ down _ unit may be set first, and then the down-sampling target pixel point N _ target may be calculated by the following formula:
N_target = (N_input / N_unit ) × N_down_unit
in the above formula, N _ input is the total number of pixels of the depth map, N _ unit is a downsampling base number, which may be set to 2000, N _ down _unitis a unit downsampling point number, which may be set to 1000-1500, and may be specifically determined according to the total number of pixels of the depth map.
After the number of downsampling target pixel points N _ target is calculated through the above formula, the sampling step lengths in the X direction and the Y direction may be calculated, and in one example, if the sampling step lengths in the X direction and the Y direction are equal, the sampling step length in the X direction may be equal to X n The sampling step in the Y direction may be equal to Y n /M, wherein X n The number of pixels in a line in the X direction in the depth map, Y n The number of a column of pixel points in the Y direction in the depth map is M, and the number of the down-sampling target pixel points N _ target is a value obtained by opening the square of the number of the down-sampling target pixel points N _ target.
Of course, in practical applications, the sampling step size in the X direction and the sampling step size in the Y direction may also be directly set, and the sampling manner of the depth map is not limited in this embodiment.
As shown in fig. 2B, the black-filled pixel points are sampling points, and after sampling, a down-sampling depth map as shown in fig. 2C can be obtained, and as can be seen from fig. 2C, the number of pixel points included in the down-sampling depth map is smaller than the number of pixel points of the depth map before down-sampling in fig. 2B, which can reduce the data amount of subsequent processing and improve the speed of determining the reference plane.
S202, aiming at each pixel point in the down-sampling depth map, determining a neighborhood of the pixel point.
In an optional embodiment, each pixel point in the downsampled depth map may be used as a center, and a region within a preset radius range is determined as a neighborhood of the pixel point, or K neighboring pixel points of the pixel point are selected to form a neighborhood of the pixel point.
As shown in fig. 2C, in an example, for the pixel a, a circular region B with the pixel a as a center may be selected as a neighborhood, and for the pixel C, 8 pixels adjacent to the pixel C may be selected to form a neighborhood D of the pixel C.
In another embodiment, the down-sampling depth map has ordering, that is, each pixel point in the down-sampling depth map may be marked with a sequence number as an index, so that each pixel point may retrieve neighboring pixel points through the ordering to form a neighborhood, and thus neighboring pixel points of each pixel point may be quickly found through the ordering of the depth map to form a neighborhood.
Of course, in practical applications, other ways may be used by those skilled in the art to determine the neighborhood of a pixel point.
S203, three-dimensional coordinates of pixel points in the neighborhood are obtained, and a local plane is fitted according to the three-dimensional coordinates, so that a normal vector of the local plane is obtained.
In an optional embodiment, for each neighborhood, the three-dimensional coordinates of each pixel point in the neighborhood can be read from the down-sampling depth map, and then a local plane is fitted to the three-dimensional coordinates of the pixel points in the neighborhood by using a least square method to obtain a normal vector of the local plane, which specifically includes the following steps:
establishing a local plane equation:
aX + bY + cZ - d = 0
according to the principle of the least square method, the fitted local plane is to be optimal, and actually, the sum of squares of distances from the space points corresponding to the three-dimensional coordinates associated with each pixel point in the neighborhood to the local plane is minimized, namely, the solution:
in the above formula, d i Is a point p in the neighborhood i =（x i ，y i ，z i ) Distance to local plane, i.e. d i =| ax i + by i + z i + d | to minimize the sum of squares e of the distances, the solution can be performed by SVD matrix decomposition, which includes the following steps:
and (3) solving a center of mass m = (x, y, z) of the neighborhood, wherein the center of mass m is the center point of all points in the neighborhood, namely:
n is the number of points contained in the neighborhood, and there are:
assuming a matrix:
the purpose of fitting the local plane is that all points in the neighborhood are on the local plane, and in practical cases, some points are out of the local plane, and the purpose of fitting is that the sum of distances from the local plane to all points is minimum, that is, the objective function of fitting optimization is:
min||AX||
the constraint condition is | | | X | =1, and Singular Value Decomposition (Singular Value Decomposition) can be performed on the matrix a:
in the singular value decomposition, U is an m × m matrix, D is an m × n matrix, D is all 0 except for elements on a main diagonal, each element on the main diagonal is called a singular value, V is an m × n matrix, and U and V are unitary matrices.
From the decomposition result, eigenvalues and eigenvectors corresponding to the matrix a can be obtained, the minimum value of e is the minimum eigenvalue of the matrix a, the corresponding eigenvectors are the plane parameters a, b, c, and the normal vector n = (a, b, c) can be obtained.
Determining a neighborhood for each pixel point in the down-sampling depth map, and fitting a local plane for the neighborhood to obtain a normal vector of the local plane.
And S204, clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets.
In an optional embodiment, for pixel points in a down-sampling depth map, one pixel point can be randomly selected as a seed point, other pixel points in a preset neighborhood are determined, the Euclidean distance between the other pixel points and the seed point is calculated by adopting a three-dimensional coordinate aiming at each other pixel point, the normal vector of the other pixel points and the normal vector of the seed point are adopted to calculate the vector included angle, when the Euclidean distance is smaller than a preset distance threshold value and the normal vector included angle is smaller than the preset included angle, the other pixel points and the seed point are added into a pixel point set, then one pixel point is randomly selected from the pixel point set to serve as the seed point, the step of determining the other pixel points in the preset neighborhood is returned, and the pixel point set is output until no new pixel point exists in the pixel point set; and randomly selecting one pixel point from the rest pixel points as a seed point, returning to the step of determining other pixel points in the preset neighborhood until all the pixel points are added into the pixel point set, and obtaining a plurality of pixel point sets.
The clustering process is a region growing clustering algorithm, and the specific process is as follows:
s1, randomly selecting a pixel point from a down-sampling depth map as a seed point, setting a neighborhood of a search radius r, traversing the down-sampling depth map, searching other pixel points of which the normal vector included angles and the Euclidean distances meet a threshold value in the neighborhood of the radius r, and adding the pixel point and the seed point into a pixel point set U1 in the same category;
s2, randomly selecting a pixel point from the pixel point set U1 as a new seed point, and repeating the step S1;
and S3, sequentially iterating until the pixel point set U1 is not newly added with pixel points, and outputting the iteration to obtain the pixel point set U1.
S4, repeating S1 to S3 for other pixel points except the pixel point set U1, and sequentially outputting pixel point sets U2, U3, … … and Un;
and S5, if the number of the residual pixel points is less than the preset number or the iteration frequency exceeds the preset frequency, stopping iteration and outputting all pixel point sets.
In the embodiment, for each pixel point in the down-sampling image, similar pixel points are searched by the distance and the normal vector smaller than the preset threshold, and the pixel points belonging to the same plane can be searched.
Of course, in practical applications, besides the clustering method, clustering algorithms such as K-Means (K-Means) clustering, mean shift clustering, density-based clustering (DBSCAN) and the like may be used to cluster the pixel points, and the algorithm for clustering the pixel points is not limited in this embodiment.
S205, point set attribute data of a plurality of pixel point sets are obtained.
For each pixel point set, the number of the pixel points included in the pixel point set is different, the perimeter and the area of the minimum bounding rectangle of all the pixel points included in each pixel point set and the number of the pixel points can be used as the attribute data of the pixel point set, and certainly, the attribute data of the pixel point set can also include other data, for example, the average value of the depth coordinates in the three-dimensional coordinates associated with the pixel points and the like.
And S206, calculating the confidence coefficient of the pixel point set by adopting the point set attribute data.
Specifically, a weight may be given to each point set attribute data of the pixel point set, and then the confidence of the pixel point set is obtained through weighted summation, exemplarily, the point set attribute data includes the perimeter and the area of the minimum circumscribed rectangle of all the pixel points included in the pixel point set, and the number of the pixel points, weights may be set for the perimeter, the area, and the number of the pixel points, respectively, and the weighted sum of the perimeter, the area, and the number of the pixel points is calculated as the confidence of the pixel point set.
And S207, determining the pixel point set with the confidence coefficient larger than a preset threshold value as a target point set.
Generally speaking, on a board card to be tested, the areas of the tops of components such as resistors, capacitors, inductors, chips and the like are usually smaller, the number of pixels included in the bottom plate area corresponding to a down-sampling depth map is smaller, the perimeter and the area of an external rectangle of the bottom plate area are smaller, the area of the bottom plate area is usually larger, the number of pixels included in the bottom plate area corresponding to the down-sampling depth map is larger, a pixel point set belonging to the bottom plate area can be screened out through confidence, specifically, a confidence threshold can be set, when the confidence is larger than the confidence threshold, the pixel point set is explained as the pixel point set of the bottom plate area, the pixel point set is taken as a target point set, it needs to be explained that the number of the target point set is more than one, in addition, the confidence threshold can be determined according to the bottom plate area exposed on the board card to be tested, the density of electronic components and the top area of the electronic components, different confidence thresholds can be set for different board cards to be tested, and the target point sets belonging to the target point set belonging to the bottom plate area through the confidence thresholds.
And S208, generating a mask diagram of the bottom plate area of the board card to be detected by adopting the target point set.
In an optional embodiment, for each target point set, a sub-region including pixel points in the target point set is generated in the downsampling depth map, a plurality of sub-regions are merged to obtain an initial mask region, a preset color is filled in the initial mask region to obtain an initial mask map, and the initial mask map is upsampled to obtain a mask map with the size same as that of the depth map, so that the mask map is used as a mask map of a bottom plate region of the board card to be tested.
In an example, since each target point set includes a plurality of pixel points, a minimum circumscribed polygon including the plurality of pixel points may be generated, where the minimum circumscribed polygon is a sub-region including the pixel points in the target point set, as shown in fig. 2D, the sub-regions A1, A2, and A3 are generated by the plurality of target point sets, the sub-regions A1, A2, and A3 are merged to obtain an initial mask region, and the initial mask region is filled with a preset color to obtain an initial mask image as shown in fig. 2E, as can be seen from fig. 2D to fig. 2E, the sub-region A1 is communicated with the sub-region A2, and the initial mask region a12 is obtained after merging, where the size of the initial mask image is the same as that of the downsampled depth map, and the mask region is filled with white, and certainly may be filled with other colors.
The initial mask map is generated based on the downsampling depth map, and the initial mask map may be enlarged by upsampling, for example, the initial mask map is enlarged to a mask map with the same size as the depth map, where the mask map is a mask map of a bottom plate area of the board card to be tested, that is, a white-filled area in the mask map of the bottom plate area of the board card to be tested is a bottom plate area.
S209, determining a bottom plate area from the depth map according to the mask map, and fitting a reference plane by adopting the three-dimensional coordinates of the pixel points in the bottom plate area.
In an example, a mask map may be superimposed on the depth map, since the mask map and the depth map have the same size, a region on the mask map, which is filled with white color, corresponding to the region in the depth map is a bottom plate region, three-dimensional coordinates of all pixel points of the bottom plate region may be read out, and a plane is fitted through the read three-dimensional coordinates of the bottom plate region, where the plane obtained by fitting is a reference plane, and a specific fitting process may refer to S203, which is not described in detail herein.
In the embodiment, after a depth map is downsampled to obtain a downsampled depth map, a neighborhood of each pixel point in the downsampled depth map is determined and a local plane is fitted for the neighborhood to obtain a normal vector, clustering is further performed according to three-dimensional coordinates and normal vectors of pixel points in the downsampled depth map to obtain a plurality of pixel point sets, the confidence coefficient of each pixel point set is calculated through point set attribute data of the pixel point sets, the pixel point set with the confidence coefficient larger than a threshold value is used as a target point set of a bottom plate region, a sub-region containing the pixel points of the target point set is generated in the depth map through the target point set, an initial mask map is obtained after a sub-region is filled with preset colors, a mask map of the bottom plate region of a board to be tested is obtained after the initial mask map is upsampled, the mask map is superimposed into the depth map, and the fitting plane of the pixel points in the depth map covered by the mask region is read to be a reference plane. The bottom plate area can be determined through the depth map, the reference surface is fitted by adopting the three-dimensional coordinates of the pixel points in the bottom plate area, color interference in the depth map is avoided, the bottom plate area does not need to be marked manually, the problem that the color extraction method and the manual marking extraction method are inaccurate in the bottom plate area is solved, the method is suitable for board cards with different colors, the extracted reference surface is high in precision, and the universality is high.
Furthermore, the depth map is down-sampled firstly, the mask map is obtained, then the mask map with the same size as the depth map is obtained through up-sampling, the number of pixel points needing to be processed is reduced through down-sampling, and the speed of determining the reference plane is improved.
Furthermore, a pixel point set of similar pixel points is determined by clustering through distance and normal vectors, the confidence coefficient is calculated through attribute data of the pixel point set, the pixel point set with the confidence coefficient larger than a preset threshold value is determined as a target point set of the bottom plate area to determine the bottom plate area, the influence of the color of the bottom plate area is avoided, the bottom plate area can be accurately determined, the method is suitable for bottom plates of various colors, and the universality is high.
EXAMPLE III
Fig. 3 is a flowchart of a detection method according to a third embodiment of the present invention, where the method is applicable to a situation where a depth map is corrected to detect a defect of a board, and the method may be executed by a detection device, where the detection device may be implemented in a form of hardware and/or software, and the detection device may be configured in a board detection device. As shown in fig. 3, the detection method includes:
s301, obtaining an initial depth map of the board card to be tested, wherein each pixel point in the initial depth map comprises an initial three-dimensional coordinate.
The board card to be tested may be a circuit board that needs to detect a height from a top of an electronic component to a bottom board of the board card, in an example, the initial depth map may be a depth map in which a calibration plane is used as a measurement reference, in the initial depth map, a three-dimensional coordinate value of each pixel point is based on the calibration plane, for example, a center point of the calibration plane is used as an origin of a coordinate system, in the three-dimensional coordinates (x, y, z) of each pixel point, x and y represent a position of the electronic component on the board card corresponding to the pixel point, and z represents a distance from the top of the electronic component to the calibration plane, generally, the calibration plane is a surface of a workbench of the board card testing apparatus, and may be a plane on which the board card to be tested is placed, for example. After the board card detection equipment acquires images of the board card through the camera, the distance from the top of each electronic element on the board card to a calibration plane can be calculated through calibration parameters of the camera to obtain a z coordinate, and an x coordinate and a y coordinate are determined through an imaging relation, so that each pixel point and a three-dimensional coordinate are associated to obtain an initial depth map.
And S302, determining a reference plane based on the initial depth map.
In the embodiment of the invention, for the pixel points in the initial depth map, the pixel points are fitted to the local plane according to the three-dimensional coordinates to obtain the normal vector of the local plane, the pixel points are clustered according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets, the target point sets are determined from the plurality of pixel point sets, and the three-dimensional coordinates of the pixel points in the target point sets are adopted to fit the reference plane.
And S303, calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference plane.
Specifically, the distance from the spatial point to the plane may be calculated, and in this embodiment, the equation of the reference plane obtained in S302 is assumed to be:
Ax+By+Cz+D=0
the initial three-dimensional coordinates of the pixel point are (x 0, y0, z 0), then the distance is calculated as follows:
the distance d is a distance from a position of the pixel point on the board card to be tested to the reference plane, that is, a distance from the position of the pixel point on the board card to be tested to the bottom plate area of the board card to be tested.
And S304, replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map.
After the distance is obtained by calculation, the distance of each pixel point can be used to replace a depth coordinate value z in the initial depth map, so as to obtain a target depth map, that is, a depth coordinate value in a three-dimensional coordinate associated with each pixel point in the target depth map is a distance from a component on the board card corresponding to the pixel point to the reference plane, that is, a height from the top of the component to the bottom board area, that is, for the target depth map, a three-dimensional coordinate of each pixel point is (x 0, y0, d 0).
S305, determining the detection result of the board card to be detected by adopting the target depth map and a preset standard depth map.
The standard depth map can be a depth map which is manufactured in advance, wherein the depth map takes a bottom plate area of a qualified board card as a reference surface and takes the distance from the top of a component to the bottom plate area as a coordinate z value, after a target depth map is obtained, the coordinate d0 value of each pixel point in the target depth map is read, the coordinate z value of the corresponding pixel point in the standard depth map is read, the d0 value and the z value are compared, an error is calculated, if the error is within a preset range, the height from the top of the component corresponding to the pixel point to the bottom plate area is qualified, otherwise, the height is unqualified, and the detection result of each pixel point is generated to be used as the detection result of the board card to be detected.
In this embodiment, after obtaining the initial depth map of the board card to be detected, the reference plane is determined by the reference plane determining method of the first embodiment or the second embodiment, the distance is further calculated according to the initial three-dimensional coordinates of the pixel points and the reference plane, the distance is used to replace the depth coordinate value in the initial three-dimensional coordinates to obtain the target depth map, and the target depth map and the preset standard depth map are used to determine the detection result of the board card to be detected.
Example four
Fig. 4 is a schematic structural diagram of a reference plane determining apparatus according to a fourth embodiment of the present invention. As shown in fig. 4, the reference surface determining apparatus includes:
the plane fitting module 401 is configured to obtain a three-dimensional coordinate of each pixel point in a depth map of the board card to be tested, and fit a local plane for the pixel point according to the three-dimensional coordinate to obtain a normal vector of the local plane;
a clustering module 402, configured to cluster the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
a target point set determining module 403, configured to determine a target point set from the multiple pixel point sets, where the target point set is a pixel point set in a bottom plate area of the board to be tested;
and a reference plane fitting module 404, configured to fit a reference plane according to the three-dimensional coordinates of the pixel points in the target point set.
Optionally, the plane fitting module 401 includes:
the down-sampling module is used for down-sampling the depth map of the board card to be detected to obtain a down-sampling depth map;
a neighborhood determination module, configured to determine, for each pixel point in the downsampled depth map, a neighborhood of the pixel point;
and the fitting module is used for acquiring the three-dimensional coordinates of the pixel points in the neighborhood and fitting a local plane according to the three-dimensional coordinates to obtain a normal vector of the local plane.
Optionally, the neighborhood determination module includes:
the neighborhood determining unit is used for determining a region in a preset radius range as a neighborhood of the pixel point by taking each pixel point in the sampling depth map as a center; or; and selecting K adjacent pixel points of the pixel points to form a neighborhood of the pixel points.
Optionally, the fitting module comprises:
and the fitting unit is used for acquiring the three-dimensional coordinates of each pixel point in each neighborhood, and fitting the three-dimensional coordinates of the pixel points in the neighborhood to a local plane by adopting a least square method to obtain a normal vector of the local plane.
Optionally, the clustering module 402 includes:
the seed point selection unit is used for randomly selecting one pixel point as a seed point and determining other pixel points in a preset range;
the distance sum normal vector difference value calculating unit is used for calculating Euclidean distances between the other pixel points and the seed point by adopting the three-dimensional coordinates and calculating normal vector included angles by adopting normal vectors of the other pixel points and the seed point;
the point set generating unit is used for adding the other pixel points and the seed point into a pixel point set when the Euclidean distance is smaller than a preset distance threshold and the normal vector included angle is smaller than a preset included angle;
the first seed point updating unit is used for randomly selecting a pixel point from the pixel point set as a seed point, returning to the seed point selecting unit and outputting the pixel point set until no new pixel point is added in the pixel point set;
and the second seed point updating unit is used for randomly selecting one pixel point from the rest pixel points as a seed point, and returning to the step of the seed point selecting unit until all the pixel points are added into the pixel point set to obtain a plurality of pixel point sets.
Optionally, the target point set determining module 403 includes:
a point set attribute data acquisition unit for acquiring point set attribute data of a plurality of pixel point sets;
a confidence coefficient calculation unit for calculating the confidence coefficient of the pixel point set by using the point set attribute data;
the target point determining and calculating unit is used for determining the pixel point set with the confidence coefficient larger than a preset threshold value as a target point set;
optionally, the reference plane fitting module 404 includes:
the mask image generating module is used for generating a mask image of the bottom plate area of the board card to be detected by adopting the target point set;
and the fitting module is used for determining a bottom plate area from the depth map according to the mask map and fitting a reference plane by adopting the three-dimensional coordinates of the pixel points in the bottom plate area.
Optionally, the mask map generating module includes:
a sub-region generating unit, configured to generate, for each target point set, a sub-region including a pixel point in the target point set in the downsampled depth map;
the initial mask image generating unit is used for solving a union set of a plurality of subregions to obtain an initial mask region and filling a preset color into the initial mask region to obtain an initial mask image;
and the up-sampling unit is used for up-sampling the initial mask image to obtain a mask image with the same size as the depth image so as to be used as the mask image of the bottom plate area of the board card to be detected.
The reference plane determining device provided by the embodiment of the invention can execute the reference plane determining method provided by the first embodiment and the second embodiment of the invention, and has corresponding functional modules and beneficial effects of the executing method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a detecting device according to a fifth embodiment of the present invention. The detection device is used for detecting defects of the board, and as shown in fig. 5, the detection device comprises:
the initial depth map acquiring module 501 is configured to acquire an initial depth map of a board to be tested, where each pixel point in the initial depth map includes an initial three-dimensional coordinate;
a reference plane determining module 502 for determining a reference plane based on the initial depth map;
a distance calculating module 503, configured to calculate a distance according to the initial three-dimensional coordinates of the pixel point and the reference plane;
a depth map updating module 504, configured to replace the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
a detection result generation module 505, configured to determine a detection result of the board card to be detected by using the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determining method described in the first embodiment or the second embodiment.
The detection device provided by the embodiment of the invention can execute the detection method provided by the third embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
EXAMPLE six
Fig. 6 shows a schematic structural diagram of a board detection apparatus 60 that can be used to implement the embodiment of the present invention. Board detection apparatus is intended to mean an apparatus containing various forms of digital computers, such as, for example, an apparatus containing a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers.
As shown in fig. 6, the board detection apparatus includes at least one processor 61, and a memory communicatively connected to the at least one processor 61, such as a Read Only Memory (ROM) 62, a Random Access Memory (RAM) 63, and the like, where the memory stores a computer program executable by the at least one processor, and the processor 61 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 62 or the computer program loaded from the storage unit 68 into the Random Access Memory (RAM) 63. In the RAM 63, various programs and data necessary for the operation of the board detection apparatus 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other by a bus 64. An input/output (I/O) interface 65 is also connected to bus 64.
A plurality of components in the board detection apparatus 60 are connected to the I/O interface 65, including: an input unit 66 such as a keyboard, a mouse, a camera that acquires a depth image, and the like; an output unit 67 such as various types of displays, speakers, and the like; a storage unit 68 such as a magnetic disk, optical disk, or the like; and a communication unit 69 such as a network card, modem, wireless communication transceiver, etc. The communication unit 69 allows the board detection device 60 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the reference plane determining method, and/or the detecting method, may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 68. In some embodiments, part or all of the computer program may be loaded and/or installed onto the board detection apparatus 60 via the ROM 62 and/or the communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the above-described reference plane determining method, and/or the detecting method, may be performed. Alternatively, in other embodiments, the processor 41 may be configured to perform the reference plane determination method, and/or the detection method, in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described herein may be implemented on a board detection apparatus having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the board detection apparatus. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired result of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations, and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (11)
1. A reference plane determining method, comprising:
acquiring a three-dimensional coordinate of each pixel point in a depth map of a board to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane;
clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
determining a target point set from the pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
for each target point set, generating a sub-region containing pixel points in the target point set in a down-sampling depth map, wherein the down-sampling depth map is a depth map obtained after down-sampling the depth map;
obtaining an initial mask area by solving a union set of a plurality of subregions, and filling a preset color into the initial mask area to obtain an initial mask image;
sampling the initial mask image to obtain a mask image with the same size as the depth image, and taking the mask image as a mask image of a bottom plate area of the board card to be detected;
and determining a bottom plate area from the depth map according to the mask map, and fitting a reference plane by adopting the three-dimensional coordinates of the pixel points in the bottom plate area.
2. The method for determining the reference plane according to claim 1, wherein the obtaining of the three-dimensional coordinates of each pixel point in the depth map of the board card to be tested and the fitting of the pixel point to the local plane according to the three-dimensional coordinates to obtain the normal vector of the local plane comprises:
downsampling the depth map of the board card to be tested to obtain a downsampled depth map;
determining a neighborhood of the pixel points for each pixel point in the downsampled depth map;
and acquiring the three-dimensional coordinates of the pixel points in the neighborhood, and fitting a local plane according to the three-dimensional coordinates to obtain a normal vector of the local plane.
3. The reference plane determination method of claim 2, wherein determining, for each pixel point in the downsampled depth map, a neighborhood of the pixel point comprises:
determining a region within a preset radius range as a neighborhood of the pixel point by taking each pixel point in the downsampled depth map as a center;
or;
k adjacent pixel points of the pixel points are selected to form a neighborhood of the pixel points.
4. The method for determining a reference plane according to claim 2, wherein the obtaining of the three-dimensional coordinates of the pixel points in the neighborhood and fitting a local plane according to the three-dimensional coordinates to obtain a normal vector of the local plane comprises:
and aiming at each neighborhood, acquiring the three-dimensional coordinates of each pixel point in the neighborhood, and fitting a local plane to the three-dimensional coordinates of the pixel points in the neighborhood by adopting a least square method to obtain a normal vector of the local plane.
5. The method for determining the reference plane according to any one of claims 1 to 4, wherein the clustering the pixels according to the three-dimensional coordinates of the pixels and the normal vector to obtain a plurality of pixel point sets comprises:
randomly selecting one pixel point as a seed point, and determining other pixel points in a preset neighborhood;
aiming at each other pixel point, calculating Euclidean distances between the other pixel points and the seed point by adopting the three-dimensional coordinates, and calculating a normal vector included angle by adopting normal vectors of the other pixel points and normal vectors of the seed point;
when the Euclidean distance is smaller than a preset distance threshold value and the normal vector included angle is smaller than a preset included angle, adding the other pixel points and the seed point into a pixel point set;
randomly selecting a pixel point from the pixel point set as a seed point, returning to the step of determining other pixel points in a preset neighborhood, and outputting the pixel point set until no new pixel points exist in the pixel point set;
and randomly selecting one pixel point from the rest pixel points as a seed point, returning to the step of determining other pixel points in the preset neighborhood until all the pixel points are added into the pixel point set, and obtaining a plurality of pixel point sets.
6. The method of determining a reference plane as claimed in any one of claims 2 to 4, wherein said determining a set of target points from a plurality of said sets of pixels comprises:
acquiring point set attribute data of a plurality of pixel point sets;
calculating the confidence of the pixel point set by adopting the point set attribute data;
and determining the pixel point set with the confidence coefficient larger than a preset threshold value as a target point set.
7. A detection method is characterized in that the method is used for detecting the defects of a board card to be detected, and comprises the following steps:
acquiring an initial depth map of a board card to be tested, wherein each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
determining a reference plane based on the initial depth map;
calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference surface;
replacing the depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
determining a detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determination method of any one of claims 1 to 6.
8. A reference plane determining apparatus, comprising:
the plane fitting module is used for obtaining a three-dimensional coordinate of each pixel point in a depth map of the board card to be tested, and fitting a local plane for the pixel points according to the three-dimensional coordinate to obtain a normal vector of the local plane;
the clustering module is used for clustering the pixel points according to the three-dimensional coordinates of the pixel points and the normal vector of the local plane to obtain a plurality of pixel point sets;
the target point set determining module is used for determining a target point set from the pixel point sets, wherein the target point set is a pixel point set of a bottom plate area of the board card to be tested;
the datum plane fitting module is used for fitting a datum plane according to the three-dimensional coordinates of the pixel points in the target point set;
the reference plane fitting module comprises:
the mask image generating module is used for generating a mask image of the bottom plate area of the board card to be detected by adopting the target point set;
the fitting module is used for determining a bottom plate area from the depth map according to the mask map and fitting a datum plane by adopting three-dimensional coordinates of pixel points in the bottom plate area;
the mask map generation module comprises:
a sub-region generating unit, configured to generate, for each target point set, a sub-region including pixel points in the target point set in a downsampled depth map, where the downsampled depth map is a depth map obtained by downsampling the depth map;
the initial mask image generating unit is used for solving a union set of a plurality of subregions to obtain an initial mask region and filling a preset color into the initial mask region to obtain an initial mask image;
and the up-sampling unit is used for up-sampling the initial mask image to obtain a mask image with the same size as the depth image so as to be used as the mask image of the bottom plate area of the board card to be detected.
9. The utility model provides a detection device, its characterized in that for carry out defect detection to the integrated circuit board that awaits measuring, include:
the initial depth map acquisition module is used for acquiring an initial depth map of the board card to be detected, and each pixel point in the initial depth map comprises an initial three-dimensional coordinate;
a reference plane determining module for determining a reference plane based on the initial depth map;
the distance calculation module is used for calculating the distance according to the initial three-dimensional coordinates of the pixel points and the reference plane;
the depth map updating module is used for replacing a depth coordinate value in the initial three-dimensional coordinate with the distance to obtain a target depth map;
the detection result generation module is used for determining the detection result of the board card to be detected by adopting the target depth map and a preset standard depth map;
wherein the reference plane is determined by the reference plane determination method of any one of claims 1 to 6.
10. The utility model provides a board card check out test set which characterized in that, board card check out test set includes:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the datum level determination method of any one of claims 1-6 and/or the detection method of claim 7.
11. A computer-readable storage medium, having stored thereon computer instructions for causing a processor to execute a method of determining a reference plane as claimed in any one of claims 1 to 6 and/or a method of detecting as claimed in claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210881034.0A CN114943778B (en) | 2022-07-26 | 2022-07-26 | Reference plane determining method, detecting method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210881034.0A CN114943778B (en) | 2022-07-26 | 2022-07-26 | Reference plane determining method, detecting method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114943778A CN114943778A (en) | 2022-08-26 |
| CN114943778B true CN114943778B (en) | 2023-01-13 |
Family
ID=82910732
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210881034.0A Active CN114943778B (en) | 2022-07-26 | 2022-07-26 | Reference plane determining method, detecting method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114943778B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115854892B (en) * | 2022-12-05 | 2024-02-02 | 上海赫立智能机器有限公司 | Zero plane correction method for PCB detection |
| CN117975429B (en) * | 2024-03-29 | 2024-05-31 | 之江实验室 | Method and device for determining physical properties of black hole jet flow and storage medium |
| CN118134995A (en) * | 2024-04-30 | 2024-06-04 | 成都飞机工业（集团）有限责任公司 | Method, device, equipment and storage medium for arranging measurement points of long and narrow patches |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111402251A (en) * | 2020-04-01 | 2020-07-10 | 苏州苏映视图像软件科技有限公司 | Visual inspection method and system for 3D defect detection |
| CN111932537A (en) * | 2020-10-09 | 2020-11-13 | 腾讯科技（深圳）有限公司 | Object deformation detection method and device, computer equipment and storage medium |
| CN111986299A (en) * | 2019-05-24 | 2020-11-24 | 北京京东尚科信息技术有限公司 | Point cloud data processing method, device, equipment and storage medium |
| CN112200771A (en) * | 2020-09-14 | 2021-01-08 | 浙江大华技术股份有限公司 | Height measuring method, device, equipment and medium |
| CN113102269A (en) * | 2021-04-07 | 2021-07-13 | 沈阳建筑大学 | Prefabricated part finished product quality detection system based on three-dimensional stereoscopic vision and application thereof |
| CN113689420A (en) * | 2021-09-03 | 2021-11-23 | 凌云光技术股份有限公司 | Method and device for detecting welding spot defects of circuit board and detection equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105825499A (en) * | 2016-03-09 | 2016-08-03 | 京东方科技集团股份有限公司 | Reference plane determination method and determination system |
| CN113850726A (en) * | 2020-06-28 | 2021-12-28 | 华为技术有限公司 | Image transformation method and device |
-
2022
- 2022-07-26 CN CN202210881034.0A patent/CN114943778B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111986299A (en) * | 2019-05-24 | 2020-11-24 | 北京京东尚科信息技术有限公司 | Point cloud data processing method, device, equipment and storage medium |
| CN111402251A (en) * | 2020-04-01 | 2020-07-10 | 苏州苏映视图像软件科技有限公司 | Visual inspection method and system for 3D defect detection |
| CN112200771A (en) * | 2020-09-14 | 2021-01-08 | 浙江大华技术股份有限公司 | Height measuring method, device, equipment and medium |
| CN111932537A (en) * | 2020-10-09 | 2020-11-13 | 腾讯科技（深圳）有限公司 | Object deformation detection method and device, computer equipment and storage medium |
| CN113102269A (en) * | 2021-04-07 | 2021-07-13 | 沈阳建筑大学 | Prefabricated part finished product quality detection system based on three-dimensional stereoscopic vision and application thereof |
| CN113689420A (en) * | 2021-09-03 | 2021-11-23 | 凌云光技术股份有限公司 | Method and device for detecting welding spot defects of circuit board and detection equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于点云三维重建的装配件尺寸测量系统;余永维等;《组合机床与自动化加工技术》;20220630;第105页摘要-第109页第4节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114943778A (en) | 2022-08-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114943778B (en) | Reference plane determining method, detecting method, device, equipment and storage medium | |
| US10268917B2 (en) | Pre-segment point cloud data to run real-time shape extraction faster | |
| KR100367303B1 (en) | Vector correlation system for automatically locating patterns in an image | |
| CN108537834A (en) | A kind of volume measuring method, system and depth camera based on depth image | |
| CN110910445B (en) | Object size detection method, device, detection equipment and storage medium | |
| CN114998773B (en) | Characteristic mismatching elimination method and system suitable for aerial image of unmanned aerial vehicle system | |
| CN115205380A (en) | Volume estimation method and device, electronic equipment and storage medium | |
| CN115661115A (en) | Component detection method, device, electronic equipment and storage medium | |
| CN116152306A (en) | Method, device, apparatus and medium for determining masonry quality | |
| CN108427110A (en) | Distance measuring method, device and electronic equipment | |
| CN115311624B (en) | Slope displacement monitoring method and device, electronic equipment and storage medium | |
| CN114596362B (en) | High-point camera coordinate calculation method and device, electronic equipment and medium | |
| CN116205889A (en) | Offset detection method, offset detection device, electronic equipment and storage medium | |
| CN111199191A (en) | Pointer instrument automatic reading method based on scale searching | |
| CN114820488A (en) | Sample component analysis method, device, equipment and storage medium | |
| CN113160414B (en) | Automatic goods allowance recognition method, device, electronic equipment and computer readable medium | |
| CN112833775B (en) | Object packaging volume detection method and device, terminal and readable storage medium | |
| CN108073934A (en) | Nearly multiimage detection method and device | |
| CN114943769B (en) | Positioning method, positioning device, electronic equipment and medium | |
| CN112446895A (en) | Checkerboard angular point automatic extraction method, system, equipment and medium | |
| CN116258714B (en) | Defect identification method and device, electronic equipment and storage medium | |
| CN113129255A (en) | Method, computing device, system and storage medium for detecting package | |
| CN112733852B (en) | Region determination method, device, computer equipment and readable storage medium | |
| CN112733817B (en) | Method for measuring precision of point cloud layer in high-precision map and electronic equipment | |
| CN114581647A (en) | Confidence post-processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |