CN115035031A

CN115035031A - Defect detection method and device for PIN (personal identification number) PIN, electronic equipment and storage medium

Info

Publication number: CN115035031A
Application number: CN202210492875.2A
Authority: CN
Inventors: 周赏; 史为平; 刘羽; 周璐; 李铭
Original assignee: Zhejiang Huaray Technology Co Ltd
Current assignee: Zhejiang Huaray Technology Co Ltd
Priority date: 2022-05-07
Filing date: 2022-05-07
Publication date: 2022-09-09

Abstract

The application discloses a defect detection method and device of a PIN needle, electronic equipment and a storage medium, which belong to the technical field of defect detection, and the method comprises the following steps: the method comprises the steps of acquiring point cloud data of a circuit board to be detected acquired by N line laser cameras, generating a first depth map of a PIN needle on the circuit board to be detected based on the point cloud data acquired by each line laser camera, judging whether the PIN needle on the circuit board to be detected is missing or not based on the first depth map and a second depth map of the PIN needle on a reference circuit board, and outputting a defect detection result at least comprising position indication information of the missing PIN needle on the circuit board to be detected, wherein N is determined based on the size of the circuit board to be detected and the acquisition range of a single line laser camera, and the PIN needle on the reference circuit board does not have defects, so that a scheme for detecting the defects of the PIN needle by means of the line laser cameras is provided.

Description

Defect detection method and device for PIN (personal identification number) PIN, electronic equipment and storage medium

Technical Field

The present application relates to the field of defect detection technologies, and in particular, to a method and an apparatus for detecting defects of a PIN, an electronic device, and a storage medium.

Background

The PIN is a common electronic device connector, has the advantages of reliable communication connection and convenient disassembly and assembly, and is widely applied to the microelectronic and automotive electronics industries due to the advantages.

Generally, the mounting positions of PIN PINs on a circuit board are strictly required, and if the mounting positions of the PIN PINs are deviated, the PIN PINs cannot be mounted or communication fails, so that the defect detection of the PIN PINs is particularly important. However, how to detect defects of the PIN is an urgent technical problem to be solved.

Disclosure of Invention

The embodiment of the application provides a defect detection method and device for a PIN, electronic equipment and a storage medium, and is used for providing a reasonable defect detection scheme for the PIN.

In a first aspect, an embodiment of the present application provides a defect detection method for a PIN, including:

acquiring point cloud data of a circuit board to be detected, acquired by N line laser cameras, wherein N is determined based on the size of the circuit board to be detected, the PIN setting mode and the acquisition range of a single line laser camera;

generating a first depth map of the PIN needle on the circuit board to be detected based on the point cloud data acquired by each line of laser cameras;

judging whether the PIN on the circuit board to be detected is missing or not based on the first depth map and a second depth map of the PIN on the reference circuit board, wherein the PIN on the reference circuit board is not defective;

and outputting a defect detection result, wherein the defect detection result at least comprises the position indication information of the missing PIN needle on the circuit board to be detected.

In some embodiments, the N line laser cameras include a master camera and N-1 slave cameras, and the generating of the first depth map of the PIN on the circuit board to be detected based on the point cloud data acquired by each line laser camera includes:

converting the point cloud data collected by each slave camera into the coordinate system of the master camera based on the conversion relation between the coordinate systems of the slave cameras to the master camera;

fusing the point cloud data converted by each slave camera and the point cloud data of the master camera to obtain complete point cloud data of the circuit board to be detected;

and generating the first depth map based on the complete point cloud data.

In some embodiments, generating the first depth map based on the complete point cloud data comprises:

selecting target point cloud data of the PIN needle on the circuit board to be detected from the complete point cloud data;

and performing projection processing on the target point cloud data to obtain the first depth map.

In some embodiments, when at least two line laser cameras are arranged side by side, a common view area and a data acquisition delay exist between adjacent line laser cameras arranged side by side, and the data acquisition delay satisfies that the adjacent cameras do not repeatedly perform data acquisition on the common view area.

In some embodiments, the transformation relation between the coordinate systems of each slave camera and the master camera is obtained by calibrating a special-shaped calibration block, the top surface of the special-shaped calibration block is provided with at least one polygonal block, and two non-adjacent side surfaces of the special-shaped calibration block are symmetrically provided with rectangular blocks.

In some embodiments, determining whether the PIN on the circuit board to be detected is missing based on the first depth map and a second depth map of the PIN on the reference circuit board includes:

acquiring the outline of the PIN on the circuit board to be detected from the first depth map;

comparing the outline of each PIN marked in the second depth map with the outline of each PIN on the circuit board to be detected;

and if the outline which is successfully compared on the circuit board to be detected does not exist, determining that the PIN needle at the corresponding position on the circuit board to be detected is absent.

In some embodiments, obtaining the profile of the PIN on the circuit board to be detected from the first depth map includes:

extracting the contour of the first depth map to obtain a plurality of stitch contours;

grouping the PIN profiles according to the rule that the two PIN profiles belonging to the same PIN are nearest to each other;

and determining the contour of each group of PINs as the contour of one PIN PIN on the circuit board to be detected.

In some embodiments, further comprising:

detecting whether the un-missing PIN needles are skewed and/or abnormal in height based on point cloud data corresponding to the un-missing PIN needles on the circuit board to be detected; and

the defect detection result also comprises a skew detection result and/or a height abnormity detection result of each PIN which is not missed on the circuit board to be detected.

In some embodiments, detecting whether the PIN which is not missed is skewed based on the point cloud data corresponding to each PIN which is not missed on the circuit board to be detected includes:

determining three-dimensional coordinates of two stitches in each non-missing PIN needle based on the point cloud data in the two stitch profiles of each non-missing PIN needle and the three-dimensional scaling of the first depth map;

determining the space between the two stitches of the non-missing PIN needle based on the three-dimensional coordinates of the two stitches;

and if the spacing is not within the range of the spacing of the marks at the corresponding position on the reference circuit board, determining that the missing PIN is skewed.

In some embodiments, detecting whether the PIN PINs which are not missing are highly abnormal based on the point cloud data corresponding to each PIN which is not missing on the circuit board to be detected includes:

determining the height of each stitch in the non-missing PIN needle based on the three-dimensional coordinates of the stitch;

and if the height difference between the height of any PIN in the PIN without missing and the marking height at the corresponding position on the reference circuit board exceeds a set height range, determining that the height of the PIN without missing is abnormal.

In a second aspect, an embodiment of the present application provides a defect detection apparatus for a PIN, including:

the acquisition module is used for acquiring point cloud data of the circuit board to be detected, which are acquired by N line laser cameras, wherein N is determined based on the size of the circuit board to be detected and the acquisition range of a single line laser camera;

the generating module is used for generating a first depth map of the PIN needle on the circuit board to be detected based on the point cloud data acquired by each line of laser cameras;

the detection module is used for judging whether the PIN on the circuit board to be detected is missing or not and judging whether the PIN on the reference circuit board is not defective or not based on the first depth map and a second depth map of the PIN on the reference circuit board;

and the output module is used for outputting a defect detection result, wherein the defect detection result at least comprises the position indication information of the missing PIN on the circuit board to be detected.

generating the first depth map based on the complete point cloud data.

In some embodiments, the transformation relationship between the coordinate systems of each slave camera and the master camera is obtained by calibrating a special-shaped calibration block, the top surface of the special-shaped calibration block is provided with at least one polygonal block, and two non-adjacent side surfaces of the special-shaped calibration block are symmetrically provided with rectangular blocks.

comparing the outline of each PIN marked in the second depth map with the outline of the PIN on the circuit board to be detected;

In some embodiments, further comprising:

detecting whether the un-missing PIN is skewed and/or abnormal in height based on point cloud data corresponding to each un-missing PIN on the circuit board to be detected; and

In some embodiments, detecting whether the PIN needles not missing are skewed based on the point cloud data corresponding to each PIN needle not missing on the circuit board to be detected includes:

In some embodiments, detecting whether the un-missing PIN is highly abnormal based on the point cloud data corresponding to each un-missing PIN on the circuit board to be detected includes:

and if the height difference between the height of any PIN in the PIN which is not missed and the marking height at the corresponding position on the reference circuit board exceeds a set height range, determining that the height of the PIN which is not missed is abnormal.

In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor, and a memory communicatively coupled to the at least one processor, wherein:

the memory stores a computer program executable by at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the above-described PIN defect detection method.

In a fourth aspect, embodiments of the present application provide a storage medium, where when a computer program in the storage medium is executed by a processor of an electronic device, the electronic device is capable of executing the above-mentioned defect detection method for PIN.

In the embodiment of the application, point cloud data of a circuit board to be detected collected by N line laser cameras are obtained, point cloud data collected by each line laser camera are used for generating a first depth map of a PIN needle on the circuit board to be detected, a second depth map of the PIN needle on the circuit board to be detected is used for judging whether the PIN needle on the circuit board to be detected is lost or not, and then a defect detection result at least comprising position indication information of the PIN needle lost on the circuit board to be detected is output, wherein N is determined based on the size of the circuit board to be detected and the collection range of a single line laser camera, and the PIN needle on the reference circuit board does not have defects, so that a scheme for detecting the defects of the PIN needle by means of the line laser cameras is provided.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic structural diagram of a PIN provided in an embodiment of the present application;

FIG. 2 is a schematic view of a detection system provided in an embodiment of the present application;

fig. 3 is a three-view diagram of a special-shaped calibration block provided in an embodiment of the present application;

fig. 4 is a depth map of PIN PINs on a circuit board according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram of marking missing PIN needles and non-missing PIN needles in a depth map according to an embodiment of the present application;

FIG. 6 is a schematic diagram of markings on skewed PIN PINs and untwisted PIN PINs in a depth map according to an embodiment of the present application;

fig. 7 is a flowchart of a defect detection method for a PIN according to an embodiment of the present disclosure;

fig. 8 is a flowchart of generating a first depth map of PIN PINs on a circuit board to be detected according to an embodiment of the present disclosure;

fig. 9 is a flowchart for determining whether a PIN on a circuit board to be detected is missing according to an embodiment of the present application;

fig. 10 is a flowchart for detecting whether an undeleted PIN is skewed according to an embodiment of the present application;

fig. 11 is a flowchart for detecting whether a PIN that is not missing is highly abnormal according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a defect detection apparatus for a PIN according to an embodiment of the present application;

fig. 13 is a schematic hardware structure diagram of an electronic device for implementing a defect detection method for a PIN according to an embodiment of the present application.

Detailed Description

In order to provide a reasonable defect detection scheme for a PIN, the embodiment of the application provides a defect detection method and device for the PIN, an electronic device and a storage medium.

The preferred embodiments of the present application will be described below with reference to the accompanying drawings of the specification, it should be understood that the preferred embodiments described herein are merely for illustrating and explaining the present application, and are not intended to limit the present application, and that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

To facilitate understanding of the present application, the present application refers to technical terms in which:

the line laser camera generally refers to 3D line laser high accuracy camera, and the principle of line laser camera is laser triangle range finding method, specifically is: a line of laser irradiates a measured object at a certain incidence angle, the laser is reflected and refracted on the measured object, the reflected laser is converged and imaged by a lens at another angle, and a light spot is imaged on a Charge-coupled Device (CCD) position sensor. When the measured object moves along the laser direction, the light spot on the position sensor moves, and the displacement corresponds to the moving distance of the measured object, so that the moving distance of the measured object is calculated.

Point cloud data, typically includes a plurality of laser points, each containing rich measurement information such as coordinates, brightness, reflection intensity, etc.

The embodiments of the present application will be described with reference to specific embodiments.

I, PIN needle.

Generally, the circuit board may have PIN PINs on one side or PIN PINs on both sides. The circuit board has a width of 10-30 cm, a length of 20-50 cm, and a PIN height of 1-5 cm. The PIN generally includes two PINs, fig. 1 is a schematic structural diagram of a PIN provided in an embodiment of the present application, and B indicates a distance between two PINs of the PIN.

The defect types of PIN PINs generally include the following three categories:

1. missing a PIN needle: the quantity and the position of PIN needle on the circuit board of fixed specification are fixed, and PIN needle disappearance will appear in PIN needle neglected loading to communication fault when leading to the use.

2. The PIN is crooked: PIN PINs on the circuit board are all vertically mounted, and one PIN PIN comprises two PINs, and the skew of any PIN can cause the failure of connecting the electronic components.

3. PIN height anomaly: the PIN height at the fixed position on the fixed-size circuit board is fixed, and if the PIN height is too high or too low (i.e. the height difference with the mark height at the fixed position exceeds the set height range), the connection of the electronic component is failed.

And II, detecting the system.

Because the width of the circuit board can reach 30cm, and the measuring range of a single line laser camera is generally 20cm, the circuit board can be collected by two line laser cameras to realize the detection of the PIN needle on the widest circuit board, and the common viewing area of the adjacent line laser cameras can be 3-4 cm, so that the installation difficulty of the line laser cameras is reduced. In addition, blue line laser can be adopted for data acquisition, and the top of the PIN needle with a small area can be shot because the blue line laser has strong capability of resisting ambient light interference.

Taking the example of detecting whether the PIN on one side of the circuit board is defective, fig. 2 is a schematic diagram of a detection system provided by the embodiment of the present application, and includes a motion platform 1, two line laser cameras 2, a circuit board 3 and a mounting bracket 4. The installation method is as follows: two line laser camera 2 fix in the motion platform 1 directly over side by side, and two line laser camera 2 have 3 ~ 4 cm's overlapping field of vision.

Assuming that the two line laser cameras are L1 and L2, when the line laser cameras L1 and L2 are installed in parallel, the laser line generated when the line laser camera L1 scans will enter the field of view of the line laser camera L2, thereby generating overlapping point clouds. For this reason, a trigger delay may be set, i.e., the line laser camera L2 does not trigger shooting when the line laser camera L1 emits blue laser lines. The setting method comprises the following steps: the trigger delay of the line laser camera L1 is 0, and the trigger delay of the line laser camera L2 is greater than the exposure value. It should be noted that if more line laser cameras are needed for splicing, the trigger delay setting methods are consistent.

After the motion platform is started, the motion platform can drive the circuit board to move, after the line laser cameras L1 and L2 are started, the line laser cameras L1 and L2 respectively detect the circuit board in a visual field to obtain multi-frame point cloud data, after the detection is finished, the line laser cameras L1 and L2 respectively fuse the multi-frame point cloud data acquired by the line laser cameras to obtain the point cloud data of the circuit board acquired by the line laser cameras, and then the line laser camera L1 (assumed as a master camera) can fuse the point cloud data acquired by the line laser cameras and the line laser camera L2 (assumed as a slave camera), so that complete point cloud data of the circuit board are obtained.

Therefore, the defects of the PIN needle are measured on the motion platform, the automatic assembly of a factory is facilitated, and the PIN needle is conveniently linked with the mechanical arm.

And thirdly, calibrating the line laser camera.

When the line laser cameras L1 and L2 are calibrated, the special-shaped calibration block can be placed in the common-view area of the line laser cameras L1 and L2 on the motion platform, even if the line laser cameras L1 and L2 can acquire the special-shaped calibration block. After the point cloud data acquired by the line laser cameras L1 and L2 are obtained, a rotation matrix (R) and a translation matrix (T) between the point cloud data acquired by the line laser camera L2 and the point cloud data acquired by the line laser camera L1 can be calculated by using a Fast Robust-ICP algorithm, and R and T are calibration results of the line laser cameras L1 and L2. In subsequent measurement, point cloud data acquired by the line laser camera L2 can be converted into a coordinate system of the line laser camera L1 through rigid transformation (R, T), and then the point cloud data of the line laser camera L1 and the point cloud data of the line laser camera L2 are spliced to obtain complete point cloud data of the circuit board.

Fig. 3 is a three-dimensional view of a special-shaped calibration block provided in an embodiment of the present application, wherein three polygonal blocks (i.e., a triangular block, a circular block, and a square block) are disposed at the top of the special-shaped calibration block in a dispersed manner, and rectangular blocks are symmetrically disposed on two opposite sides of the special-shaped calibration block.

In the detection system shown in fig. 2, the common view area of the two line laser cameras installed in parallel is the top surface of the special-shaped calibration block, so that the two line laser cameras can shoot images with abundant geometric features together, point cloud matching between the two line laser cameras can be completed more accurately, the conversion relation between the two line laser cameras is calibrated more accurately, and namely R and T between the two line laser cameras are determined more accurately.

In addition, it should be noted that, in some scenes, for example, when detecting whether PIN PINs on two sides of a circuit board have defects, two line laser cameras are installed oppositely (that is, the two line laser cameras are in opposite emission), at this time, the two line laser cameras can both collect the circuit board bottom board but have no overlapped view, and in this case, the protruding rectangular blocks on two sides of the special-shaped calibration block can be used to complete calibration of the two line laser cameras. That is, the irregular calibration block shown in fig. 3 can complete the calibration work of the dual cameras in various installation forms.

Fourthly, marking a PIN needle.

Due to the fact that the heights and the positions of PIN PINs on circuit boards of different batches are not consistent, in order to complete automatic measurement, one or more reference circuit boards can be selected from the circuit boards of the batch, and the selection criteria of the reference circuit boards are as follows: the height of the PIN PINs on the circuit board is normal, the positions of the PIN PINs are correct, and the PIN PINs are not inclined. Then, point cloud data of the top ends of the PIN PINs on the reference circuit board can be obtained through scanning, a depth map of the PIN PINs on the reference circuit board is generated based on the point cloud data, the positions of all the PIN PINs are located on the depth map, two PINs of each PIN PIN are numbered, and the height of each PIN and the distance between the PINs are recorded.

And fifthly, detecting defects.

1. And (5) filtering the point cloud.

Due to the fact that the placing heights of the PIN PINs on the circuit boards with different specifications are different, the point cloud data of the tops of the PIN PINs on the circuit boards cannot be directly filtered at high heights, therefore, the plane of the circuit board can be located on the basis of the complete point cloud data, the plane of the circuit board is used as the reference height, and the point cloud data above the plane is the point cloud data of the PIN PINs on the circuit boards.

2. PIN missing detection.

The first step is as follows: and projecting the point cloud data of the PIN into a depth map along the direction of a normal vector of the plane of the circuit board.

Generally, the point cloud data of the PIN includes a plurality of laser points, each laser point has three-dimensional coordinates (x, y, z), first, the maximum value and the minimum value of the x, y, z coordinate values in the point cloud data relative to the circuit board plane can be calculated, the difference value of the maximum value and the minimum value of the x coordinate is divided by a preset x-direction scaling factor to be used as the width of the depth map, the difference value of the maximum value and the minimum value of the y coordinate is divided by a preset y-direction scaling factor to be used as the height of the depth map, and the difference value of the maximum value and the minimum value of the z coordinate is divided by 255 to be used as the height represented by the unit gray value of a single pixel point on the depth map. Then, each laser point in the PIN point cloud data may be traversed, and the laser point is projected into the depth map according to a scaling ratio, so as to obtain the depth map shown in fig. 4, each white origin represents one PIN of the PIN, and the brighter the white origin, the higher the height of the corresponding PIN is.

The second step is that: and positioning the top of the PIN needle.

The point cloud data at the top of the PIN needle is high in height, the gray value on the depth map is large, and the point cloud data is represented as a bright spot (a white origin in fig. 4), namely the bright spot is the position of the top of the PIN needle and is close to a circle, so that the bright spot can be positioned in the depth map by using an LOG algorithm, and the pixel position of the top of each PIN needle is obtained. Then, the located PIN positions may be matched with the PIN positions marked in the depth map of the reference circuit board to determine whether the PIN at the corresponding position on the current circuit board is missing, and the positions of the missing PIN and the non-missing PIN may also be marked on the depth map of the current circuit board, as shown in fig. 5, the bright gray frame marks the position of the non-missing PIN, and the dark gray frame marks the position of the missing PIN.

Therefore, the whole circuit board plane is used as a reference plane, so that the measured value can be more accurate, only the point cloud data of the PIN needle is projected to be a depth map, the time consumption of the algorithm can be reduced, and the detection speed is improved.

3. And detecting the height of the PIN needle.

According to the two bright spots positioned at the top end of each PIN needle on the depth map of the current circuit board, the center coordinate and the average gray value of each bright spot can be obtained, the three-dimensional coordinate corresponding to the top end of the corresponding PIN can be calculated based on the center coordinate, the average gray value, the scaling factor in the x and y directions and the maximum and minimum value of the point cloud z values, the distance from the three-dimensional coordinate to the plane of the circuit board is calculated, and the height of the corresponding PIN can be obtained. And comparing the height of each PIN with the height marked at the corresponding position on the reference circuit board to judge whether the height of the PIN PIN is abnormal or not, and if so, marking an abnormal position on the depth map.

4. And detecting the distance between the PIN needles.

A PIN has two PINs, and the tips of the two PINs appear as two bright spots on the depth map. The distance between two PINs of one PIN needle on the circuit board is smaller than the distance between any two PIN needles, so that the bright spot closest to one bright spot is two PINs of the same PIN needle. The bright spot matching can be carried out according to the circle center position of the bright spot, namely the PIN matching of the PIN needle.

Specifically, all the bright spots are traversed, all the bright spots closest to the bright spot are matched, and a unidirectional bright spot pair is formed. And then, performing secondary matching on the unidirectional bright spot pairs, namely judging whether the matched points are all closest in distance, completing matching if the matched bright spot pairs are all closest in distance, and otherwise, recording the bright spots failed in matching. Therefore, the nearest bright spots are searched for the first time by using two times of distance nearest matching, whether the bright spot pairs are nearest to each other is checked for the second time, the mismatching rate can be effectively reduced by two times of bright spot matching, and the position positioning accuracy of the PIN needle is improved.

For the successfully matched bright spot pairs, the three-dimensional coordinates of the circle center positions of the bright spots can be obtained through calculation according to the circle center coordinates, the gray values and the scaling factors of the bright spots on the depth map, the distance between the three-dimensional coordinates of the circle center positions of the two bright spots in the bright spot pairs is the distance between the two PINs of the PIN, the distance is compared with the distance marked at the corresponding position on the reference circuit board, whether the PIN is skewed or not can be judged, if the PIN is skewed, the skewed position is marked on the depth map, as shown in fig. 6, the dark gray marks are the PIN with the skewed position, the other color marks are the PIN without the skewed position, the number on the left side of each PIN in the map represents the number of the PIN, and the number on the right side of each PIN represents the distance between the two PINs in the PIN.

And sixthly, outputting the detection result.

And for the PIN on one circuit board, if the defects such as PIN missing, PIN deflection, PIN height abnormity and the like do not exist, displaying OK, otherwise, displaying NG, outputting a depth map, and marking the defect position and defect type in the depth map. Table 1 is a tabular form of the measurements, as follows:

TABLE 1 measurement results

The scheme provided by the embodiment of the application has the following advantages:

1. the measurement precision is high.

In the proposal, the used 3D line laser can scan the surface of an object to obtain three-dimensional point cloud data, a plane obtained by fitting the circuit board is used as a reference plane, the point cloud data of the PIN needle is projected to the plane where the circuit board is located, and the projection precision can reach 0.01 mm. And positioning the top of the PIN needle on the depth map, mapping the top of the PIN needle into a three-dimensional coordinate, calculating the distance from a three-dimensional point corresponding to the top of the PIN needle to the circuit board to obtain the height of the PIN needle at the corresponding position, wherein the measurement accuracy can reach 0.02 mm.

2. The universality is high.

In this proposal, if the size of the circuit board is larger, more 3D line lasers can be erected, and detection is completed by adopting a similar method. If the heights or the positions of the PIN PINs on the circuit board are inconsistent, the reference circuit board can be replaced at the stage of marking the PIN PINs so as to change the heights and the positions of the PIN PINs for comparison, so that defect detection of the PIN PINs on the circuit board with any specification is realized.

3. And the expansibility is strong.

The proposal comprises a camera calibration stage, a PIN defect detection stage and a result output stage, and each stage can be developed in a modularization mode, so that similar modules can be borrowed when similar detection is met, the development period is saved, and the project portability is good.

The following describes the defect detection of the PIN proposed in the present application with reference to a flowchart.

Fig. 7 is a flowchart of a method for detecting defects of a PIN according to an embodiment of the present application, where the method is applicable to one line laser camera and also applicable to a device independent of the line laser camera, and the method includes the following steps.

In step 701, point cloud data of the circuit board to be detected, which is acquired by N line laser cameras, is acquired, wherein N is determined based on the size of the circuit board to be detected, the PIN setting mode and the acquisition range of a single line laser camera.

Wherein, PIN needle setting mode includes that the single face sets up and two-sided setting.

When a PIN needle is arranged on a single side of a circuit board to be detected, if the size of the circuit board to be detected exceeds the acquisition range of a single line laser camera, at least two line laser cameras can be arranged side by side to acquire data of the circuit board to be detected, the installation mode of the line laser cameras can be shown in figure 2, at the moment, a common viewing area exists between adjacent line laser cameras, data acquisition time delay exists between the adjacent line laser cameras, and the data acquisition time delay can meet the condition that the data acquisition is not repeatedly performed on the common viewing area by the adjacent line laser cameras. Therefore, the installation requirements of the line laser cameras can be reduced, and the adjacent line laser cameras can be ensured not to repeatedly acquire data of the common visual area.

When waiting to detect circuit board two-sided setting PIN needle, two groups line laser camera subtend sets up, a group line laser camera is used for scanning a face PIN needle, do not have the region of looking altogether between two groups line laser camera, but to arbitrary group line laser camera, if the size of the circuit board of waiting to detect exceeds the collection scope of single line laser camera, then this group line laser camera can include two at least line laser camera, it is similar with the single face setting PIN needle, at this moment, adjacent line laser camera that sets up side by side can have visual area and data acquisition time delay altogether, data acquisition time delay can satisfy this and does not carry out data acquisition to the visual area of altogether repeatedly to adjacent line laser camera.

In step 702, a first depth map of the PIN on the circuit board to be detected is generated based on the point cloud data acquired by each line of laser cameras.

Taking the example that the N line laser cameras include one master camera and N-1 slave cameras, a first depth map of the PIN on the circuit board to be detected may be generated according to the process shown in fig. 8, where the process includes the following steps.

In step 7021, the point cloud data collected from the cameras is converted into the coordinate system of the master camera based on the conversion relationship between the coordinate systems of each slave camera to the master camera.

The conversion relation between the coordinate systems of each slave camera and the master camera can be represented as R and T between point cloud data collected by the slave camera and point cloud data collected by the master camera, the conversion relation between the coordinate systems of each slave camera and the master camera is obtained by calibrating a special-shaped calibration block, the top surface of the special-shaped calibration block is provided with at least one polygonal block, two non-adjacent side surfaces of the special-shaped calibration block are symmetrically provided with rectangular blocks, and the shape of the special-shaped calibration block can be seen in fig. 3.

In step 7022, the point cloud data converted by each slave camera and the point cloud data of the master camera are fused to obtain complete point cloud data of the circuit board to be detected.

And performing fusion processing, such as splicing the point cloud data according to the coordinates of the laser points in the point cloud data.

In step 7023, a first depth map is generated based on the complete point cloud data.

For example, target point cloud data of a PIN on a circuit board to be detected is selected from the complete point cloud data, and then projection processing is performed on the target point cloud data to obtain a first depth map. Therefore, the time consumption of the algorithm can be reduced, and the detection speed is improved.

In step 703, it is determined whether the PIN on the circuit board to be detected is missing based on the first depth map and the second depth map of the PIN on the reference circuit board.

The reference circuit board and the circuit board to be detected belong to the same batch of circuit boards, and the PIN needles on the reference circuit board have no defects.

For example, whether the PIN on the circuit board to be detected is missing is determined according to the flow shown in fig. 9, which includes the following steps.

In step 7031, the profile of the PIN on the circuit board to be tested is obtained from the first depth map.

During specific implementation, the first depth map can be subjected to contour extraction to obtain a plurality of PIN contours, then, according to the rule that two PIN contours belonging to the same PIN needle are nearest to each other, the PIN contours are grouped, and then each group of PIN contours is determined as the contour of one PIN needle on the circuit board to be detected.

In step 7032, the outline of each PIN marked in the second depth map is compared with the outline of the PIN on the circuit board to be detected.

In step 7033, it is determined whether there is a successfully compared outline on the circuit board to be detected, if yes, step 7034 is performed, and if not, step 7035 is performed.

In step 7034, it is determined that the PIN at the corresponding location on the circuit board to be tested is not missing.

In step 7035, it is determined that PIN PINs at corresponding locations on the circuit board to be tested are missing.

In step 704, whether the PIN without missing PIN is skewed and/or abnormal in height is detected based on the point cloud data corresponding to each PIN without missing PIN on the circuit board to be detected.

In some embodiments, whether the non-missing PIN is skewed may be detected according to the flow shown in fig. 10, which includes the following steps.

In step 7041a, three-dimensional coordinates of two stitches in the non-missing PIN needle are determined based on the point cloud data in the two stitch profiles of each non-missing PIN needle and the three-dimensional scaling of the first depth map.

For example, for each PIN profile i (represented as a bright spot) of the PIN needle which is not missing, an average value of three-dimensional coordinates (xi, yi, zi) of each laser point included in the point cloud data in the PIN profile i is calculated

Will be provided with

As the three-dimensional coordinates of the corresponding stitch, kx, ky, and kz are the scaling ratios of the first depth map on the x-axis, the y-axis, and the z-axis, respectively, and i is 1 and 2.

In step 7042a, the spacing between the two PINs of the non-missing PIN is determined based on the three-dimensional coordinates of the two PINs.

For example, the spacing d between two PINs of a PIN that is not missing is determined according to the following formula:

wherein, the first and the second end of the pipe are connected with each other,

is the three-dimensional coordinates of the first PIN of the PIN that is not missing,

is the three-dimensional coordinate of the second PIN of the PIN that is not missing.

In step 7043a, it is determined whether the distance is within the mark distance range at the corresponding position on the reference circuit board, if so, the process proceeds to step 7044a, and if not, the process proceeds to step 7045 a.

In step 7044a, it is determined that the non-missing PIN is not skewed.

In step 7045a, it is determined that the missing PIN is skewed.

In some embodiments, whether the non-missing PIN is highly abnormal may be detected according to the flow shown in fig. 11, which includes the following steps.

In step 7041b, three-dimensional coordinates of two stitches in the non-missing PIN needle are determined based on the point cloud data in the two stitch profiles of each non-missing PIN needle and the three-dimensional scaling of the first depth map.

The step can be performed in step 7041a, and is not described in detail here.

In step 7042b, the height of each PIN in the non-missing PIN is determined based on the three-dimensional coordinates of that PIN.

Specifically, the distance from the three-dimensional coordinate of each PIN in the PIN which is not lost to the plane of the circuit board to be detected is calculated, and then the height of the PIN can be obtained.

In step 7043b, it is determined whether the height difference between the height of each PIN in the PIN that is not missing and the height of the mark at the corresponding position on the reference circuit board does not exceed the set height range, if so, the process proceeds to 7044b, and if so, the process proceeds to 7045 b.

In step 7044b, it is determined that there is no anomaly in the height of the non-missing PIN.

In step 7045b, a high anomaly of non-missing PIN needles is determined.

In step 705, a defect detection result is output, where the defect detection result includes position indication information of missing PIN on the circuit board to be detected, and a skew detection result and/or a height anomaly detection result of each PIN that is not missing.

In the embodiment of the application, the point cloud data of the circuit board to be detected with any specification can be acquired by using the line laser cameras, whether the PIN on the circuit board to be detected is missing or not can be judged by means of the point cloud data, whether the PIN on the circuit board to be detected is not missing or not is inclined and/or abnormal in height can also be judged, and the defect detection type is comprehensive.

Based on the same technical concept, the embodiment of the application further provides a defect detection device for the PIN, and the principle of solving the problem of the defect detection device for the PIN is similar to that of the defect detection method for the PIN, so that the implementation of the defect detection device for the PIN can be referred to the implementation of the defect detection method for the PIN, and repeated parts are not repeated.

Fig. 12 is a schematic structural diagram of a defect detection apparatus for a PIN according to an embodiment of the present application, and includes an obtaining module 1201, a generating module 1202, a detecting module 1203, and an outputting module 1204.

The acquisition module 1201 is used for acquiring point cloud data of the circuit board to be detected, acquired by N line laser cameras, wherein N is determined based on the size of the circuit board to be detected and the acquisition range of a single line laser camera;

the generating module 1202 is configured to generate a first depth map of the PIN on the circuit board to be detected based on the point cloud data acquired by each line of laser cameras;

a detecting module 1203, configured to determine whether the PIN on the circuit board to be detected is missing or not and whether the PIN on the reference circuit board is not defective or not based on the first depth map and a second depth map of the PIN on the reference circuit board;

an output module 1204, configured to output a defect detection result, where the defect detection result at least includes position indication information of a missing PIN on the circuit board to be detected.

In some embodiments, the N line laser cameras include one master camera and N-1 slave cameras, and the generation module 1202 is specifically configured to:

generating the first depth map based on the complete point cloud data.

In some embodiments, the generating module 1202 is specifically configured to:

In some embodiments, when at least two line laser cameras are arranged side by side, a common view area and a data acquisition delay exist between adjacent line laser cameras arranged side by side, and the data acquisition delay satisfies that the adjacent cameras do not repeatedly acquire data of the common view area.

In some embodiments, the detection module 1203 is specifically configured to:

In some embodiments, the detection module 1203 is further configured to:

In some embodiments, the detecting module 1203 is specifically configured to detect whether the PIN that is not missing is skewed according to the following steps:

In some embodiments, the detecting module 1203 is specifically configured to detect whether the PIN is not missing and whether the PIN is not missing or is not missing according to the following steps:

determining three-dimensional coordinates of two stitches of each un-missing PIN needle based on the point cloud data in the two stitch profiles of each un-missing PIN needle and the three-dimensional scaling of the first depth map;

The division of the modules in the embodiments of the present application is schematic, and only one logic function division is provided, and in actual implementation, there may be another division manner, and in addition, each function module in each embodiment of the present application may be integrated in one processor, may also exist alone physically, or may also be integrated in one module by two or more modules. The coupling of the various modules to each other may be through interfaces that are typically electrical communication interfaces, but mechanical or other forms of interfaces are not excluded. Thus, modules described as separate components may or may not be physically separate, may be located in one place, or may be distributed in different locations on the same or different devices. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.

Having described the defect detection method and apparatus of the PIN needles according to the exemplary embodiments of the present application, an electronic device according to another exemplary embodiment of the present application will be described next.

An electronic device 130 implemented according to this embodiment of the present application is described below with reference to fig. 13. The electronic device 130 shown in fig. 13 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 13, the electronic device 130 is represented in the form of a general electronic device. The components of the electronic device 130 may include, but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 that connects the various system components (including the memory 132 and the processor 131).

Bus 133 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

The memory 132 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)1321 and/or cache memory 1322, and may further include Read Only Memory (ROM) 1323.

Memory 132 may also include a program/utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may comprise an implementation of a network environment.

The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), with one or more devices that enable a user to interact with the electronic device 130, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 130 to communicate with one or more other electronic devices. Such communication may occur via input/output (I/O) interfaces 135. Also, the electronic device 130 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 136. As shown, network adapter 136 communicates with other modules for electronic device 130 over bus 133. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

In an exemplary embodiment, there is also provided a storage medium, and when a computer program in the storage medium is executed by a processor of an electronic device, the electronic device is capable of executing the above-described defect detection method for PIN needles. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, the electronic device of the present application may include at least one processor, and a memory communicatively connected to the at least one processor, where the memory stores a computer program executable by the at least one processor, and the computer program, when executed by the at least one processor, may cause the at least one processor to perform the steps of any one of the PIN defect detection methods provided in the embodiments of the present application.

In an exemplary embodiment, a computer program product is also provided, which, when executed by an electronic device, enables the electronic device to implement any of the exemplary methods provided herein.

Also, a computer program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable Disk, a hard Disk, a RAM, a ROM, an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a Compact Disk Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The program product for defect detection of the PIN in the embodiment of the present application may employ a CD-ROM and include program code, and may be run on a computing device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, Radio Frequency (RF), etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device over any kind of Network, such as a Local Area Network (LAN) or Wide Area Network (WAN), or may be connected to external computing devices (e.g., over the internet using an internet service provider).

It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, according to embodiments of the application. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.

Further, while the operations of the methods of the present application are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A defect detection method of a PIN needle is characterized by comprising the following steps:

judging whether the PIN on the circuit board to be detected is missing or not and judging whether the PIN on the reference circuit board is not defective or not based on the first depth map and a second depth map of the PIN on the reference circuit board;

2. The method of claim 1, wherein the N line laser cameras include a master camera and N-1 slave cameras, and generating a first depth map of the PIN on the circuit board to be detected based on the point cloud data acquired by each line laser camera comprises:

and generating the first depth map based on the complete point cloud data.

3. The method of claim 2, wherein generating the first depth map based on the complete point cloud data comprises:

4. The method according to claim 2, wherein when at least two line laser cameras are arranged side by side, a common view area and a data acquisition delay exist between adjacent line laser cameras arranged side by side, and the data acquisition delay satisfies that the adjacent cameras do not repeatedly perform data acquisition on the common view area.

5. The method of claim 2, wherein the transformation relationship between the coordinate systems of each slave camera and the master camera is obtained by calibrating a special-shaped calibration block, the top surface of the special-shaped calibration block is provided with at least one polygonal block, and two non-adjacent sides of the special-shaped calibration block are symmetrically provided with rectangular blocks.

6. The method of claim 1, wherein determining whether the PIN on the circuit board to be detected is missing based on the first depth map and a second depth map of the PIN on a reference circuit board comprises:

7. The method of claim 6, wherein obtaining the profile of the PIN on the circuit board to be tested from the first depth map comprises:

8. The method of any of claims 1 to 7, further comprising:

9. The method of claim 8, wherein detecting whether the un-missing PIN is skewed based on the point cloud data corresponding to each un-missing PIN on the circuit board to be detected comprises:

10. The method of claim 8, wherein detecting whether the un-missing PIN is highly abnormal based on the point cloud data corresponding to each un-missing PIN on the circuit board to be detected comprises:

11. A defect detection device of PIN needle, characterized by, includes:

12. An electronic device, comprising: at least one processor, and a memory communicatively coupled to the at least one processor, wherein:

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

13. A storage medium, characterized in that, when the computer program in the storage medium is executed by a processor of an electronic device, the electronic device is capable of performing the method according to any one of claims 1-10.