CN114494460A - Calibration method, multi-camera probe device and flying probe tester - Google Patents

Abstract

The invention discloses a calibration method, a multi-camera probe device and a flying probe tester. According to the calibration method, the multi-camera probe device and the flying probe testing machine, the calibration process of the camera system is simple, the calibration precision is high, and after the calibration of the camera system is completed, one-key calibration of the probe can be realized without manually teaching an actual position for many times; after the probe is replaced, the position of the probe does not need to be manually taught again, and the actual position of the new probe in the system can be directly determined on the premise of previous calibration through visual shooting. The calibration method has the advantages that the calibration precision of the camera can reach below 0.7 mu m, and the calibration precision of the probe can reach below 3.5 mu m.

Description

Calibration method, multi-camera probe device and flying probe tester

technical field

The invention relates to the technical field of machine vision, in particular to a calibration method, a multi-camera probe device and a flying probe tester.

Background technique

Camera calibration refers to the process of using the camera system to capture the image of the calibration plate, and using the three-dimensional coordinates of the known feature points in the calibration plate and the corresponding image coordinates on the image to calculate the camera parameters. In the field of machine vision and image measurement, the main equipment for completing measurement and positioning tasks is the camera. In order to determine the relationship between the three-dimensional geometric position of a point on the surface of a space object and its corresponding point in the image, a geometric model of camera imaging must be established. and accurately calibrate its parameters. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very critical link. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera work.

At present, for the calibration of the external components of the camera, such as probes, cutting heads, laser inkjet heads, etc., it is generally used to manually teach the actual position multiple times, and then use the camera to take pictures for calibration. The calibration is time-consuming and labor-intensive, and the accuracy is very low. , which cannot meet the high-precision positioning application scenarios.

Therefore, in view of the above technical problems, it is necessary to provide a new calibration method.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a calibration method, a multi-camera probe device and a flying probe tester, the calibration method has a simple calibration implementation process and high calibration accuracy.

For achieving the above object, the technical scheme provided by the invention is as follows:

In a first aspect, the present invention provides a multi-camera calibration method, which includes the following steps:

establishing the image coordinate system XYZ _p1 and the mechanical coordinate system XYZ _m1 of the first camera, the image coordinate system XYZ _p2 and the mechanical coordinate system XYZ _m2 of the second camera, and the image coordinate system XYZ _p3 of the third camera;

The physical coordinate system XYZ _mb of the calibration board is established with the preset point on the calibration board as the origin, and the physical coordinates [x _mb , y _mb ] ^T of the calibration point are calculated according to the actual physical distance between the preset point and the calibration point on the calibration board;

The images of the calibration plate are captured by the first camera, the second camera and the third camera, and the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y of the calibration point in the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 are determined _p2 ] ^T and [x _p3 , y _p3 ] ^T ;

According to the physical coordinates [x _mb , y _mb ] ^T of the calibration point and the image coordinates of the calibration point [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T , [x _p3 , y _p3 ] ^T , through the homogeneous Transform to obtain the transformation matrix M _mb→p1 , M _mb→p2 and M _mb→p3 from the coordinate system XYZ _mb to the coordinate system XYZ _p1 , XYZ _p2 and XYZ _p3 ;

According to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, obtain the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 , and obtain the coordinate system XYZ _p1 through affine transformation to the transformation matrix M _p1→m1 of the coordinate system XYZ _m1 , and the transformation matrix M _p2→m2 of the coordinate system XYZ _p2 to the coordinate system XYZ _m2 ;

According to the transformation matrices M _mb→p1 , M _mb→p2 , M _mb→p3 , M _p1→m1 and M _p2→m2 , the mechanical coordinate system XYZ _m1 of the first camera, the mechanical coordinate system XYZ _m2 of the second camera, and the third The transformation relationship between the image coordinate systems XYZ _p3 of the three cameras.

In one or more embodiments, the origin of the coordinate system XYZ _p1 is the image center of the image captured by the first camera, the origin of the coordinate system XYZ _p2 is the image center of the image captured by the second camera, and the coordinates The origin of the system XYZ _p3 is the image center of the image captured by the third camera.

In one or more embodiments, the origin of the coordinate system XYZ _m1 is the mechanical position where the first camera is located when the first camera shoots the calibration plate, and the origin of the coordinate system XYZ _m2 is when the second camera shoots the calibration board The mechanical position where the second camera is located.

In one or more embodiments, the first camera, the second camera and the third camera capture images of the calibration plate, and determine the image coordinates of the calibration point in the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 [x _p1 ,y _p1 ] ^T , [x _p2 ,y _p2 ] ^T and [x _p3 ,y _p3 ] ^T , including:

The calibration plate is photographed by the first camera, the second camera and the third camera, the first image, the second image and the third image of the calibration plate are obtained, and the pixels in the first image, the second image and the third image according to the calibration point Position, determine the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T and [x _p3 , y _p3 ] ^T of the calibration point in the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 .

In one or more embodiments, the homogeneous transformation formula is:

Among them, Z _c is the vertical height from the calibration plane of the calibration plate to the plane of the image coordinate system of the camera, (u, v) is the point in the image of the calibration plate obtained by the camera, dX×dY is the physical size of a single pixel, (u ₀ , v ₀ ) is the translation amount from the upper left corner of the calibration plate image obtained by the camera to the origin of the camera image coordinate system, f is the focal length of the camera lens, (X _w , Y _w , Z _w ) is the point coordinate in the physical space, M is the The transformation matrix between the physical coordinate system of the calibration board and the camera image coordinate system.

In one or more embodiments, the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 are obtained according to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point ,include:

According to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, calculate the pixel distance between the calibration point and the center of the image, and obtain the calibration point and the center of the first camera and the second camera through a single pixel According to the mechanical coordinates of the center of the first camera in the coordinate system XYZ _m1 and the mechanical coordinates of the center of the second camera in the coordinate system XYZ _m2 , the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 are obtained.

In one or more embodiments, the mechanical coordinate system XYZ _m1 of the first camera is obtained according to transformation matrices M _mb→p1 , M _mb→p2 , M _mb→p3 , M _p1→m1 and M _p2→m2 . The transformation relationship between the mechanical coordinate system XYZ _m2 of the second camera and the image coordinate system XYZ _p3 of the third camera includes:

Let a point in the coordinate system XYZ _mb be (x _mb1 , y _mb1 ), its coordinates in the coordinate system XYZ _m1 are (x _m11 , y _m11 ), and its coordinates in the coordinate system XYZ _m2 are (x _m21 , y _m21 ), its coordinates in the coordinate system XYZ _p3 are (x _p31 , y _p31 ), then there are

According to formula (1), the projection matrix of coordinate system XYZ _m2 to coordinate system XYZ _m1 is M _mb→p1 ·M _p1→m1 ·M _p2→m2 ^-1 ·M _mb→p2 ^-1 ; coordinate system XYZ _m1 to The projection matrix of the coordinate system XYZ _m2 is M _mb→p2 ·M _p2→m2 ·M _p1→m1 ^-1 ·M _mb→p1 ^-1 ; the projection matrix of the coordinate system XYZ _p3 to the coordinate system XYZ _m1 is M _mb→p1 · M _p1→m1 ·M _mb→p3 ^-1 ; the projection matrix of the coordinate system XYZ _p3 to the coordinate system XYZ _m2 is M _mb→p2 ·M _p2→m2 ·M _mb→p3 ^-1 .

In one or more embodiments, the preset point and the calibration point on the calibration plate are set to be visible on both sides, and the first camera and the second camera take pictures of the first surface of the calibration plate. the preset point and the calibration point, and the third camera shoots the preset point and the calibration point through the second surface of the calibration plate.

In one or more embodiments, the calibration plate is provided with a mark for determining the position of the preset point and for determining the direction of the physical coordinate system XYZ _mb of the calibration plate.

In a second aspect, the present invention provides a probe calibration method, which is applied to a flying probe testing machine. The flying probe testing machine includes a first testing axis, a second testing axis, and a third camera. The first testing axis is provided with a There is a first camera and a first probe, the second test shaft is provided with a second camera and a second probe, and the probe calibration method includes the following steps:

Based on the multi-camera calibration method described above, the calibration of the first camera, the second camera and the third camera is completed;

Move the first probe into the field of view of the third camera, take an image of the first probe through the third camera, and obtain the coordinates of the first probe tip in the coordinate system XYZ _p3 (x _{p1 needle} , y _{p1 needle} , 1), according to the conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m1 , obtain the coordinates of the first probe tip in the coordinate system XYZ _m1 (x _{m1 needle} , y _{m1 needle} , 1) ^T ;

According to the mechanical position of the first test axis (x _m1c , y _m1c ) when the first camera shoots the calibration board, and the mechanical position of the first test axis when the third camera shoots the first probe (x _{1 pin} , y _{1 pin} , z _{1 pin} ), obtain the actual offset (Δx ₁ , Δy ₁ ) from the tip of the first probe to the center of the first camera, and complete the calibration of the first camera by the first probe;

Move the second probe into the field of view of the third camera, take an image of the second probe through the third camera, and obtain the coordinates of the second probe tip in the coordinate system XYZ _p3 (x _{p2 needle} , y _{p2 needle} , 1), according to the conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m2 , obtain the coordinates of the second probe tip in the coordinate system XYZ _m2 (x _{m2 needle} , y _{m2 needle} , 1) ^T ;

According to the mechanical position of the second test axis (x _m2c , y _m2c ) when the second camera shoots the calibration board, and the mechanical position of the second test axis when the third camera shoots the second probe (x _{2 pins} , y _{2 needles} , z _{2 needles} ) to obtain the actual offset (Δx ₂ , Δy ₂ ) from the second probe tip to the center of the second camera, and complete the calibration of the second probe to the second camera.

In one or more embodiments, the coordinates (x _{m1 needle} , y _{m1 needle} , 1) ^T of the first probe tip in the coordinate system XYZ _m1 are:

(x _{m1 pin} , y _{m1 pin} , 1) ^T = M _{mb → p1} · M _{p1 → m1} · M _{mb → p3} ^-1 · (x _{p1 pin} , y _{p1 pin} , 1) ^T .

In one or more embodiments, the actual offset (Δx ₁ , Δy ₁ ) of the first probe tip to the center of the first camera is:

(Δx ₁ ,Δy ₁ )=(x _{1 pin-} x _m1c +x _{m1 pin} , y _{1 pin} -y _m1c +y _{m1 pin} ).

In one or more embodiments, the coordinate (x _{m2 needle} , y _{m2 needle} , 1) ^T of the second probe tip in the coordinate system XYZ _m2 is:

(x _{m2 pin} , y _{m2 pin} , 1) ^T = M _{mb → p2} · M _{p2 → m2} · M _{mb → p3} ^-1 · (x _{p2 pin} , y _{p2 pin} , 1) ^T .

In one or more embodiments, the actual offset (Δx ₂ , Δy ₂ ) of the second probe tip to the center of the second camera is:

(Δx ₂ , Δy ₂ ) = (x _{2 stitches} - x _m2c + x _{m2 stitches} , y _{2 stitches} - y _m2c + y _{m2 stitches} ).

In a third aspect, the present invention provides a multi-camera probe device for the aforementioned probe calibration method, comprising: a base frame, a first driving module, a second driving module, a first test axis, a second test axis axis and a third camera; the first drive module and the second drive module are arranged on the base frame, the first test shaft is mounted on the first drive module, the second The test shaft is installed on the second drive module; the first test shaft includes a first bracket, a first camera and a first probe, and the first bracket is installed on the first drive module, so The first camera and the first probe are installed on the first bracket; the second test shaft includes a second bracket, a second camera and a second probe, and the second bracket is installed on the first bracket On the two drive modules, the second camera and the second probe are mounted on the second bracket.

In one or more embodiments, a calibration plate fixing member is further provided on the base frame, and the third camera is fixed under the calibration plate fixing member.

In a fourth aspect, the present invention provides a flying probe tester, which includes a memory, a processor and the aforementioned multi-camera probe device, wherein a computer program is stored in the memory, and when the computer program is executed by the processor , so that the processor performs the steps of the probe calibration method of claim 7 .

Compared with the prior art, the calibration method, the multi-camera probe device and the flying probe tester provided by the present invention have simple calibration process for the camera system, high calibration accuracy, and can realize the calibration after the calibration of the camera system is completed. One-key calibration of the probe without the need to manually teach the actual position multiple times; after replacing the probe, there is no need to manually teach the probe position again. actual location in the system. The calibration accuracy of the calibration method for the camera can reach below 0.7 μm, and the calibration accuracy for the probe can reach below 3.5 μm.

Description of drawings

FIG. 1 is a schematic three-dimensional structure diagram of a multi-camera probe device according to an embodiment of the present invention;

2 is a flowchart of a multi-camera calibration method in an embodiment of the present invention;

3 is a flowchart of a probe calibration method in an embodiment of the present invention;

FIG. 4 is a structural block diagram of a flying probe tester in an embodiment of the present invention.

Description of main reference signs:

1-Multi-camera probe device, 11-Base frame, 12-First drive module, 13-Second drive module, 14-First test axis, 15-Second test axis, 16-Third camera, 17 -Calibration plate fixing member, 121-first moving axis mechanism, 122-first transfer mechanism, 131-second moving axis mechanism, 132-second transfer mechanism, 141-first bracket, 142-first camera, 143-first probe, 151-second bracket, 152-second camera, 153-second probe.

Detailed ways

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

Unless expressly stated otherwise, throughout the specification and claims, the term "comprising" or its conjugations such as "comprising" or "comprising" and the like will be understood to include the stated elements or components, and Other elements or other components are not excluded.

Referring to FIG. 1 , a multi-camera probe device 1 in an embodiment of the present invention can be applied to a flying probe tester. The multi-camera probe device 1 includes: a base frame 11 , a first driving module 12 , The second driving module 13 , the first test shaft 14 , the second test shaft 15 and the third camera 16 .

In an exemplary embodiment, the base frame 11 is generally configured as a rectangular parallelepiped, and the first driving module 12 and the second driving module 13 are mounted on the top of the base frame 11 . The first testing shaft 14 is mounted on the first driving module 12 , and the first testing shaft 14 can move under the driving of the first driving module 12 . The second testing shaft 15 is mounted on the second driving module 13 , and the second testing shaft 15 can move under the driving of the second driving module 13 .

Specifically, the first driving module 12 includes a first moving axis mechanism 121 arranged along the x-axis direction and a first transfer mechanism 122 arranged along the y-axis direction. The first moving shaft mechanism 121 is installed on the top of the base frame 11, the first transfer mechanism 122 is installed on the first moving shaft mechanism 121, and the first test shaft 14 is installed on the first transfer mechanism 122; The transfer mechanism 122 can move in the x-axis direction under the driving of the first moving shaft mechanism 121 , and the first test shaft 14 mechanism can move in the y-axis direction under the driving of the first transfer mechanism 122 .

Specifically, the second driving module 13 includes a second moving axis mechanism 131 arranged along the x-axis direction and a second transfer mechanism 132 arranged along the y-axis direction. The second moving shaft mechanism 131 is installed on the top of the base frame 11, the second transfer mechanism 132 is installed on the second moving shaft mechanism 131, and the second test shaft 15 is installed on the second transfer mechanism 132; The transfer mechanism 132 can move in the x-axis direction under the driving of the second moving shaft mechanism 131 , and the second test shaft 15 mechanism can move in the y-axis direction under the driving of the second transfer mechanism 132 .

The first driving module 12 and the second driving module 13 are independent of each other, and the first testing shaft 14 and the second testing shaft 15 can move independently under the driving of the first driving module 12 and the second driving module 13 respectively.

Specifically, the first test shaft 14 includes a first bracket 141 , a first camera 142 and a first probe 143 . The first bracket 141 is installed on the first transfer mechanism 122 of the first driving module 12 , and the first camera 142 and the first probe 143 are installed on the first bracket 141 . Under the driving of the first driving module 12, the first camera 142 and the first probe 143 can move synchronously.

Specifically, the second test shaft 15 includes a second bracket 151 , a third camera and a second probe. Wherein, the second bracket 151 is installed on the first transfer mechanism 122 of the second driving module 13 , and the second camera 152 and the first-person probe are installed on the second bracket 151 . Under the driving of the second driving module 13, the second camera 152 and the second probe 153 can move synchronously.

In an exemplary embodiment, the base frame 11 is further provided with a calibration plate fixing member 17 . When performing camera calibration, the calibration plate can be fixed on the calibration plate fixing member 17 so that the camera can photograph the calibration plate and obtain image information of the calibration plate.

Specifically, the third camera 16 is fixed below the calibration plate fixing member 17 . In the multi-camera probe device 1, the third camera 16 is fixed, so during the calibration process, the camera center of the third camera 16 is fixed, that is, the image coordinate system established based on the third camera 16 is also fixed, There is no position offset.

Specifically, the first camera 142 , the second camera 152 and the third camera 16 are also provided with light sources, and the light sources can provide lighting when the first camera 142 , the second camera 152 and the third camera 16 capture images, so that the cameras A clearer image can be captured.

Please refer to FIG. 2 , which is a flowchart of a multi-camera calibration method according to an embodiment of the present invention, which can be used in the aforementioned multi-camera probe device 1 . The multi-camera calibration method includes the following steps:

S101 : Establish the image coordinate system XYZ _p1 and the mechanical coordinate system XYZ _m1 of the first camera 142 , the image coordinate system XYZ _p2 and the mechanical coordinate system XYZ _m2 of the second camera 152 , and the image coordinate system XYZ _p3 of the third camera 16 .

Specifically, the image coordinate system XYZ _p1 of the first camera 142 is a coordinate system established based on the image captured by the first camera 142 , and the origin of the coordinate system XYZ _p1 is the image center of the image captured by the first camera 142 . For example, a plane perpendicular to the shooting direction (lens orientation) of the first camera 142 and passing through the center of the first camera 142 may be the XY plane of the coordinate system XYZ _p1 ; at this time, the center of the first camera 142 is the first The image center of the image captured by the camera 142 is the origin of the coordinate system XYZ _p1 .

The image coordinate system XYZ _p2 of the second camera 152 is a coordinate system established based on the image captured by the second camera 152 , and the origin of the coordinate system XYZ _p2 is the image center of the image captured by the second camera 152 . For example, a plane perpendicular to the shooting direction (lens orientation) of the second camera 152 and passing through the center of the second camera 152 may be the XY plane of the coordinate system XYZ _p2 ; at this time, the center of the second camera 152 is the second The image center of the image captured by the camera, that is, the origin of the coordinate system XYZ _p2 .

The image coordinate system XYZ _p3 of the third camera 16 is a coordinate system established based on the image captured by the third camera 16 , and the origin of the coordinate system XYZ _p3 is the image center of the image captured by the third camera 16 . For example, a plane perpendicular to the shooting direction (lens orientation) of the third camera 16 and passing through the center of the third camera 16 may be the XY plane of the coordinate system XYZ _p3 ; at this time, the center of the third camera 16 is the third The image center of the image captured by the camera 16 is the origin of the coordinate system XYZ _p3 .

In other embodiments, the origins of the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 may also be selected from other points on the image, and may be set according to actual needs.

Specifically, the origin of the mechanical coordinate system XYZ _m1 of the first camera 142 is the mechanical position where the first camera 142 is located when the first camera 142 shoots the calibration plate, and the mechanical position can pass through the first test axis 14 in the first drive module The position (x _m1c , y _m1c ) on 12 is determined.

The origin of the mechanical coordinate system XYZ _m2 of the second camera 152 is the mechanical position of the second camera 152 when the second camera 152 shoots the calibration plate. position (x _m2c , y _m2c ) to determine.

S102: The physical coordinate system XYZ _mb of the calibration board is established with the preset point on the calibration board as the origin, and the physical coordinates [x _mb , y _mb ] of the calibration point are calculated according to the actual physical distance between the preset point and the calibration point on the calibration board ^T.

Before establishing the physical coordinate system XYZ _mb of the calibration plate, the calibration plate needs to be positioned on the calibration plate fixing member 17 of the multi-camera probe device 1 first.

Specifically, the physical coordinate system XYZ _mb of the calibration plate can be constructed from the calibration points captured by the camera, and a preset point can be selected from the field of view acquired by the camera as the origin of the coordinate system XYZ _mb , while the calibration plane of the calibration plate is Can be used as the XY plane of the coordinate system XYZ _mb .

Both the preset point and the calibration point are located on the XY plane of the coordinate system XYZ _mb . According to the actual physical distance between the preset point and the calibration point, the physical coordinates of the calibration point in the coordinate system XYZ _mb can be calculated [x _mb ,y _mb ] ^T.

S103: The first camera 142, the second camera 152 and the third camera 16 capture images of the calibration plate, and determine the image coordinates [x _p1 , y _p1 ] ^T of the calibration point in the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 , [x _p2 ,y _p2 ] ^T and [x _p3 ,y _p3 ] ^T .

Specifically, the calibration plate is captured by the first camera 142, the second camera 152 and the third camera 16, and the first image captured by the first camera 142, the second image captured by the second camera, and the third captured calibration plate of the calibration plate are obtained. The third image captured by the camera 16, according to the pixel position of the calibration point in the first image, the second image and the third image, calculate the calibration point in the first image, the second image and the third image relative to each image center ( The image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] T and [x _p1 , y p1 ] T , [x _p2 , y _p2 ] ^T and [x _p3 ,y _p3 ] ^T .

Specifically, the preset points and calibration points on the calibration plate are set to be visible on both sides, and the first camera 142 and the second camera 152 can photograph the preset points and calibration points of the calibration plate through the first surface (top surface) of the calibration plate. To fix the point, the third camera 16 can photograph the preset point and the calibration point of the calibration plate through the second surface (bottom surface) of the calibration plate. The calibration plate is also provided with signs for determining the position of the preset point and for determining the direction of the physical coordinate system XYZ _mb of the calibration plate. Through this identification combined with image analysis, the first camera 142, the second camera 152 and the third camera 16 can automatically identify the position of the preset point and the direction of the coordinate system XYZ _mb , so that the same calibration point on the calibration plate can be placed on the first image, Correspondence is made in the second image and the third image.

S104: According to the physical coordinates [x _mb , y _mb ] ^T of the calibration point and the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T , [x _p3 , y _p3 ] ^T of the calibration point, pass Homogeneous transformation to obtain transformation matrices M _mb→p1 , M _mb→p2 and M _mb→p3 from the coordinate system XYZ _mb to the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 .

Specifically, the homogeneous transformation formula is:

Among them, Z _c is the vertical height from the calibration plane of the calibration plate (XY plane of the coordinate system XYZ _mb ) to the image coordinate system plane of the camera (XY plane of the coordinate system XYZ _p1 , XYZ _p2 and XYZ _p3 ), (u, v) is the point in the calibration plate image obtained by the camera, dX×dY is the physical size of a single pixel, (u ₀ , v ₀ ) is the translation amount from the upper left corner of the calibration plate image obtained by the camera to the origin of the camera image coordinate system (camera center) , f is the focal length of the camera lens, (X _w , Y _w , Z _w ) are the point coordinates in the physical space, and M is the transformation matrix between the physical coordinate system of the calibration board and the camera image coordinate system.

S105: According to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, obtain the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 , and obtain the coordinate system through affine transformation A transformation matrix M _p1→m1 from XYZ _p1 to coordinate system XYZ _m1 , and a transformation matrix M _p2→m2 from coordinate system XYZ _p2 to coordinate system XYZ _m2 .

Specifically, according to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, the pixel distance between the calibration point and the image center (corresponding to the camera center) is calculated, and the calibration point is obtained by a single pixel quantity The physical distance from the center of the first camera 142 and the center of the second camera 152, according to the mechanical coordinates of the center of the first camera 142 in the coordinate system XYZ _m1 and the mechanical coordinates of the center of the second camera 152 in the coordinate system XYZ _m2 , the calibration point can be obtained. Mechanical coordinates in coordinate systems XYZ _m1 and XYZ _m2 .

Based on the image coordinates [x _p1 , y _p1 ] ^T and [x _p2 , y _p2 ] ^T of the calibration point, and the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 , through affine transformation, the coordinate system XYZ can be obtained The transformation matrix M _p1→m1 from _p1 to the coordinate system XYZ _m1 , and the transformation matrix M _p2→m2 from the coordinate system XYZ _p2 to the coordinate system XYZ _m2 .

S106 : According to the transformation matrices M _mb→p1 , M _mb→p2 , M _mb→p3 , M _p1→m1 and M _p2→m2 , obtain the mechanical coordinate system XYZ _m1 of the first camera 142 and the mechanical coordinate system of the second camera 152 The conversion relationship between XYZ _m2 and the image coordinate system XYZ _p3 of the third camera 16 .

Specifically, after the transformation matrices M _mb→p1 , M _mb→p2 , M _mb→p3 , M _p1→m1 and M _p2→m2 are obtained, the coordinate system of the first camera 142 can be obtained through the coordinate system shared by the transformation matrices. The conversion relationship between the mechanical coordinate system XYZ _m1 , the mechanical coordinate system XYZ _m2 of the second camera 152 , and the image coordinate system XYZ _p3 of the third camera 16 . The process is as follows:

Let a point in the coordinate system XYZ _mb be (x _mb1 , y _mb1 ), its coordinates in the coordinate system XYZ _m1 are (x _m11 , y _m11 ), and its coordinates in the coordinate system XYZ _m2 are (x _m21 , y _m21 ), its coordinates in the coordinate system XYZ _p3 are (x _p31 , y _p31 ), then there are

From the above formula, the projection matrix of coordinate system XYZ _m2 to coordinate system XYZ _m1 can be obtained as M _mb→p1 ·M _p1→m1 ·M _p2→m2 ^-1 ·M _mb→p2 ^-1 ; coordinate system XYZ _m1 to coordinate system XYZ The projection matrix of _m2 is M _mb→p2 ·M _p2→m2 ·M _p1→m1 ^-1 ·M _mb→p1 ^-1 ; the projection matrix of coordinate system XYZ _p3 to coordinate system XYZ _m1 is M _mb→p1 ·M _{p1→ m1} ·M _mb→p3 ^-1 ; the projection matrix of the coordinate system XYZ _p3 to the coordinate system XYZ _m2 is M _mb→p2 ·M _p2→m2 ·M _mb→p3 ^-1 , that is, the calibration of the camera system is completed.

Based on the aforementioned multi-camera calibration method, the present invention also provides a probe calibration method. Please refer to FIG. 3 , which is a flowchart of a probe calibration method in an embodiment of the present invention. The probe calibration method can be applied to a flying probe tester having the aforementioned multi-camera probe device 1 . The multi-camera calibration method includes the following steps:

S201 : Complete the calibration of the first camera 142 , the second camera 152 and the third camera 16 based on the multi-camera calibration method.

Specifically, after the calibration of the camera is completed, the calibration plate positioned on the calibration plate fixing member 17 needs to be removed to prevent the calibration plate from affecting the subsequent shooting of the probe by the third camera 16 .

S202: Move the first probe 143 into the field of view of the third camera 16, and use the third camera 16 to capture an image of the first probe 143 to obtain the coordinates of the tip of the first probe 143 in the coordinate system XYZ _p3 , according to The conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m1 obtains the coordinates of the needle tip of the first probe 143 in the coordinate system XYZ _m1 .

Specifically, after the third camera 16 acquires the image of the first probe 143, the coordinates of the tip of the first probe 143 in the coordinate system XYZ _p3 can be obtained through BLOB analysis (x _{p1 needle} , y _{p1 needle} , 1).

According to the conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m1 , the coordinates (x _{m1 needle} , y _{m1 needle} , 1) ^T of the needle tip of the first probe 143 in the coordinate system XYZ _m1 are obtained. Specifically:

(x _{m1 pin} , y _{m1 pin} , 1) ^T = M _{mb → p1} · M _{p1 → m1} · M _{mb → p3} ^-1 · (x _{p1 pin} , y _{p1 pin} , 1) ^T .

S203: Obtain the first probe according to the mechanical position of the first test shaft 14 when the first camera 142 photographed the calibration plate, and the mechanical position of the first test shaft 14 when the third camera 16 photographed the first probe 143 143 The actual offset of the needle tip to the center of the first camera 142 completes the calibration of the first camera 142 by the first probe 143 .

Specifically, the mechanical position of the first test shaft 14 when the first camera 142 photographs the calibration plate is (x _m1c , y _m1c ), and the mechanical position of the first test shaft 14 when the third camera 16 photographs the first probe 143 The position is (x _{1 pin} , y _{1 pin} , z _{1 pin} ), then the deviation between the calibration position of the first probe 143 and the calibration position of the first camera 142 is (x _{1 pin-} x _m1c , y _{1 pin} -y _m1c ).

Then according to the coordinates of the needle tip of the first probe 143 in the coordinate system XYZ _m1 (x _{m1 needle} , y _{m1 needle} , 1) ^T , the actual offset from the needle tip of the first probe 143 to the center of the first camera 142 (Δx ₁ , Δy ₁ )=(x _{1 pin-} x _m1c +x _{m1 pin} , y _{1 pin} -y _m1c +y _{m1 pin} ), that is, the calibration of the first probe 143 to the center of the first camera 142 is completed.

S204: Move the second probe 153 into the field of view of the third camera 16, and use the third camera 16 to capture an image of the second probe 153 to obtain the coordinates of the tip of the second probe 153 in the coordinate system XYZ _p3 , according to The conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m2 obtains the coordinates of the needle tip of the second probe 153 in the coordinate system XYZ _m2 .

Specifically, after the third camera 16 acquires the image of the second probe 153, the coordinates of the needle tip of the second probe 153 in the coordinate system XYZ _p3 can be obtained through BLOB analysis (x _{p2 needle} , y _{p2 needle} , 1).

According to the conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m2 , the coordinates (x _{m2 needle} , y _{m2 needle} , 1) ^T of the needle tip of the first probe 143 in the coordinate system XYZ _m2 are obtained. Specifically:

(x _{m2 pin} , y _{m2 pin} , 1) ^T = M _{mb → p2} · M _{p2 → m2} · M _{mb → p3} ^-1 · (x _{p2 pin} , y _{p2 pin} , 1) ^T .

S205: Obtain the second probe according to the mechanical position of the second test shaft 15 when the second camera 152 photographed the calibration plate, and the mechanical position of the second test shaft 15 when the third camera 16 photographed the second probe 153 153 The actual offset of the needle tip to the center of the second camera 152 completes the calibration of the second camera 152 by the second probe 153 .

Specifically, the mechanical position of the second test axis 15 when the second camera 152 shoots the calibration plate is (x _m2c , y _m2c ), and the mechanical position of the second test axis when the third camera 16 shoots the second probe 153 is (x _{2 pins} , y _{2 pins} , z _{2 pins} ), then the deviation between the calibration position of the second probe 153 and the calibration position of the second camera 152 is (x _{2 pins-} x _m2c , y _{2 pins} -y _m2c ) .

Then according to the coordinates of the needle tip of the second probe 153 in the coordinate system XYZ _m2 (x _{m2 needle} , y _{m2 needle} , 1) ^T , the actual offset from the needle tip of the second probe 153 to the center of the second camera 152 (Δx ₂ , Δy ₂ )=(x _{2 pins-} x _m2c +x _{m2 pins} , y _{2 pins} -y _m2c +y _{m2 pins} ), that is, the calibration of the second probe 153 to the center of the second camera 152 is completed.

As shown in FIG. 4 , an embodiment of the present invention further provides a flying probe testing machine 2 . The flying probe testing machine 2 includes a processor 21 , a memory 22 (for example, a non-volatile memory), a memory 23 , and a communication interface 24 and the multi-camera probe device 1 as previously described, and the processor 21 , the memory 22 , the memory 23 and the communication interface 24 are connected together via the bus 25 . The processor 21 is configured to invoke at least one program instruction stored or encoded in the memory 22, so that the processor 21 performs various operation steps and functions of the probe calibration method described in the various embodiments of this specification. When the processor 21 executes the probe calibration method described in the various embodiments of this specification, specific operation steps can be implemented by the multi-camera probe device 1 .

The memory 22 may be any available medium or data storage device that can be accessed by a computer, including, but not limited to, magnetic memory (eg, floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical memory (eg, CD, DVD, BD, etc.) , HVD, etc.), and semiconductor memory (eg, ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid state disk (SSD), etc.).

To sum up, the calibration method, the multi-camera probe device and the flying probe tester provided by the present invention have a simple process of calibrating the camera system, high calibration accuracy, and can realize the calibration of the probe after the calibration of the camera system is completed. One-key calibration without the need to manually teach the actual position multiple times; after replacing the probe, there is no need to manually teach the probe position again. By taking a visual photo, you can directly determine that the new probe is in the system under the premise of the previous calibration. actual location. The calibration accuracy of the calibration method for the camera can reach below 0.7 μm, and the calibration accuracy for the probe can reach below 3.5 μm.

The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise form disclosed, and obviously many changes and modifications are possible in light of the above teachings. The exemplary embodiments were chosen and described for the purpose of explaining certain principles of the invention and their practical applications, to thereby enable one skilled in the art to make and utilize various exemplary embodiments and various different aspects of the invention. Choose and change. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims (13)

1. a multi-camera calibration method, is characterized in that, comprises the following steps: establishing the image coordinate system XYZ _p1 and the mechanical coordinate system XYZ _m1 of the first camera, the image coordinate system XYZ _p2 and the mechanical coordinate system XYZ _m2 of the second camera, and the image coordinate system XYZ _p3 of the third camera; The physical coordinate system XYZ _mb of the calibration board is established with the preset point on the calibration board as the origin, and the physical coordinates [x _mb , y _mb ] ^T of the calibration point are calculated according to the actual physical distance between the preset point and the calibration point on the calibration board; The images of the calibration plate are captured by the first camera, the second camera and the third camera, and the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y of the calibration point in the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 are determined _p2 ] ^T and [x _p3 , y _p3 ] ^T ; According to the physical coordinates [x _mb , y _mb ] ^T of the calibration point and the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T , [x _p3 , y _p3 ] ^T of the calibration point, the Transform to obtain the transformation matrix M _mb→p1 , M _mb→p2 and M _mb→p3 from the coordinate system XYZ _mb to the coordinate system XYZ _p1 , XYZ _p2 and XYZ _p3 ; According to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, obtain the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 , and obtain the coordinate system XYZ _p1 through affine transformation to the transformation matrix M _p1→m1 of the coordinate system XYZ _m1 , and the transformation matrix M _p2→m2 of the coordinate system XYZ _p2 to the coordinate system XYZ _m2 ; According to the transformation matrices M _mb→p1 , M _mb→p2 , M _mb→p3 , M _p1→m1 and M _p2→m2 , the mechanical coordinate system XYZ _m1 of the first camera, the mechanical coordinate system XYZ _m2 of the second camera, and the third The transformation relationship between the image coordinate systems XYZ _p3 of the three cameras. 2 . The multi-camera calibration method according to claim 1 , wherein the origin of the coordinate system XYZ _p1 is the image center of the image captured by the first camera, and the origin of the coordinate system XYZ _p2 is captured by the second camera 2 . The image center of the image, the origin of the coordinate system XYZ _p3 is the image center of the image captured by the third camera. 3. The multi-camera calibration method according to claim 1, wherein the origin of the coordinate system XYZ _m1 is the mechanical position of the first camera when the first camera shoots the calibration plate, and the coordinate system XYZ _m2 is at the mechanical position. The origin is the mechanical position where the second camera is located when the second camera shoots the calibration plate. 4. The multi-camera calibration method according to claim 1, wherein the first camera, the second camera and the third camera are used to capture images of the calibration plate, and it is determined that the calibration point is in the coordinate systems XYZ _p1 , XYZ _p2 and Image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T and [x _p3 , y _p3 ] ^T in XYZ _p3 , including: The calibration plate is photographed by the first camera, the second camera and the third camera, the first image, the second image and the third image of the calibration plate are obtained, and the pixels in the first image, the second image and the third image according to the calibration point position, and determine the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T and [x _p3 , y _p3 ] ^T of the calibration point in the coordinate systems XYZ _p1 , XYZ _p2 and XYZ _p3 . 5. The multi-camera calibration method according to claim 1, wherein the homogeneous transformation formula is:

Among them, Z _c is the vertical height from the calibration plane of the calibration plate to the image coordinate system plane of the camera, (u, v) is the point in the image of the calibration plate obtained by the camera, dX×dY is the physical size of a single pixel, (u ₀ , v ₀ ) is the translation amount from the upper left corner of the calibration plate image obtained by the camera to the origin of the camera image coordinate system, f is the focal length of the camera lens, (X _w , Y _w , Z _w ) is the point coordinates in the physical space, M is the The transformation matrix between the physical coordinate system of the calibration board and the camera image coordinate system. 6 . The multi-camera calibration method according to claim 1 , wherein, according to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, the coordinate system of the calibration point is obtained. 7 . Mechanical coordinates in XYZ _m1 and XYZ _m2 , including: According to the image coordinates [x _p1 , y _p1 ] ^T , [x _p2 , y _p2 ] ^T of the calibration point, calculate the pixel distance between the calibration point and the center of the image, and obtain the calibration point and the center of the first camera and the second camera through a single pixel According to the mechanical coordinates of the center of the first camera in the coordinate system XYZ _m1 and the mechanical coordinates of the center of the second camera in the coordinate system XYZ _m2 , the mechanical coordinates of the calibration point in the coordinate systems XYZ _m1 and XYZ _m2 are obtained. 7 . The multi-camera calibration method according to claim 1 , wherein, according to the transformation matrices M _mb→p1 , M _mb→p2 , M _mb→p3 , M _p1→m1 and M _p2→m2 , the first The conversion relationship between the mechanical coordinate system XYZ _m1 of a camera, the mechanical coordinate system XYZ _m2 of the second camera, and the image coordinate system XYZ _p3 of the third camera, including: Let a point in the coordinate system XYZ _mb be (x _mb1 , y _mb1 ), its coordinates in the coordinate system XYZ _m1 are (x _m11 , y _m11 ), and its coordinates in the coordinate system XYZ _m2 are (x _m21 , y _m21 ) ), its coordinates in the coordinate system XYZ _p3 are (x _p31 , y _p31 ), then there are

According to formula (1), the projection matrix of coordinate system XYZ _m2 to coordinate system XYZ _m1 is M _mb→p1 ·M _p1→m1 ·M _p2→m2 ^-1 ·M _mb→p2 ^-1 ; coordinate system XYZ _m1 to The projection matrix of the coordinate system XYZ _m2 is M _mb→p2 ·M _p2→m2 ·M _p1→m1 ^-1 ·M _mb→p1 ^-1 ; the projection matrix of the coordinate system XYZ _p3 to the coordinate system XYZ _m1 is M _mb→p1 · M _p1→m1 ·M _mb→p3 ^-1 ; the projection matrix of the coordinate system XYZ _p3 to the coordinate system XYZ _m2 is M _mb→p2 ·M _p2→m2 ·M _mb→p3 ^-1 . 8 . The multi-camera calibration method according to claim 1 , wherein the preset point and the calibration point on the calibration plate are set to be visible on both sides, and the first camera and the second camera pass through the The first surface of the calibration plate captures the preset point and the calibration point, and the third camera captures the preset point and the calibration point through the second surface of the calibration plate. 9. The multi-camera calibration method according to claim 8, wherein the calibration plate is provided with a position for determining the preset point and a direction for determining the physical coordinate system XYZ _mb of the calibration plate 's identification. 10. A probe calibration method, applied to a flying probe testing machine, the flying probe testing machine comprises a first test shaft, a second test shaft and a third camera, and the first test shaft is provided with a first camera and a third camera. A probe with a second camera and a second probe on the second test shaft, characterized in that the probe calibration method includes the following steps: Based on the multi-camera calibration method according to any one of claims 1 to 9, the calibration of the first camera, the second camera and the third camera is completed; Move the first probe into the field of view of the third camera, take an image of the first probe through the third camera, and obtain the coordinates of the first probe tip in the coordinate system XYZ _p3 (x _{p1 needle} , y _{p1 needle} , 1), according to the conversion relationship between the coordinate system XYZ _p3 and the coordinate system XYZ _m1 , obtain the coordinates of the first probe tip in the coordinate system XYZ _m1 (x _{m1 needle} , y _{m1 needle} , 1) ^T ; According to the mechanical position (x _m1c , y _m1c ) of the first test axis when the first camera shoots the calibration board, and the mechanical position of the first test axis when the third camera shoots the first probe (x _{1 pin} , y _{1 pin} , z _{1 pin} ), obtain the actual offset (Δx ₁ , Δy ₁ ) from the tip of the first probe to the center of the first camera, and complete the calibration of the first camera by the first probe; Move the second probe into the field of view of the third camera, and take an image of the second probe through the third camera to obtain the coordinates of the second probe tip in the coordinate system XYZ _p3 (x _{p2 needle} , y _{p2 needle} , 1), according to the conversion relationship between coordinate system XYZ _p3 and coordinate system XYZ _m2 , obtain the coordinates of the second probe tip in coordinate system XYZ _m2 (x _{m2 needle} , y _{m2 needle} , 1) ^T ; According to the mechanical position of the second test axis (x _m2c , y _m2c ) when the second camera shoots the calibration board, and the mechanical position of the second test axis when the third camera shoots the second probe (x _{2 pins} , y _{2 pins} , z _{2 pins} ), obtain the actual offset (Δx ₂ , Δy ₂ ) from the second probe tip to the center of the second camera, and complete the calibration of the second probe to the second camera; wherein, (x _{m1 pin} , y _{m1 pin} , 1) ^T = M _mb→p1 M _p1→m1 M _mb→p3 ^-1 (x _{p1 pin} , y _{p1 pin} , 1) ^T (Δx ₁ , Δy ₁ )=(x _{1 pin-} x _m1c +x _{m1 pin} , y _{1 pin} -y _m1c +y _{m1 pin} ) (x _{m2 pin} , y _{m2 pin} , 1) ^T = M _mb→p2 M _p2→m2 M _mb→p3 ^-1 (x _{p2 pin} , y _{p2 pin} , 1) ^T (Δx ₂ , Δy ₂ ) = (x _{2 stitches} - x _m2c + x _{m2 stitches} , y _{2 stitches} - y _m2c + y _{m2 stitches} ). 11. A camera probe device for the probe calibration method according to claim 10, characterized in that, comprising: a base frame, a first drive module, a second drive module, a first test shaft, a first Two test axes and a third camera; The first drive module and the second drive module are arranged on the base frame, the first test shaft is installed on the first drive module, and the second test shaft is installed on the on the second drive module; The first test shaft includes a first bracket, a first camera and a first probe, the first bracket is mounted on the first drive module, the first camera and the first probe are mounted on the on the first bracket; The second test shaft includes a second bracket, a second camera and a second probe, the second bracket is mounted on the second drive module, the second camera and the second probe are mounted on on the second bracket. 12 . The multi-camera probe device according to claim 11 , wherein a calibration plate fixing member is further provided on the base frame, and the third camera is fixed under the calibration plate fixing member. 13 . 13. A flying probe testing machine, characterized by comprising a memory, a processor and the multi-camera probe device according to claim 11 or 12, wherein a computer program is stored in the memory, and the computer program is When executed by the processor, the processor is caused to execute the steps of the probe calibration method of claim 10 .

Images