WO2021208231A1 - Gap measuring system and measuring method - Google Patents

Abstract

Disclosed are a gap measuring system and measuring method. The measuring system comprises a mechanical arm (4), an upper computer (1), a control cabinet (2) and a visual system. The upper computer (1) is connected to the control cabinet (2) to control the mechanical arm (4); the visual system comprises a binocular camera (5) and a structured light projector (6), with the binocular camera (5) and the structured light projector (6) both being fixed to an end of the mechanical arm (4); the mechanical arm (4) moves to a position above a gap, and the structured light projector (6) projects light onto the gap, such that an image of the gap is formed; the binocular camera (5) is used for collecting an image of the gap, and sends the image of the gap to the upper computer (1), wherein the upper computer (1) processes the image, and the upper computer (1) obtains, according to an image processing result, three-dimensional spatial coordinates of the gap and generates a spatial three-dimensional reconstruction model of the gap; and the upper computer (1) rotates the spatial three-dimensional reconstruction model of the gap to be parallel to a Z-axis of a world coordinate system, and the spatial three-dimensional reconstruction model of the gap after rotation is respectively projected onto different coordinate planes to calculate the width and face difference values of the gap.

Description

Gap measurement system and measurement method Technical field

The invention relates to the technical field of machine vision measurement, in particular to a gap measurement system and a measurement method.

Background technique

Gap measurement is a very important inspection item in industrial production. In recent years, with the continuous development of society and the continuous improvement of industrial production levels, in the field of intelligent manufacturing, it is often necessary to obtain online geometric feature information of industrial manufacturing objects. Among these geometric features, the gap between the workpieces is a relatively important feature information.

Existing gap detection methods are divided into two types: contact type and non-contact type. The contact type detection accuracy is low, and the measurement result cannot be processed in real time. Non-contact methods include three-coordinate measuring machine for measurement, theodolite measuring system measuring method, three-dimensional laser measuring instrument measuring method, and methods based on capacitance and ultrasound. However, these methods can only perform sampling tests on the test objects, and the number of sampling tests is extremely limited, the cost is high, and the actual production needs cannot be met.

Summary of the invention

The purpose of the present invention is to provide a gap measurement system and measurement method to solve the problem of high cost of existing gap measurement.

In order to solve the above technical problems, the present invention provides a gap measurement system. The gap measurement system includes a mechanical arm, a host computer, a control cabinet, and a vision system, wherein:

The host computer is connected to the control cabinet to control the mechanical arm;

The vision system includes a binocular camera and a structured light projector, and both the binocular camera and the structured light projector are fixed to the end of the mechanical arm;

The mechanical arm moves above the gap, and the structured light projector projects on the gap to form an image of the gap;

The binocular camera is used to collect the image of the gap, and send the image of the gap to the upper computer, the upper computer performs image processing, and the upper computer obtains the three-dimensional space of the gap according to the result of the image processing Coordinates, and generate a three-dimensional reconstruction model of the gap;

The space three-dimensional reconstruction model of the rotating slot of the host computer is parallel to the Z axis of the world coordinate system, and the space three-dimensional reconstruction model of the rotating slot is respectively projected to different coordinate planes to obtain the slot width and face difference values.

Optionally, in the gap measurement system, the gap measurement system further includes a motor control system, and the motor control system is used to adjust the baseline distance between the binocular camera and the structured light projector.

Optionally, in the gap measurement system, the motor control system includes a DC brushless motor and a DC brushless motor controller, and the DC brushless motor controller adopts double closed-loop control of a speed loop and a current loop;

When starting the DC brushless motor, the current loop is turned on, and the current increases so that the motor quickly reaches the specified speed, and at the same time the maximum current limit is started;

After the speed of the brushless DC motor stabilizes, the current loop acts as an inner loop for current limiting protection, and the speed loop acts as an outer loop to maintain a constant speed;

The brushless DC motor controller includes a hardware module and a software module. The hardware module is composed of a development board, a driver board, an encoder, an AD acquisition circuit, and a simulator; the development board outputs a PWM wave output waveform, and the signal is amplified by the driver board. Finally, it is input into the motor to control the start and stop of the motor and the speed and current of the motor; the encoder and AD acquisition circuit feedback the speed signal and current signal to the software module, and correct the duty cycle of the PWM wave output waveform through feedback to complete the closed loop of the motor control;

The software module sets the chip register of the development board, controls the timing and interrupt function of the development board chip, and makes it output PWM waveform; and according to the encoder to detect the motor speed and the feedback signal of the AD sampling circuit, adjust the PWM wave duty cycle, and then Adjust the size of the speed and the size of the current.

Optionally, in the gap measurement system, the mechanical arm includes a first single hollow joint and a second single hollow joint, and the first single hollow joint and the second single hollow joint are used to place a direct current sensor. Brush motor, the first single hollow joint and the second single hollow joint are integrated in a spherical shell in a staggered manner;

The spherical housing also has a first driver and a second driver. The first driver is used to drive the structured light projector and the binocular camera, and the second driver is used to drive the first single hollow joint and the The second single hollow joint;

The DC brushless motor is a hollow frameless motor, the rotor of the hollow frameless motor is directly connected to the harmonic reducer through a centrally controlled transmission shaft, and the end face of the harmonic reducer is used as a direct output end face; The other end of the motor is connected to the gear through the centrally controlled drive shaft, and the drive shaft is biased through the gear drive, and an electromagnetic brake and an incremental encoder are installed on the biased drive shaft.

Optionally, in the gap measurement system, the relationship matrix between the end of the robotic arm and the binocular camera is a hand-eye relationship matrix;

Before the binocular camera collects the image of the gap, calibrate the hand-eye relationship matrix and the camera internal and external parameters of the binocular camera;

The image of the gap collected by the binocular camera is a structured light image;

The binocular camera sends the image of the slit to the image processing system of the upper computer;

The image processing system processes the structured light image collected by the binocular camera, and uses a gap detection algorithm based on the Canny operator to perform edge extraction to obtain more contour information;

The result of edge extraction is stereo-matched, the coordinates of the slot matching point are obtained, and a spatial three-dimensional reconstruction model is formed, and the width of the slot and the value of the face difference are obtained.

Optionally, in the gap measurement system, calibrating the hand-eye relationship matrix includes: setting a world coordinate system on the work plane, and the world coordinate system does not coincide with the robot coordinate system. After completing the binocular camera After the internal and external parameters are calibrated, the position of the object in the world coordinate system is calculated; the coordinates of the object in the robot coordinate system are obtained, and the hand-eye relationship matrix is calculated according to the conversion between the world coordinate system and the robot coordinate system;

The edge extraction includes: adopting image preprocessing technology to reduce noise and enhance edge contour, and extend on the basis of Canny operator, so as to perform edge extraction of gap contour;

The stereo matching includes: now obtaining the coordinates of the edge points according to the result of edge contour extraction, and performing binocular stereo matching on the edge points under the restriction of the edge area, that is, first extracting the point coordinates, then matching cost calculation, then cost aggregation, and finally Disparity calculation;

Forming the spatial three-dimensional reconstruction model includes: obtaining the left and right projection images through a binocular camera, obtaining the correspondence between the left and right projection images through stereo matching, and obtaining the gap by using the calibration result of the binocular camera, that is, the coordinates of the edge contour points in the left and right projection images. The three-dimensional coordinates of the edge in space, and the three-dimensional reconstruction of the slit line according to the three-dimensional coordinates of the slit contour, and the method of rotating the slit surface to be parallel to a certain coordinate plane to obtain the slit width and the surface difference value.

Optionally, in the gap measurement system, the local coordinate system O _g -X _g Y _g Z _g of a single camera in the binocular camera coincides with the camera coordinate system O _c -X _c Y _c Z _c, and All are right-handed coordinate systems;

The origin of light emitted by the structured light projector is N, O _c is the center of the optical axis of the binocular camera; the emission point N of the structured light plane is on the O _c X _c Z _c coordinate plane, and the structured light plane is orthogonal to O _c X _c Z _c coordinate plane; the intersection line is PN, point P is the intersection point of the optical axis O _c Z _c and the structured light plane, the distance between the exit point N of the light plane and the optical center O _{c of the} camera, that is, the structured light vision measurement The baseline distance of the system |NO _c | = D, the angle between the structured light plane and the baseline PNO _c = α, the angle between the optical axis and the laser beam O _c PN = β;

In the model, the coordinate values are unified in the local coordinate system O _g -X _g Y _g Z _g , and the projection model for transforming the object coordinates in the local coordinate system to the image point in the image coordinate system is

In the formula: s = 1, 0 ^T = (0 0 0) ^T , R is a 3X3 unit orthogonal matrix, t is a 1x3 translation vector; it is described by structural parameters with clear physical meaning composed of D, α and β The light plane equation of structured light is

From the projection model (1) and the light plane equation (2), the structured light vision measurement mode is obtained as shown in equation (3):

It can be seen from (3) that the accurate determination of the coordinate value of the object point is closely related to the baseline distance D; the camera and light source are driven by a motor to change the baseline distance D to achieve accurate measurement.

Optionally, in the gap measurement system, the method of obtaining the three-dimensional geometric information of the object according to two or more images, assuming that there is an object in space, the left image plane I _{1 is obtained by the No. 1 camera and the No. 2 camera.} , The right image plane I ₂ , the coordinate of a point P on the object in space is [X Y Z] ^T , the projection points on the left image plane I ₁ and the right image plane I _{2 are respectively P l} and P _r , their homogeneity The coordinates are [u ₁ v ₁ 1] ^T and [u ₂ v ₂ 1] ^{T respectively} , then P _l and P _r have the following corresponding relations:

in,

M _L and M _R are the projection matrices of the No. 1 camera and the No. 2 camera _{respectively, and A l} and A _r are the internal parameters of the No. 1 camera and the No. 2 camera respectively.

Respectively external parameter matrix Camera 1 and Camera 2, wherein R _l, R _r are the number 1 camera 2 camera rotation matrix, t _l, t _r are the number 1 camera and the No. 2 camera translation vector , The internal parameters and external parameters of the camera are obtained through camera calibration. When we obtain the projection matrix M _L and M _R according to the internal and external parameters of the camera, we can eliminate Z _c1 or Z _c2 from the above equation to obtain the information about X, Y, and Z Four linear equations:

Among them: (u ₁ ,v ₁ ,1) ^T , (u ₂ ,v ₂ ,1) ^T are the homogeneous coordinates of points p _l and p _r in the left image plane I ₁ and the right image plane I _{2, (} X, Y, Z, 1) ^T is the homogeneous coordinates in the world coordinate system to be sought,

Is the element in the i-th row and j-th column of the projection matrix M _{L, similarly,}

Is the element in the i-th row and j-th column of the projection matrix M _R , and the above equations are solved, and the least square solution is the space coordinate sought, that is, the three-dimensional reconstruction of the cover gap is realized.

The present invention also provides a gap measurement method. The gap measurement method includes:

The upper computer controls the robotic arm by connecting to the control cabinet;

The binocular camera and structured light projector of the vision system are both fixed to the end of the robotic arm;

The mechanical arm moves above the gap, and the structured light projector projects on the gap to form an image of the gap;

The binocular camera is used to collect the image of the gap, and send the image of the gap to the upper computer, the upper computer performs image processing, and the upper computer obtains the three-dimensional space of the gap according to the result of the image processing Coordinates, and generate a three-dimensional reconstruction model of the gap;

The space three-dimensional reconstruction model of the rotating slot of the host computer is parallel to the Z axis of the world coordinate system, and the space three-dimensional reconstruction model of the rotating slot is respectively projected to different coordinate planes to obtain the slot width and face difference values.

In the gap measurement system and measurement method provided by the present invention, the control cabinet is connected to the upper computer to control the mechanical arm, the mechanical arm moves to the top of the gap, and the structured light projector is projected on the gap to form an image of the gap, and the binocular camera collects The image of the gap is sent to the upper computer, and the upper computer performs image processing. The upper computer obtains the three-dimensional space coordinates of the gap according to the result of the image processing, and generates a three-dimensional reconstruction model of the gap, and the upper computer rotates the three-dimensional space of the gap The reconstruction model is parallel to the Z axis of the world coordinate system, and the space three-dimensional reconstruction model of the rotated gap is projected to different coordinate planes to obtain the width and surface difference of the gap, which realizes the non-contact measurement of the gap. The measurement accuracy is high, and the software algorithm is used to realize the measurement. There is no need for large-scale investment in hardware. Only software algorithms are required for measurement and image processing. The hardware cost is low, which solves the problem of high cost of gap measurement.

Description of the drawings

FIG. 1 is a schematic diagram of the structure of a mechanical arm based on structured light vision according to an embodiment of the present invention;

2 is a schematic diagram of a checkerboard used by Zhang Youzheng's calibration method according to another embodiment of the present invention;

Figure 3 is a general diagram of a brushless DC motor control system according to another embodiment of the present invention;

4 is a flowchart of a hardware module of a brushless DC motor control system according to another embodiment of the present invention;

Figure 5 is a cross-sectional view of a mechanical arm joint according to another embodiment of the present invention;

Fig. 6 is a structured light vision measurement model of another embodiment of the present invention;

7 is a schematic diagram of the three-dimensional reconstruction of the slit line represented in the detection method of another embodiment of the present invention;

FIG. 8 is a schematic diagram of the rotation of the spatial straight line represented in the detection method of another embodiment of the present invention;

9 is a schematic diagram of the structure of a brushless DC motor according to another embodiment of the present invention;

Fig. 10 is a left measurement image taken by a binocular vision system according to another embodiment of the present invention;

Fig. 11 is a right measurement image taken by a binocular vision system according to another embodiment of the present invention;

12 is a schematic diagram of the three-dimensional reconstruction of the slit line represented in the detection method of another embodiment of the present invention;

FIG. 13 is a schematic diagram of the rotation of the spatial straight line represented in the detection method of another embodiment of the present invention;

14 is a schematic diagram of the projection of the slit line on the X-Z plane in the detection method of another embodiment of the present invention;

15 is a schematic diagram showing the projection of the slit line on the Y-Z plane in the detection method according to another embodiment of the present invention;

As shown in the figure: 1-Upper computer; 2-Control cabinet; 3-Motor control system; 4-Manipulator; 41-The first single hollow joint; 42-The second single hollow joint; 43-The first driver; 44- The second driver; 5-binocular camera; 6-structured light projector.

Detailed ways

The gap measurement system and measurement method proposed by the present invention will be further described in detail below in conjunction with the drawings and specific embodiments. According to the following description and claims, the advantages and features of the present invention will be clearer. It should be noted that the drawings all adopt a very simplified form and all use imprecise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

The core idea of the present invention is to provide a gap measurement system and measurement method to solve the existing problem of high cost of gap measurement.

In order to realize the above ideas, the present invention provides a gap measurement system and a measurement method. The gap measurement system includes a mechanical arm, an upper computer, a control cabinet, and a vision system, wherein: the upper computer is connected to the control cabinet to control The robotic arm; the vision system includes a binocular camera and a structured light projector, the binocular camera and the structured light projector are both fixed to the end of the robotic arm; the robotic arm moves to above the gap, so The structured light projector is projected on the gap to form an image of the gap; the binocular camera is used to collect the image of the gap, and send the image of the gap to the upper computer, and the upper computer performs image processing , The host computer obtains the three-dimensional space coordinates of the gap according to the result of image processing, and generates a spatial three-dimensional reconstruction model of the gap; The space three-dimensional reconstruction model of the gap is projected to different coordinate planes to obtain the width and surface difference of the gap.

<Example One>

This embodiment proposes a vision-based gap measurement system, as shown in Figure 1, which includes a motor control system 3, a robotic arm 4, and a vision system. The control cabinet 2 controls and connects to the robotic arm 4, and the upper computer 1 is connected to the control cabinet 2. The vision system includes a binocular camera 5 and a structured light projector 6. The motor control system 3 adjusts the baseline distance between the binocular camera 5 and the structured light projector 6. The binocular camera 5 is used to collect the image projected by the structured light projector 6, and The image is transmitted to the image processing system composed of the software and hardware of the host computer. The binocular camera 5 is fixed to the end of the robotic arm 4, and the structured light projector 6 is projected on the gap. The relationship matrix between the end of the robotic arm 4 and the binocular camera 5 The internal and external parameters of the camera are calibrated first, the relationship matrix between the end of the robotic arm 4 and the binocular camera 5, that is, the hand-eye relationship matrix; then the structured light image collected by the binocular camera 5 is processed, and the gap detection based on the Canny operator is used The algorithm performs edge extraction to obtain more contour information; then, the result of edge extraction is stereo-matched to obtain the coordinates of the gap matching point, and then three-dimensional reconstruction is performed to obtain the gap width and the surface difference value.

In the gap measurement system and measurement method provided by the present invention, the control cabinet is connected to the upper computer to control the mechanical arm, the mechanical arm moves to the top of the gap, and the structured light projector is projected on the gap to form an image of the gap, and the binocular camera collects The image of the gap is sent to the upper computer, and the upper computer performs image processing. The upper computer obtains the three-dimensional space coordinates of the gap according to the result of the image processing, and generates a three-dimensional reconstruction model of the gap, and the upper computer rotates the three-dimensional space of the gap The reconstruction model is parallel to the Z axis of the world coordinate system, and the space three-dimensional reconstruction model of the rotated gap is projected to different coordinate planes to obtain the width and surface difference of the gap, which realizes the non-contact measurement of the gap. The measurement accuracy is high, and the software algorithm is used to realize the measurement. There is no need for large-scale investment in hardware. Only software algorithms are required for measurement and image processing. The hardware cost is low, which solves the problem of high cost of gap measurement.

Specifically, the calibration of the hand-eye relationship matrix includes: calibrating the internal parameter matrix and distortion parameters of the camera by the Zhang Zhengyou calibration method; as shown in FIG. 2, the camera is used at different distances, different orientations, and different tilts to the chessboard are required. Take several pictures of a checkerboard from a complete angle. Detect inner corners in the picture, that is, corner detection and extraction of sub-pixel information;

In addition, the calibration of the internal and external camera parameters of the binocular camera includes: before the calibration operation, the position coordinates of the space coordinate system of each internal corner point on the chessboard need to be initialized. The calibration result generated under the default parameters is the camera internal parameter matrix camera Matrix , The camera's 5 distortion coefficients distCoeffs, and each image will generate its own translation vector and rotation vector. The calibration camera external parameter matrix is used for the conversion between image coordinates and world coordinates; the following formula is the conversion formula between pixel coordinates and world coordinates. The first matrix on the right is the camera internal parameter matrix, and the second matrix is the camera external parameter matrix. The image coordinates are known, and the camera's internal parameter matrix has been obtained through calibration. It is also necessary to calculate the scale factor s and the external parameter matrix.

Among them, s is the scale factor

The conversion formula can be simplified as:

Among them, C is the parameter matrix in the camera, R is the rotation matrix, t is the translation matrix, and Z _const is the height of the world coordinate system, which can be set to 0.

The following formula can be obtained through matrix transformation:

S can be calculated by solving the rotation matrix and the translation matrix.

Set N feature points (N>3), calculate their world coordinates, move the working end of the robotic arm to the feature point, record the end coordinates, and obtain N sets of data;

Use linear algebra to solve the R and t of the two sets of data, where the world coordinates of the feature point are the data of group A, and the end coordinates are the data of group B;

For two point sets A and B, in order to find the rotation matrix R and the translation matrix t between the two point sets. This problem can be modeled as the following formula:

B=R*A+t

Among them, R is the rotation matrix;

Calculate the center point

Point set recentralization

Calculate the covariance matrix between point sets

Calculate the optimal rotation matrix through singular value decomposition

[U,S,V]=SVD(H)

R=VU ^T

Translation matrix

t=-R×μ _A +μ _B

Further, the motor control system is used to adjust the baseline distance between the binocular camera and the structured light projector, including the DC brushless motor control system scheme. Specifically, the DC brushless motor and the control system adopt the double closed loop control of the speed loop and the current loop. system. When starting the motor, the current loop works. By continuously increasing the current, the motor quickly reaches the specified speed, while limiting the maximum starting current to prevent damage to the motor due to excessive current; after the speed is stable, the current loop acts as an inner loop It is mainly used for current-limiting protection, and the speed loop is outside. Its main purpose is to maintain a constant speed. In the case of low power and constant torque requirement, the current loop can keep the motor torque stable through stable current. On the other hand, it can also increase the system's anti-disturbance ability. Figure 3 is the overall block diagram of the DC brushless motor control system.

The brushless DC motor control system includes hardware modules and software modules. The hardware module consists of a development version, a driver board and an emulator. The development chip is controlled by the PC computer. There is a development board chip that outputs the PWM wave waveform. After the signal is amplified by the drive board, it is finally input into the motor to control the start and stop of the motor and the speed and current of the motor. The encoder and AD acquisition circuit in the motor feed back the speed signal and current signal to the computer, and correct the duty cycle of the PWM wave output waveform through feedback to complete the closed-loop control of the motor. Figure 4 is the control block diagram of the hardware module of the brushless motor control system. The software module includes two parts: the setting of chip registers on the development board and the algorithm design. The purpose of setting the chip register of the development board is to control the basic functions such as timing and interrupt of the development board chip, and make it output the PWM waveform; the purpose of the algorithm design is mainly to detect the motor speed according to the encoder and the AD sampling circuit. The feedback signal adjusts the duty cycle of the PWM wave, and then adjusts the size of the speed and the size of the current.

The manipulator 4 adopts a staggered manner of two separate joints, and integrates the two separate joints into a spherical shell. There are two drivers in the spherical shell, one is used to drive the independent joints, and the other is used to drive the light source and the camera. Figure 5 is a cross-sectional view of the spherical joint, where 41 and 42 are the first single hollow joint and the second single hollow joint, respectively, used to place the hollow frameless motor; 43 and 44 are the first driver and the second driver, respectively, 43 drive structure The light projector and the binocular camera 44 are used to drive the robotic arm 4, and the two drivers are exchanged using CAN signals.

A hollow frameless motor is selected for a single hollow joint, which is convenient for internal hollow wiring. The rotor of the motor is directly connected to the harmonic reducer through the central control drive shaft. The end face of the harmonic reducer is used as the direct output end face; the other end of the motor is controlled by the central control The transmission shaft is connected with the gear, and the transmission shaft is offset by gear transmission. The electromagnetic brake and incremental encoder are installed on the offset transmission shaft. This structure enables two independent central control joints to be installed in a staggered manner, making the entire The structure of the spherical joint is more compact.

For a single camera, the visual model of structured light measurement is shown in Figure 6. The local coordinate system O _g -X _g Y _g Z _g and the camera coordinate system O _c -X _c Y _c Z _c are all coincident and are all right-handed coordinate systems. . The origin of the laser is N, and O _c is the center of the camera's optical axis. The exit point N of the structured light plane is on the O _c X _c Z _c coordinate plane, and the structured light plane is orthogonal to the O _c X _c Z _c coordinate plane. The line of intersection is PN, point P is the intersection of the optical axis O _c Z _c and the structured light plane. The distance between the point N of the light plane and the optical center of the camera O _c is the baseline distance of the structured light vision measurement system |NO _c |=D, the angle between the structured light plane and the baseline PNO _c =α, the angle between the optical axis and the laser beam O _c PN=β.

In the model, the coordinate values are unified in the local coordinate system O _g -X _g Y _g Z _g , and the projection model for transforming the object coordinates in the local coordinate system to the image point in the image coordinate system is

In the formula: s = 1, 0 ^T = (0 0 0) ^T , R is a 3X3 unit orthogonal matrix, and t is a 1x3 translation vector. The light plane equation of structured light described by structural parameters with clear physical meaning composed of D, α and β is

From the projection model (1) and the light plane equation (2), the structured light vision measurement mode can be obtained as shown in equation (3):

It can be seen from (3) that the accurate calculation of the coordinate value of the object point is closely related to the baseline distance D.

The camera and light source are driven by a motor to change the baseline distance D to achieve accurate measurement.

Specifically, the edge extraction includes preprocessing methods such as enhancing the edges by using logical operations between images, median filter processing, and image grayscale adjustment enhancement to reduce the complexity of image operations, and then Edge extraction is performed on the basis of canny operator.

Further, calculate the gradient vector everywhere, the image gradient is to find the derivative, and the derivative can reflect the place where the image changes the most, and the place where the image changes the most is the edge of the image.

When the sobel operator cannot clearly reflect the edge of the image, switch to the scharr operator;

Further, non-maximum suppression in the gradient direction refers to finding the local maximum of the pixel point, and setting the gray value corresponding to the non-maximum point to 0, so that a large part of non-edge points can be eliminated. As shown in Figure 3, in order to perform non-maximum value suppression, it is first necessary to determine whether the gray value of pixel C is the largest in its 8-value neighborhood. The direction of the blue line in Figure 3 is the gradient direction of point C, so that it can be determined that its local maximum must be distributed on this line, that is, outside the point C, the intersection of the gradient direction dTmp1 and dTmp2 are two The point value may also be a local maximum. Therefore, judging the gray level of point C and the gray level of these two points can determine whether point C is the local maximum gray point in the neighborhood. If it is judged that the gray value of point C is less than either of these two points, it means that point C is not a local maximum, and then point C can be excluded as an edge. This is how non-maximum suppression works.

Specifically, edge tracing includes: In addition to the edge provided by the canny operator, there is actually a wealth of local gradient information, and the gaps extracted by this scheme are all straight lines, so the gradient information is very valuable. Further, for straightness judgment, after all straight lines have grown, the eigenvalue decomposition is performed on each contour, and the smaller eigenvalue is used for linearity judgment. Furthermore, the line segments are paired, the lengths of the two line segments are similar, the centroids of the two line segments are close, and the gradient vectors of the two line segments are opposite.

Specifically, stereo matching includes: the first step is to perform feature matching on the image, that is, to extract the coordinates of the edge contour points of the gap to obtain the initial matching point; the second step is to calculate the matching cost to measure the pixel to be matched and the candidate pixel The third step is cost aggregation, so that the cost value can accurately reflect the correlation between pixels; the fourth step, the disparity calculation, the optimal view of each pixel is determined by the cost matrix after the cost aggregation Difference.

Specifically, the three-dimensional reconstruction and result analysis include: referring to Figure 8, O ₁ and O _{2 in the} figure are the optical centers of the No. 1 camera and the No. 2 camera. The method of obtaining the three-dimensional geometric information of the object based on two or more images, Assuming an object in space, the left image plane I ₁ and the right image plane I ₂ can be obtained by camera No. 1 and No. 2. The coordinate of a point P on the object in space is [X Y Z] ^T , in the left image plane I ₁ , The projection points on the right image plane I _{2 are P l} and P _r , and their homogeneous coordinates are [u ₁ v ₁ 1] ^T , [u ₂ v ₂ 1] ^T , then P _l and P _r are as follows Correspondence:

in,

M _L and M _R are the projection matrices of the No. 1 camera and the No. 2 camera _{respectively, and A l} and A _r are the internal parameters of the No. 1 camera and the No. 2 camera respectively;

Respectively external parameter matrix Camera 1 and Camera 2, wherein R _l, R _r are the number 1 camera 2 camera rotation matrix, t _l, t _r are the number 1 camera and the No. 2 camera translation vector The internal parameters and external parameters of the camera are obtained by camera calibration. After the projection matrix M _L and M _R are calculated according to the internal and external parameters of the camera, the above formula can be eliminated Z _c1 or Z _c2 , and the information about X, Y, Z can be obtained Four linear equations:

Among them: (u ₁ ,v ₁ ,1) ^T , (u ₂ ,v ₂ ,1) ^T are the homogeneous coordinates of points p _l and p _r in the left image plane I ₁ and the right image plane I _{2, (} X, Y, Z, 1) ^T is the homogeneous coordinates in the world coordinate system to be sought,

Is the element in the i-th row and j-th column of the projection matrix M _{L, similarly,}

Is the element in the i-th row and j-th column of the projection matrix M _R , and the above equations are solved, and the least square solution is the space coordinate sought, that is, the three-dimensional reconstruction of the cover gap is realized.

Specifically, hand-eye calibration includes: first run the Matlab software, add the path where the calibration toolbox is located to the Matlab path environment, and start the main calibration function calib-gui.m. Install the No. 1 camera and the No. 2 camera in the proper position, and adjust the No. 1 camera and the No. 2 camera, change the angle and orientation, shoot about 10 images, and then store them in the computer through the image capture card.

Run the Calibration calibration program, you will get the results of the calibration of the No. 1 camera and the No. 2 camera, and by running the Analyze error program, you can perform error analysis on the calibration results.

Specifically, the motor drive includes: the motor is in the entire control system, and the motor is the driving part of the entire control system, and its selection is of vital importance to the system. Considering the motor's rated torque, speed, and geometric dimensions and other related parameters, Kollmorgen TBM(s)60 series motors were selected. The specific parameters are shown in Table 1-1.

Table 1-1 Motor parameters

The domestic green harmonic reducer is selected, its products meet the work needs and the price is moderate, which can save costs on the basis of ensuring the quality of the developed products. The harmonic reducer also adopts hollow wiring. In order to further ensure the compact structure, the LHD hollow ultra-flattened series of harmonics are selected. After comprehensively considering the final output torque and the allowable torque of the harmonics, the harmonic model is determined to be LHD-17-100. In the selection of gears, in order to ensure that the size of the central control wiring groove is large enough, the diameter of the inner circular hole of the large gear is selected to be 20mm.

On the other hand, in order to make the overall mechanism more compact and lighter, a thin gear with magic number 1 is used. Through the Misumi selection manual, determine the appropriate number of teeth of the large and small gears, and finally select the number of teeth of the large gear Z1=48, the number of teeth of the small gear Z2=20, and the large and small gears and the shaft are connected by flat keys.

The brake device is installed at the end of the biased pinion. The coil of the brake is connected in parallel with the motor. When the motor is powered on, the coil in the brake is energized. When the motor is powered off, the coil in the brake is out of power. The function of the brake is to decelerate the moving parts or mechanical parts in the system. At the same time, it also has a positioning function. After the motor torque is transmitted through the gear, the torque is reduced. While ensuring the braking force, in order to make the structure more compact, choose a smaller size and lighter explosion. After a comprehensive comparison, it is determined to select the KEB01.P1.310 brake. Specific parameters As shown in Table 1-4.

Table 1-2 Brake parameters

The encoder is placed at the end of the offset shaft. It encodes the input signal source or data. After the analog signal is converted by the encoder, it becomes a digital signal that can be communicated, transmitted, stored or processed by the host computer. There are very different working principles. The encoder has two types: incremental and absolute. Increment has high reliability and is far from simple, but it cannot output absolute information. Therefore, once the position of the object is changed after the power is cut off and the movement is stopped, the original position information will be lost. Absolute type can record the absolute position, the movement of the object will not lose the memory information after the power is off, and the anti-interference ability is strong. It is not in the control system, because the encoder does not need to be wired in the air, so there is no need to use a hollow encoder. Considering the output accuracy, the size of the encoder itself, and the installation method, the maxon photoelectric encoder HEDL5540 is determined to be selected. The encoder parameters are shown in Table 1-3.

Table 1-3 Encoder parameter table

The schematic diagram of the structure of the DC brushless motor is shown in Figure 9. The rotor is equipped with permanent magnet steel, and the nails are equipped with windings. The windings are energized in turn according to the read rotor position to generate a rotating magnetic field. Due to the presence of permanent magnets on the rotor, there is the main magnetic field of the rotor's magnetic poles in the air gap. The interaction of the two magnetic fields produces electromagnetic torque. At the same time, Through inertia, the commutation of the motor is completed, so that the motor can continuously rotate, so that the baseline distance D is continuously changed until it is adjusted to the best shooting position.

Specifically, the edge extraction includes: controlling the mechanical arm, placing the gap part of the vehicle door to be tested in the visual space of the binocular stereo vision system, and capturing images by the No. 1 camera and the No. 2 camera to obtain the gap image. Refer to Figure 10 and Figure 11 for the original image taken. Perform a series of preprocessing on the image, including logic operations, median filtering, and grayscale image adjustment.

Use the scharr operator to calculate the gradient vector everywhere, and then perform non-maximum suppression in the gradient direction, but it should be noted that the gradient direction is perpendicular to the edge direction. After the non-maximum suppression is completed, a binary image will be obtained. The gray values of the non-edge points are all 0, and the local gray maximum points that may be edges can be set to 128. The detection result still contains a lot of false edges caused by noise and other reasons. Therefore, further processing is required. Next, perform edge tracing, then straightness judgment, and finally match the line segments.

Specifically, the three-dimensional matching includes: after edge extraction of the door gap line to obtain the coordinates of the edge contour points, first perform feature matching on the obtained coordinate points, and then reduce the matching search space from the two-dimensional space to the two-dimensional space by the epipolar geometric constraint method. Matching in one-dimensional space can greatly reduce the amount of calculation during matching, increase the speed of matching, and save matching time.

After the epipolar line geometric matching, in order to measure the correlation between the pixel to be matched and the candidate pixel, the matching cost needs to be calculated. Whether two pixels are points with the same name or not, the matching cost can be calculated through the matching cost function. The smaller the cost, the greater the correlation and the greater the probability of being a point with the same name.

Cost aggregation. The calculation of matching cost often only considers local information, which is easily affected by image noise. The cost aggregation is to establish the connection between adjacent pixels to optimize the cost matrix. This optimization is often global.

Finally, the disparity calculation is performed. The WTA algorithm is used to calculate the winner-take-all algorithm, that is, among the cost values of all the parallaxes of a certain pixel, the parallax corresponding to the smallest cost value is selected as the optimal parallax.

Three-dimensional reconstruction includes: Refer to Figure 12, from the door gap edge points after stereo matching, combined with the camera calibration results, the coordinates of the door gap edge points in the world coordinate system can be inversely calculated, and then the points are reconstructed by the three-dimensional curve in the Matlab software. .

Refer to Figure 13. From the three-dimensional coordinates of the points on the slit line, obtain the direction vector l(a, b, c) of the slit line, and use the knowledge of spatial analytical geometry to obtain the clamp between l and the X-axis, Y-axis, and Z-axis. The angles are α, β, and γ. To rotate l parallel to the Z axis, you can first rotate l around the X axis by an angle of α, and then rotate around the Y axis by an angle of β.

Refer to Figure 14, after the slit line is rotated, first project the two slit lines in the XZ plane, and calculate the width of the slit according to the results of the projection on the XZ plane; refer to Figure 15, and then set the two slit lines in the door to YZ The plane is projected, and the surface difference of the gap is calculated according to the projection result on the YZ plane.

In summary, the above-mentioned embodiments describe in detail the different configurations of the gap measurement system. Of course, the present invention includes but is not limited to the configurations listed in the above-mentioned embodiments, and any changes are made on the basis of the configurations provided by the above-mentioned embodiments. The content all belong to the protection scope of the present invention. Those skilled in the art can draw inferences based on the content of the above-mentioned embodiments.

The above description is only a description of the preferred embodiments of the present invention and does not limit the scope of the present invention in any way. Any changes or modifications made by a person of ordinary skill in the field of the present invention based on the above disclosure shall fall within the protection scope of the claims.

Claims (9)

1. A gap measurement system, characterized in that the gap measurement system includes a mechanical arm, a host computer, a control cabinet and a vision system, wherein:

   The host computer is connected to the control cabinet to control the mechanical arm;

   The vision system includes a binocular camera and a structured light projector, and both the binocular camera and the structured light projector are fixed to the end of the mechanical arm;

   The mechanical arm moves above the gap, and the structured light projector projects on the gap to form an image of the gap;

   The binocular camera is used to collect the image of the gap, and send the image of the gap to the upper computer, the upper computer performs image processing, and the upper computer obtains the three-dimensional space of the gap according to the result of the image processing Coordinates, and generate a three-dimensional reconstruction model of the gap;

   The space three-dimensional reconstruction model of the rotating slot of the host computer is parallel to the Z axis of the world coordinate system, and the space three-dimensional reconstruction model of the rotating slot is respectively projected to different coordinate planes to obtain the slot width and face difference values.
2. The gap measurement system of claim 1, wherein the gap measurement system further comprises a motor control system, and the motor control system is used to adjust the baseline distance between the binocular camera and the structured light projector.
3. The gap measurement system according to claim 2, wherein the motor control system comprises a DC brushless motor and a DC brushless motor controller, and the DC brushless motor controller adopts double closed-loop control of a speed loop and a current loop;

   When starting the DC brushless motor, the current loop is turned on, and the current increases so that the motor quickly reaches the specified speed, and at the same time the maximum current limit is started;

   After the speed of the brushless DC motor stabilizes, the current loop acts as an inner loop for current limiting protection, and the speed loop acts as an outer loop to maintain a constant speed;

   The brushless DC motor controller includes a hardware module and a software module. The hardware module is composed of a development board, a driver board, an encoder, an AD acquisition circuit and a simulator; the development board outputs a PWM wave output waveform, and the signal is amplified by the driver board, Finally, it is input into the motor to control the start and stop of the motor and the speed and current of the motor; the encoder and AD acquisition circuit feedback the speed signal and current signal to the software module, and correct the duty cycle of the PWM wave output waveform through feedback to complete the closed loop of the motor control;

   The software module sets the chip register of the development board, controls the timing and interrupt function of the development board chip, and makes it output PWM waveform; and according to the encoder to detect the motor speed and the feedback signal of the AD sampling circuit, adjust the PWM wave duty cycle, and then Adjust the size of the speed and the size of the current.
4. The gap measurement system of claim 3, wherein the robotic arm includes a first single hollow joint and a second single hollow joint, and the first single hollow joint and the second single hollow joint are used for placing For a DC brushless motor, the first single hollow joint and the second single hollow joint are integrated in a spherical shell in a staggered manner;

   The spherical housing also has a first driver and a second driver. The first driver is used to drive the structured light projector and the binocular camera, and the second driver is used to drive the first single hollow joint and the The second single hollow joint;

   The DC brushless motor is a hollow frameless motor, the rotor of the hollow frameless motor is directly connected to the harmonic reducer through a centrally controlled transmission shaft, and the end face of the harmonic reducer is used as a direct output end face; The other end of the motor is connected to the gear through the centrally controlled drive shaft, and the drive shaft is biased through the gear drive, and an electromagnetic brake and an incremental encoder are installed on the biased drive shaft.
5. The gap measurement system according to claim 1, wherein the relationship matrix between the end of the robotic arm and the binocular camera is a hand-eye relationship matrix;

   Before the binocular camera collects the image of the gap, calibrate the hand-eye relationship matrix and the camera internal and external parameters of the binocular camera;

   The image of the gap collected by the binocular camera is a structured light image;

   The binocular camera sends the image of the slit to the image processing system of the upper computer;

   The image processing system processes the structured light image collected by the binocular camera, and uses a gap detection algorithm based on the Canny operator to perform edge extraction to obtain more contour information;

   The result of edge extraction is stereo-matched, the coordinates of the slot matching point are obtained, and a spatial three-dimensional reconstruction model is formed, and the width of the slot and the value of the face difference are obtained.
6. The gap measurement system of claim 5, wherein calibrating the hand-eye relationship matrix comprises: setting a world coordinate system on the work plane, and the world coordinate system does not coincide with the robot coordinate system. After the internal and external parameters of the camera are calibrated, the position of the object in the world coordinate system is calculated; the coordinates of the object in the robot coordinate system are obtained, and the hand-eye relationship matrix is calculated according to the conversion between the world coordinate system and the robot coordinate system;

   The edge extraction includes: adopting image preprocessing technology to reduce noise and enhance edge contour, and extend on the basis of Canny operator, so as to perform edge extraction of gap contour;

   The stereo matching includes: now obtaining the coordinates of the edge points according to the result of edge contour extraction, and performing binocular stereo matching on the edge points under the restriction of the edge area, that is, first extracting the point coordinates, then matching cost calculation, then cost aggregation, and finally Disparity calculation;

   Forming the spatial three-dimensional reconstruction model includes: obtaining the left and right projection images through a binocular camera, obtaining the corresponding relationship between the left and right projection images through stereo matching, and obtaining the gap by using the calibration result of the binocular camera, that is, the coordinates of the edge contour points in the left and right projection images. The three-dimensional coordinates of the edge in space, and the three-dimensional reconstruction of the slit line according to the three-dimensional coordinates of the slit contour, and the method of rotating the slit surface to be parallel to a certain coordinate plane to obtain the slit width and the surface difference value.
7. The gap measurement system according to claim 6, wherein the local coordinate system O _g -X _g Y _g Z _g of a single camera in the binocular camera and the camera coordinate system O _c -X _c Y _c Z _c have both coordinates. Coincident and both are right-handed coordinate systems;

   The origin of light emitted by the structured light projector is N, O _c is the center of the optical axis of the binocular camera; the emission point N of the structured light plane is on the O _c X _c Z _c coordinate plane, and the structured light plane is orthogonal to O _c X _c Z _c coordinate plane; the intersection line is PN, point P is the intersection point of the optical axis O _c Z _c and the structured light plane, the distance between the exit point N of the light plane and the optical center O _{c of the} camera, that is, the structured light vision measurement The baseline distance of the system |NO _c | = D, the angle between the structured light plane and the baseline PNO _c = α, the angle between the optical axis and the laser ray O _c PN = β;

   In the model, the coordinate values are unified in the local coordinate system O _g -X _g Y _g Z _g , and the projection model for transforming the object coordinates in the local coordinate system to the image point in the image coordinate system is

   

   In the formula: s = 1, 0 ^T = (0 0 0) ^T , R is a 3X3 unit orthogonal matrix, t is a 1x3 translation vector; it is described by structural parameters with clear physical meaning composed of D, α and β The light plane equation of structured light is

   

   From the projection model (1) and the light plane equation (2), the structured light vision measurement mode is obtained as shown in equation (3):

   

   It can be seen from (3) that the accurate determination of the coordinate value of the object point is closely related to the baseline distance D; the camera and light source are driven by a motor to change the baseline distance D to achieve accurate measurement.
8. The gap measurement system according to claim 6, wherein the method for obtaining three-dimensional geometric information of an object based on two or more images, assuming that there is an object in space, the left image plane is obtained by the No. 1 camera and the No. 2 camera I ₁ , the right image plane I ₂ , the coordinates of a point P on the object in space are [X Y Z] ^T , and the projection points on the left image plane I ₁ and the right image plane I _{2 are P l} and P _r , respectively. The homogeneous coordinates are [u ₁ v ₁ 1] ^T and [u ₂ v ₂ 1] ^{T respectively} , then P _l and P _r have the following corresponding relations:

   

   

   in,

   

   

   M _L and M _R are the projection matrices of the No. 1 camera and the No. 2 camera _{respectively, and A l} and A _r are the internal parameters of the No. 1 camera and the No. 2 camera respectively.

   

   Respectively external parameter matrix Camera 1 and Camera 2, wherein R _l, R _r are the number 1 camera 2 camera rotation matrix, t _l, t _r are the number 1 camera and the No. 2 camera translation vector , The internal parameters and external parameters of the camera are obtained through camera calibration. When we obtain the projection matrix M _L and M _R according to the internal and external parameters of the camera, we can eliminate Z _c1 or Z _c2 from the above equation to obtain the information about X, Y, and Z Four linear equations:

   

   

   

   

   Among them: (u ₁ ,v ₁ ,1) ^T , (u ₂ ,v ₂ ,1) ^T are the homogeneous coordinates of points p _l and p _r in the left image plane I ₁ and the right image plane I _{2, (} X, Y, Z, 1) ^T is the homogeneous coordinates in the world coordinate system to be sought,

   

   Is the element in the i-th row and j-th column of the projection matrix M _{L, similarly,}

   

   Is the element in the i-th row and j-th column of the projection matrix M _R , and the above equations are solved, and the least square solution is the space coordinate sought, that is, the three-dimensional reconstruction of the cover gap is realized.
9. A method for measuring a gap, characterized in that the method for measuring a gap includes:

   The upper computer controls the robotic arm by connecting to the control cabinet;

   The binocular camera and structured light projector of the vision system are both fixed to the end of the robotic arm;

   The mechanical arm moves above the gap, and the structured light projector projects on the gap to form an image of the gap;

   The binocular camera is used to collect the image of the gap, and send the image of the gap to the upper computer, the upper computer performs image processing, and the upper computer obtains the three-dimensional space of the gap according to the result of the image processing Coordinates, and generate a three-dimensional reconstruction model of the gap;

   The space three-dimensional reconstruction model of the rotating slot of the host computer is parallel to the Z axis of the world coordinate system, and the space three-dimensional reconstruction model of the rotating slot is respectively projected to different coordinate planes to obtain the slot width and face difference values.

Images