KR20090092150A - Method of 3d inspection of the object using ccd camera laser beam and apparutus thereof - Google Patents

Abstract

A method and a device for 3D location information of a target object using a CCD camera and a laser beam are provided to measure the 3D location information of the target and get the measured data rapidly. A method for 3D location information of a target object using a CCD camera and a laser beam comprises followings. The two-dimensional video of the target object is obtained using the CCD camera for the two-dimensional video(S100). A two-dimension coordinate value toward a metering point of the target object is produced(S106). The camera calibration is performed and the intrinsic factor showing the property of the charge coupled device camera for the laser measurement is produced(S108). The slit laser beam is irradiated in the measurement object body. The beam image is obtained by the charge coupled device camera for the laser measurement(S112). A plane equation and the normal vector of a plane, in which the target object is put, are produced from the obtained beam image(S120). The two-dimension coordinate value about the metering point is converted into 3D coordinate using the intrinsic factor, the plane equation and normal vector(S124).

Description

Method and apparatus for measuring 3D position information of an object using a CD camera and a laser beam {Method of 3D Inspection of the object using CCD camera laser beam and Apparutus

The present invention relates to a laser vision system, and more particularly, using a CD camera and a laser beam capable of calculating three-dimensional position information of an object to be measured using two laser slit beam generators and two CD cameras. A method and apparatus for measuring three-dimensional position information of an object are provided.

Laser Vision System is a method of detecting and measuring the necessary information using non-contact sensors such as laser and CD cameras, not by contact and manpower, but by measuring the three-dimensional information of objects that are currently being produced at the production site. Say the system. Furthermore, vision laser systems are being introduced in working environments that cannot be measured by manpower, increasing product reliability and productivity.

In addition, the factory automation to improve the productivity can produce a good product at a higher speed and has the advantage of effectively maintaining and repairing the system of the production line. Such factory automation is accelerating the development of automation lines and the introduction of more efficient automation systems.

There are two methods of measuring information of a three-dimensional object, a contact type and a non-contact type. Although the contact method guarantees precision, it takes a lot of time to measure and also requires expensive equipment. The non-contact method includes a point beam projection method, a slit beam projection method, a moire method, a three-dimensional shape measurement method using PMP, and the like.

Since the point light method can measure only the height of one point at a time, when measuring the curved shape, it is necessary to move the sensor and simultaneously read the three-dimensional position coordinates of the sensor. The slit light method is a measurement method based on the principle of the optical triangulation method, and after condensing the laser light, the slit light is made by using a rod lens or a cylindrical lens and projected onto the measurement object to obtain the slit light transformed according to the shape. The slit optical image data obtained by the CD camera is a method of calculating three-dimensional coordinates of a shape through geometric calculations. In the moire method, a regular striped grid is placed directly in front of the measurement object, and when light is emitted from one side, a wavy pattern of shadow is formed on the measurement object. This pattern has the shape information of the object and is analyzed to obtain the height value. The PMP method (Phase Measurement Shape Measurement) is a measurement method that greatly simplifies the optical system while ensuring excellent measurement accuracy by using the phase shifting method used in projection type moire as the main method. However, despite its excellent measurement accuracy, it is rarely used because it is impossible to implement in the industrial field.

An object of the present invention is to provide a method for measuring a laser vision sensor using a slit light that can be obtained automatically and suitable for an industrial site, and to automatically obtain a processing result. do.

In addition, the present invention can measure the data of the car body and parts for a variety of items, it is possible to obtain the measurement data quickly, high accuracy, high-speed laser measurement module and three-dimensional to bring more accurate measurement results Another object is to provide a method for measuring location information.

In order to achieve the above object, according to a preferred embodiment of the present invention, (a) using a two-dimensional image CD camera for obtaining a two-dimensional image of the object, the two-dimensional ( x, y) calculating coordinate values; (b) calculating and calibrating an internal factor representing a characteristic of the laser measuring CD camera installed at a predetermined angle with respect to the laser slit beams irradiated from the two laser slit beam generators; (c) irradiating two slit laser beams to an object to be measured using two laser slit beam generators and acquiring a beam image using the laser measuring CD camera; (d) calculating a plane equation and a normal vector of the plane on which the object is placed from the obtained beam image; And (e) converting two-dimensional coordinate values of the measurement point into three-dimensional coordinate values using the internal factors, the plane equation, and the normal vector. A three-dimensional position information measuring method is provided.

More preferably, the step (a) may include (a1) extracting an image of an object by applying a morphology technique to the input two-dimensional image; (a2) binarizing an image of the extracted object; And (a3) calculating a center coordinate (measurement point) of the object by applying a cross-sectional primary moment to the binarized image.

In addition, the step (d) described above, (d1) binarizing the beam image; (d2) thinning the binarized beam image; (d3) calculating an image plane linear equation for light sources 1 and 2 by applying a Hough Transform algorithm to the thinned beam image; And (d4) the image plane linear equation of light source 1, the image plane linear equation of light source 2, the slit plane equation of light source 1, the slit plane equation of light source 2, the image plane plane equation of light source 1, the image plane plane of light source 2 Equation, the plane equation of the object plane may be configured to include the step of calculating the plane equation and the normal vector of the object plane, wherein step (d4) described above is a straight line between the slit plane, the image plane and the object plane It can be configured to calculate the plane equation and normal vector of the object surface using

In addition, the step (e) described above, (e1) calculating the angle rotated on each axis by orthogonal projection of the calculated normal vector to the XZ plane and YZ plane in the CD camera coordinate system; (e2) converting two-dimensional (x, y) coordinate values for the measurement point into three-dimensional (X, Y) coordinate values using the calculated rotation angle and the internal factor; And (e3) calculating the three-dimensional Z coordinate value by substituting the converted three-dimensional (X, Y) coordinate values into the calculated plane equation of the object plane.

Here, the three-dimensional (X, Y) coordinate value of the step (e2) described above is calculated by applying two-dimensional (x, y) coordinate values independently to the conversion algorithm, the coefficient of x or y of the normal vector is negative The three-dimensional (X or Y) coordinate value is or Is calculated as

If the coefficient of x or y of the normal vector is positive, the three-dimensional (X or Y) coordinate value is or Θ ₂ is the rotation angle between the image plane of the CD camera and the object plane (XZ plane or YZ plane), and X 'or Y' is the space of x or y when there is no rotation angle θ _2. It may be characterized in that the coordinate value.

In addition, according to another preferred embodiment of the present invention, in the three-dimensional position information measuring device of the object using the CD camera and the laser beam according to the present invention, the two-dimensional image of the two-dimensional image of the measurement object to output CD camera for imaging; A laser slit beam generator for a first light source installed on the two-dimensional image CD camera at a predetermined distance on a horizontal line to irradiate a slit beam to an object to be measured; A laser slit beam generator for a second light source installed at a predetermined distance on a horizontal line of the two-dimensional image CD camera to irradiate a slit beam to a measurement target object; A CD camera for laser measurement, which is installed to have a predetermined angle with respect to the laser slit beam emitted from the laser slit beam generator and outputs a beam image of an object to which the slit beam is irradiated; Analyzing the two-dimensional image output from the two-dimensional image CD camera for calculating the two-dimensional center coordinates of the object to be measured, and performing the camera calibration for the laser measurement CD camera to calculate the internal factor, Analyze the beam image output from the CD camera for laser measurement to calculate the plane equation and the normal vector of the object plane, and use the internal factors and the plane equation and the normal vector of the object plane to determine the two-dimensional center coordinates of the object to be measured. A location information measuring unit converting the coordinates into three-dimensional coordinates and outputting the converted coordinates; And a control unit configured to control the two-dimensional image CD camera, the laser slit beam generator, the laser measurement CD camera, and the position information measuring unit to perform three-dimensional position information measurement of an object. An apparatus for measuring three-dimensional position information of an object using a digital camera and a laser beam is provided.

In this case, more preferably, the above-described position information measuring unit extracts an image of an object to be measured by applying a morphology technique to a two-dimensional image output from the two-dimensional image CD camera, and binarizes the extracted object image. A center coordinate calculation module for calculating a center coordinate (measurement point) of the object by applying a cross-sectional primary moment to the binarized image; A camera calibration module for calculating an internal factor representing a characteristic of the CD camera for laser measurement; The beam image output from the laser measuring CD camera is binarized / thinned to calculate an image plane linear equation for the light source 1 and the light source 2, an image plane linear equation for the light source 1, an image plane linear equation for the light source 2, Using the slit plane equation of light source 1, the slit plane equation of light source 2, the image plane plane equation of light source 1, the image plane plane equation of light source 2, and the plane equation of object plane, calculate the plane equation and normal vector of the object plane. A plane equation / normal vector calculation module; And orthogonally project the calculated normal vectors of the object plane to the XZ plane and the YZ plane in the CD camera coordinate system to calculate an angle rotated on each axis, and using the calculated rotation angle and the internal factor, two-dimensional (x) y) After converting the coordinate values to three-dimensional (X, Y) coordinate values, the three-dimensional Z coordinate values are calculated by substituting the converted three-dimensional (X, Y) coordinate values into the plane equation of the calculated object plane. It may be configured to include a coordinate transformation module.

As described above, according to the method for measuring three-dimensional position information of an object using a CD camera and a laser beam according to the present invention, there is an advantage that a slit light measuring method that is suitable for an industrial site and automatically obtains processing results can be used. .

In addition, according to the present invention by replacing the conventional measurement of the data of the vehicle body and parts with a contact sensor or a sampling method of the manpower by a laser vision sensor measuring method is possible to respond to a variety of items, measurement The advantage is that data can be obtained quickly and more accurate measurement results can be obtained.

In addition, according to the present invention, it is possible to precisely measure the length, distance, thickness and outer diameter of each measurement item, that is, the base material, to eliminate defects in automobile body and parts production, and furthermore, It has the advantage of improving stability.

1 is a flowchart illustrating a process of measuring three-dimensional position information of an object according to an embodiment of the present invention.

Figure 2a is a structural diagram of a measurement system using a three-dimensional position information measuring device of the object using a CD camera and a laser beam according to an embodiment of the present invention.

Figure 2b is a block diagram of a three-dimensional position information measuring device of the object using a CD camera and a laser beam according to an embodiment of the present invention.

3A to 3D are conceptual views for explaining the cross-sectional primary moment.

4A to 4E are conceptual diagrams for describing the Hough Transform.

5 is an exemplary view showing perspective projection of a one-dimensional object.

6A to 6D are diagrams showing the geometrical configuration of the XZ (or YZ) plane of the object plane and the image plane.

Figure 7 is a block diagram showing the geometrical configuration of the three-dimensional position information measuring device of the object using a CD camera and a laser beam according to an embodiment of the present invention.

8A is a conceptual diagram illustrating a coordinate system setting in a CD camera image plane.

8B is a conceptual diagram for explaining a pinhole camera model.

9A to 9B are exemplary views of beam images obtained by using a CD camera for laser measurement.

FIG. 10 is a diagram showing a moving error value according to the amount of movement on the x and y axes and an average value at the moved position; FIG.

11A to 11B are graphs showing movement errors occurring when moving to the x and y axes.

Explanation of symbols on the main parts of the drawings

100: αβ rotation stage 102: xyz stage

103: height gauge 110: CD camera for two-dimensional video

120: laser slit beam generator for the first light source

122: laser slit beam generator for the second light source

130: CD camera for laser measurement

220: location information measuring unit 222: central coordinate calculation module

224: camera calibration module 226: plane equation / normal vector calculation module

228: coordinate transformation module

The main contents pursued by the present invention are to design a laser module for three-dimensional data acquisition, and to invent a two- and three-dimensional data acquisition method, a reconstruction and a high-speed analysis method of the vehicle body and parts using the same. In this method, the present invention proposes an algorithm for measuring 3D coordinate values of a point (center point of a target) in space, which is a basic step for obtaining the area of a hole and the spacing of a slot in an automobile body.

In order to construct a 3D position measurement system using a laser sensor, the present invention first acquires a 2D image of an object using a CD camera, and then separates the desired object from the object through a preprocessing process and a binarization process, and then centers the object. Get the 2D coordinates of. Then, the laser diode of the slit is shot on the object and the plane image and the normal vector of the object are calculated using the beam image received from the CD camera, and the two-dimensional position information is three-dimensional by using the plane equation and the normal vector. The location information is converted.

The three-dimensional position information thus converted is utilized and is the basis for other-purpose measurements (such as the hole area and slot spacing described above).

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the method and apparatus for measuring 3D position information of an object using a CD camera and a laser beam.

1 is a flowchart illustrating a process of measuring three-dimensional position information of an object according to an exemplary embodiment of the present invention. As shown in FIG. 1, in the method for measuring three-dimensional position information of an object using a CD camera and a laser beam according to the present invention, a two-dimensional image of an object is obtained by using a two-dimensional image CD camera 110. (S100), extracting an image of an object by applying a morphology technique to the input two-dimensional image (S102), binarizing an image of the extracted object (S104), and applying a cross-sectional first moment to the binarized image. By calculating the center coordinates (measurement point) of the object (S106) to obtain two-dimensional information.

Next, the three-dimensional position information measuring method according to the present invention is characterized in that the CD camera 130 for laser measurement installed so as to have a constant angle with respect to the laser slit beam irradiated from the two laser slit beam generators (120, 122) Computing an internal factor indicating a by performing camera calibration (S108), irradiating two slit laser beams to the measurement target object using the two laser slit beam generators (120, 122) (S110), for laser measurement Acquiring a beam image using the CD camera 130 (S112), binarizing the obtained beam image (S114), thinning the binarized beam image (S116), and hoping on the thinned beam image. By applying a Hough Transform algorithm (S118) to calculate the image plane linear equation for the light source 1, the light source 2 to obtain the three-dimensional information.

Next, the three-dimensional position information measurement method according to the present invention is a linear equation of the image plane of the light source 1, a linear equation of the image plane of the light source 2, a slit plane equation of the light source 1, a slit plane equation of the light source 2, The plane equation of the object plane and the normal vector are calculated using the plane equation of the image plane, the plane equation of the plane of light source 2, the plane equation of the object plane and the slit plane, the image plane and the plane of the object in a straight line (S120).

Finally, the method for measuring three-dimensional position information according to the present invention rotates the normal vectors calculated using the obtained two-dimensional information and three-dimensional information on the XZ plane and the YZ plane in the CD camera coordinate system and rotates on each axis. Calculating a calculated angle (S122), converting a two-dimensional (x, y) coordinate value for a measurement point into a three-dimensional (X, Y) coordinate value using the calculated rotation angle and the internal factor (S124), Substituting the converted three-dimensional (X, Y) coordinate values into the calculated plane equation of the object plane to calculate a three-dimensional Z coordinate value (S126) is performed to calculate the three-dimensional coordinates of the center point of the object to be measured.

2B is a block diagram illustrating an apparatus for measuring 3D position information of an object using a CD camera and a laser beam according to an exemplary embodiment of the present invention. As shown in FIG. 2B, an apparatus for measuring 3D position information of an object using a CD camera and a laser beam according to an exemplary embodiment of the present invention is for a 2D image photographing and outputting a 2D image of an object to be measured. The laser camera slit beam generator 120 for the first light source, which is installed at the CD camera 110 and the CD camera 110 for two-dimensional image at a constant distance on a horizontal line to irradiate the slit beam to the measurement object, two-dimensional Irradiation from the laser slit beam generator 122 and the laser slit beam generators 120 and 122 for the second light source installed at a constant distance on the horizontal line of the image CD camera 110 to irradiate the slit beam to the measurement object. It is installed to have a certain angle with respect to the laser slit beam is a laser measurement CD camera 130 for recording and outputting the beam image of the object to which the slit beam is irradiated, CD camera for 2D image Analyze the two-dimensional image output from the 110 to calculate the two-dimensional center coordinates of the object to be measured, calculate the internal factors by performing a camera calibration for the laser camera CD camera 130, the seed for laser measurement The plane image and normal vector of the object plane are calculated by analyzing the beam image output from the camera 130, and the two-dimensional center coordinate of the object to be measured is three-dimensional by using the internal equations and the plane equation and normal vector of the object plane. Position information measuring unit 220 and the two-dimensional image CD camera 110, laser slit beam generator 120, 122, the laser measurement CD camera 130, the position information measuring unit It may include a control unit 210 to control the 220 to perform the three-dimensional position information measurement of the object.

At this time, more preferably, the above-described position information measuring unit 220 extracts an image of an object to be measured by applying a morphology technique to a two-dimensional image output from the two-dimensional image CD camera 110 and extracts the extracted object. An internal factor representing the characteristics of the center coordinate calculation module 222 and the laser camera CD camera 130 for binarizing the image of the object and applying a cross-sectional primary moment to the binarized image to calculate a center coordinate (measurement point) of the object. A beam calibration output from the camera calibration module 224 and the CD-based camera 130 for laser measurement to calculate the image plane linear equations for the light source 1 and the light source 2 is calculated. Plane linear equation, image plane linear equation of light source 2, slit plane equation of light source 1, slit plane equation of light source 2, image plane equation of light source 1, image plane equation of light source 2, plane of object plane Plane equation / normal vector calculation module 226 for calculating plane equations and normal vectors of object planes using equations, and projecting the normal vectors of the calculated object planes onto the XZ plane and YZ plane in the CD camera coordinate system Calculates the rotated angle, and converts the two-dimensional (x, y) coordinate values for the measurement points into three-dimensional (X, Y) coordinate values using the calculated rotation angle and the internal factor, and then converts the converted three-dimensional ( It may include a coordinate conversion module 228 to calculate the three-dimensional Z coordinate value by substituting the X, Y) coordinate value to the calculated plane equation of the object plane.

In the following, two-dimensional information and three-dimensional information acquisition, two-dimensional and three-dimensional coordinate transformation algorithm of the three-dimensional position measurement method and apparatus of the object according to the present invention configured as described above with reference to the attached 3 to 11 and Plane equations of objects, normal vector acquisition algorithms and experimental procedures will be explained in detail.

1. 2D Information Acquisition Process-Calculation of Center Coordinates of Objects

(1-1) Image Preprocessing

When processing gray images, Morphology should be applied to obtain clearer images.

The morphology technique is a morphological approach to extract only the desired part of the image by reflecting the information of the target object that knows the geometric shape in advance. Morphology techniques include erosion and dilation operations, and opening and closing operations that combine the two previous operations. Since a technique of specifying / extracting an image of an object to be measured from a 2D image photographed using a morphology technique adopts a known technique, a detailed description thereof will be omitted.

(1-2) Image Binarization Process

Because of the intuitive features and simplification of images, the binarization of images is used in the area of image segmentation. In the present invention, image binarization was used as a method of distinguishing a target to be measured from an object.

The average intensity of the neighbors around the point (x, y) is also binarized image g (x, y) is defined by Equation 1 as follows.

Where T is the threshold value. That is, the pixels indicated by 1 correspond to the object and the pixels indicated by 0 correspond to the background.

This binarization is global if T only depends on f (x, y) (contrast) and if T depends on both f (x, y) and g (x, y), It is called local. Also, if T also depends on the spatial coordinates x and y, then this binarization is called dynamic or adaptive. In the present invention, a basic global binarization method is used.

The following simple algorithm can be used to get the threshold value (T) automatically.

Step 1. Select the initial value of T arbitrarily.

Step 2. Use T to split the image into two groups.

G ₁ is composed of all pixels whose brightness value is greater than T, and G ₂ is composed of all pixels whose brightness value is less than or equal to T.

Step 3. Find the mean values μ ₁ and μ ₂ of the intensity of pixels in each G ₁ and G ₂ region.

Step 4. Calculate the new T.

Step 5. Repeat steps 2 to 4 until the difference in T is less than the predetermined variable T ₀ in the continuous iteration.

(1-3) Center Point Extraction Process

In the present invention, the cross-sectional primary moment is used to obtain the center coordinate of the object to be measured. The cross section primary moment is used to examine the center of the figure and the stress estimation and stability of the structure.

Referring to FIG. 3A, when the sum is obtained for the shear plane by multiplying the small area d _A with the cross section and the distance d _A from the X axis and the distance x from the Y axis to d _A , This is called a cross-sectional first moment (Starical moment) with respect to the X-axis and the Y-axis, and is denoted by the symbols G _X and G _Y , and the unit is a cube of length unit (m 3, cm 3).

Alternatively, as shown in FIG. 3B, when the micro area is subdivided into d _A1 , d _A1 , ..........., d _An , the distance from the axis to each micro area is y ₁ , y ₂ , If _n ... and x ₁ , x ₂ , ...., x _n may be defined as Equation 4 below.

※ Extract Center of Shape

Referring to FIG. 3C, the point at which the cross-sectional primary moments for all axes in any direction passing through any one point becomes zero is referred to as a center of figure. There is only one center in the figure, and the center is obtained by dividing the cross section by the area of the figure with respect to arbitrary X and Y axes.

In Equation 5, A is the area of the figure. In addition, as shown in several cross-section _{A 1, A 2, A 3} , ......, equation (6) the aggregate of the cross-section consisting of A _n divided by the total area, obtain a first cross-sectional moment of inertia about the front end as shown in Figure 3d Find the location of the center.

2. 3D Information Acquisition Process

The process performed in 3D information acquisition is as follows.

The method and apparatus for measuring 3D position information of an object using a CD camera and a laser beam according to the present invention are obtained by obtaining a laser beam image-> binarization-> thinning-> Hough Transform-> plane equation and normal vector It is configured to acquire three-dimensional information of the object by performing the extraction process.

When the two laser beams irradiated from the laser slit beam generators 120 and 122 are illuminated on the measurement object, the CD camera 130 for laser measurement receives the image thereof. The image is subjected to the binarization process described above, and then thinned to analyze the beam image.

The thinned laser beam is represented as a series of linear equations through Hough Transform, and the planar equation and normal vector of the object are acquired from the linear equations thus obtained.

(2-1) Thinning Process

The basic concept of thinning is to peel off the thick line one layer from the outermost layer and extract the last remaining line component. That is to extract the center line. This is applied in the present invention to make the thick image of the laser beam into a straight line condensed on one pixel. Thinning does not change the connectivity of the original figure, but it is important to maintain the original shape of the line.

The thinning of the set A by the structural element B is denoted by AⓧB and may be defined by a column of structural elements as shown in Equation 7 below.

This process is one pass through B ¹ to thin A, then the result is passed through B ² once to thin, and this process is passed through B ⁿ once to thin A. Continues. This whole process is repeated until no further changes occur.

(2-2) Straight Line Extraction Algorithm Using Hough Transform

Hough Transform is a good way to detect the contours, curves or lines of an image if it can be represented as a parameter.

Hough Transform I

For points (x _i , y _i ) and (x _j , y _j ) on an equation y _i = ax _i + b of a straight line, write the equation of the line as b = -x _i a + y _i Variable space), we meet at one point (a ', b'). Where a 'and b' are the slope and intercept of the line connecting (x _i , y _i ) and (x _j , y _j ) in the xy plane. Again, all points on the straight line equation y _i = ax _i + b pass (a ', b') on the ab plane. This concept is shown in Figures 4a (XY plane) and 4b (ab plane (variable space)).

Hough Transform adds the concept of an accumulator cell to this concept. The cell coordinate (i, j) corresponds to a rectangle having coordinates (a, b) in variable space and has a cumulative value A (i, j). Then, for every point (x _k , y _k ) in the image, solve for the corresponding b using the equation y _k = ax _k + b, which in turn is the closest to each value allowed for the b-axis. Find it. And the corresponding value of A (i, j) is accumulated by one. When this process is complete for all points in the image, the value of Q in A (i, j) means Q points on a line y = a _i x + b _j in the xy plane. The number of ab planes divided means the accuracy of these points on the same straight line. The ab plane is broken down through the above-described process is shown in Figure 4c. However, this algorithm has a problem in implementation because the closer the straight line is to the X axis or the Y axis, the closer the parameters a and b are to infinity.

Hough Transform II

In the Hough Transform I above, a second algorithm is used to solve the problem of the parameters a and b approaching infinity when the straight line is parallel to the Y axis.

If a straight line is expressed as a parameter of angle θ and distance ρ, it can be expressed as Equation 8, and θ and ρ as parameters instead of a and b in the same manner as in Hough Transform I. It is applied to the cumulative cell concept by mapping to θρ plane. This concept is illustrated in FIG. 4D (normal representation of a straight line) and FIG. 4E (subdivision of ρθ plane).

(2-3) Camera Calibration Process

Camera calibration, which should be basically performed before extracting the information, applies an algorithm from Zhengyou Zhang's “Camera Calibration with One-Dimensional Object.” When using this algorithm, the camera calibration is performed with a one-dimensional object. Calibration is possible.

◎ Basic notation

The 2D point on the CD camera is represented by m = [u, v] ^T , and the 3D point in space is represented by M = [X, Y, Z] ^T. The extension vector is defined as in Equation 9 below.

Wow

The point M in space and the point m on the CD camera are perspective projections, and the following equation 10 is established.

with

Where s is a scale factor and R and t of [R t] are a rotation matrix and a translation matrix indicating the relationship between the camera coordinate system and the base coordinate system.

And Is the internal matrix of the camera and has the following arguments.

(u ₀ , v ₀ ): Coordinate value of key point

(α, β): Scale factor on axis and axis on image

: Warp value between two axes on the image

The factor defined above is defined as an internal factor, and the purpose of camera calibration is to determine these five internal factors.

And, It should be written as.

◎ Camera Calibration using 1D Object

◇ Basic expression

Consider a one-dimensional object in space consisting of three points and a fixed point. Make both ends A and B, and see point A as a fixed point. Suppose you move point B with respect to fixed point A. The length of the object, that is, the distance between the points AB is obtained as shown in Equation 11 below.

And the position of the point C is given by the following equation (12).

That is, the point C is a point existing between the two end points A and B. Where λ _A and λ _B are the scale factors of points A and B and are known. If point C is the midpoint of AB, then λ _A = λ _B = 0.5.

 FIG. 5 is a diagram illustrating perspective projection of a one-dimensional object, and as shown in FIG. 5, the relationship between points A, B, and C in space and points a, b and c in the image is expressed by Equation 13 below.

Z _A , Z _B , and Z _C are values indicating the distance (depth) of the points A, B, and C to the origin O, and are unknown.

Substituting Equation 13 above into Equation 11 and changing both sides Equation 14 is obtained by canceling.

And The cross product of both sides is given by Equation 15 below.

If Equation 15 is summarized as Z _B , the following Equation 16 is obtained.

In addition, when Equation 13 is applied to Equation 11, Equation 17 is obtained.

Substituting the Z _B value derived from Equation 16 into Equation 17, Equation 18 is obtained.

And the result of expressing the equation (18) by the square on both sides is the same as the following equation (19).

here to be.

Here, h can be obtained because λ _A and λ _B are known.

But in equation (19) Z _A and It has five arguments of, a total of six unknown values. Therefore, camera calibration requires at least six images of one-dimensional objects.

◇ Closed-Form Solution

In equation (19) Let be defined by the following equation (20).

Since matrix B is a symmetric matrix, it can be expressed as a 6D vector as shown in Equation 21 below.

If h = [h ₁ , h ₂ , h ₃ ] ^T , x = Z ² _A b, Equation 19 can be arranged as Equation 21 below.

At this time, to be

If N images of the one-dimensional object are obtained, Equation 21 is expressed as Equation 22 below.

One

At this time, And 1 = to be.

Equation 22 may be expressed by Equation 23 by a least-squares solution.

One

We can find all the unknown factors in x = Z ² _A b.

here, Solving as, it is possible to extract only the internal factor and Z _A value as in Equation 24 below.

Z _B can be obtained by substituting the above result into Equation 16, and the points A, B, and C can be obtained by substituting Equation 13 as well.

In the present invention, by performing the camera calibration process as described above to calculate the internal factors according to the camera characteristics of the CD camera 130 for laser measurement.

3. 2D-3D coordinate transformation algorithm

This algorithm converts the coordinates on the CD camera to spatial coordinates using the plane equation of the plane on which the target is placed. In the present invention, it is used to convert the planar center coordinates of the target into coordinates in space.

The basic focus is on obtaining a normal vector from the plane equation of the target and orthogonally projecting the normal vector to the XZ plane and the YZ plane in the CD camera coordinate system to obtain the rotated angle in each axis. Therefore, each spatial coordinate is obtained by applying coordinates x and y on the CD camera independently to the algorithm.

The planar equation in space is derived as follows. Past a point A (x ₁ , y ₁ , z ₁ ) Summarizing this expression in the plane a (xx ₁ ) + b (yy ₁ ) + c (zz ₁ ) = 0 perpendicular to, we get ax + by + cz- (ax ₁ + by ₁ + cz ₁ ) = 0, In this case,-(ax ₁ + by ₁ + cz ₁ ) is constant, and if d is set to d, the general form of the equation of the plane can be obtained.

And above Is called the normal vector.

In the present invention, a plane equation is first obtained, and a normal vector is obtained from the equation and applied to the present algorithm.

The inclination value (θ) with the camera plane obtained by projecting the normal vector to the other axis with respect to the optical axis (Z axis) of the camera, that is, the XZ axis and the YZ axis is parallel to the plane as shown in FIG. 6A or 6C. Obtained. The obtained slope can be divided into two cases as shown in the figure. The present invention proposes a coordinate transformation algorithm in consideration of the two cases. The following description is based on the XZ plane to obtain X (X coordinate of the 3D center coordinate of the actual measurement object), and the algorithm for obtaining Y (Y coordinate of the 3D center coordinate of the actual measurement object) is the same on the YZ plane. Since the calculation is performed using an algorithm, description thereof will be omitted.

(3-1) Negative coefficients (a or b) of normal vectors are negative

FIG. 6A is a picture projected in the positive direction of the XZ plane when a (or b) of the normal vector has a negative number.

here,

f: focal length

h: vertical distance from the origin to the object surface without rotation

X _h or Y _h : 1/2 of the area visible to the camera at height

XC or YC: coordinates of the CD camera of x or y

X 'or Y': Spatial coordinate value of x or y without rotation angle θ ₂

X or Y: Spatial coordinate value of x or y when rotation angle θ ₂ is present

to be. The value of X '(or Y') is obtained as in Equation 26 below.

In this case, only the parts shown in bold in order to obtain X (or Y) are expressed as shown in FIG. 6B (the actual spatial coordinate values of X).

Where X is the value we actually get. X 'is a spatial coordinate value when θ ₂ = 0, and θ ₂ represents an inclination degree between an object and an image plane.

Therefore, X to be obtained is simply derived as in Equation 27 below.

(3-2) If the coefficients (a or b) of the normal vectors are positive

6C shows a case in which the a (or b) value of the normal vector has a positive value and is projected in the positive direction of the XZ plane.

 In the above figure, if the enlarged triangle part is enlarged to obtain X (or Y), it may be represented as shown in FIG. 6D.

It can be seen that α = θ ₁ and β = 90 ° -θ ₂ . And to derive the equation to find the final X is the following equation (28).

6C may be expressed by Equation 29.

Therefore, X to be obtained is finally obtained as shown in Equation 39 below.

Applying the above algorithm to the coordinates x and y on the camera, we can obtain the spatial coordinates X and Y regardless of how the normal vector exists in space. The Z coordinate in space can be obtained by substituting the transformed values of X and Y into the plane equation of the object.

Therefore, by applying this algorithm, we can convert the 2D coordinates on the camera into 3D coordinates in space using the information we know, that is, the plane equation of the object.

4. Obtain plane equations and normal vectors of objects

The line and cross laser shape measuring method is a technique for measuring a three-dimensional shape by the shape of a trajectory according to a measurement object using a geometrical optical between a plane laser light and a camera in space. 2A is a structural diagram of a measurement system using a CD camera and a three-dimensional position information measuring apparatus for an object using a laser beam according to an embodiment of the present invention, Figure 7 is a CD according to a preferred embodiment of the present invention This is a configuration diagram showing the geometrical configuration of a three-dimensional position information measuring apparatus of an object using a camera and a laser beam.

As shown in FIG. 2A, the laser slit beam generators 120 and 122 are positioned on the left and right sides of the CD camera 110 for a two-dimensional image 100 mm away from an object plane. When the laser slit beam generators 120 and 122 emit light, the beam is reflected on the object plane. The reflected light is received using the laser measuring CD camera 130, which is inclined at 45 ° and dropped 100 mm at the same height as the 2D CD camera 110 for image. In addition, planes A and B of FIG. 7 are defined as slit planes, and planes C and D are image planes.

The optical trajectory (profile) image formed by the image line of the camera and the laser light in the plane meet at the measurement section at an arbitrary angle is transformed according to the measurement section. Since the modified optical trajectory image has three-dimensional shape information of the measurement object, three-dimensional coordinate values of the measurement section can be obtained through correction.

(4-1) Basics

First, coordinates are set in the geometric model of FIG. 7.

First set the camera coordinates.

8A is a conceptual diagram illustrating a coordinate system setting in the CD camera image plane. Referring to the digital image coordinate system obtained from the image plane of the camera, as shown in FIG. 8A, the horizontal axis on the right side is used as the y ₀ axis and the lower vertical axis is used as the x ₀ axis. The unit of the digital image coordinate system is a pixel representing a pixel unit. However, the actual coordinate system uses a coordinate system whose origin is the center C of the image plane.

 8B is a conceptual diagram illustrating a pinhole camera model, and illustrates a basic model of the pinhole camera. Plane F is called the focal plane and plane I away from this focal plane by the distance f (focal length) is called the image plane. If the coordinate system of the focal plane is taken as shown in FIG. 8B, the coordinate system of the image plane has a coordinate system inverted up, down, left, and right. In the future, all spatial coordinates will be recognized with the coordinate system of the focal plane as the reference coordinate system. The following equation (31) holds for the relationship between two-dimensional coordinates of the image plane and three-dimensional coordinates of the focal plane. The following relationship holds between M points in space and points m in the image plane.

A virtual image plane I 'exists at a distance f away from the focal plane along the optical axis passing through point C. This is the same as inverting the image plane I. In the virtual image plane I ', the equation (32) holds.

That is, the two-dimensional image coordinate (x, y) of one point M in the three-dimensional space has the same value as the virtual image coordinate (x ', y'). Therefore, as shown in FIG. 8B, the left horizontal axis is y 'and the upper vertical axis is x' with the image plane center C as the origin. And the Z axis is in the direction of looking at the object. In the laser coordinate system of the present invention, as shown in FIG. 8B, the direction in which the laser beam travels is set to Z _L , y _L is set to the left horizontal axis, and x _L is the direction coming out using the right hand coordinate system.

(4-2) Plane Equation, Normal Vector Calculation Algorithm

(4-2-1) Plane Equation

When A, B, C, and D are obtained using the coordinates of three points that are not in a straight line on a plane, the general plane equation is given by the following equation (33).

Where (x, y, z) is a spatial point on the plane. Dividing the equation by D and solving for three points is expressed as Equation 34 below.

If the plane does not pass through the origin, it can be expressed by the general formula as shown in Equation 35 below.

,

When the plane passes the origin (when the y axis is not included), it can be expressed as Equation 36 below.

,

(4-2-2) Object plane calculation

The focus of this algorithm is that the slit plane, the image plane, and the object plane all meet in a straight line. The beam emitted from the two lasers causes two common straight lines to meet each other. Both of these straight lines are included in the object plane. Thus, by finding two common straight lines on an object plane, the equation for the plane containing those two straight lines is conversely found.

Before describing the algorithm, the necessary expressions are summarized as follows.

① Image plane linear equation of light source 1: A ₁ x + y + c ₁ = 0

② Linear equation of image plane of light source 2: A ₂ x + y + c ₂ = 0

③ Slit plane equation of light source 1 (A): a ₁ x + b ₁ y + c ₁ z + 1 = 0

④ Slit plane equation of light source 2 (B): a ₂ x + b ₂ y + c ₂ z + 1 = 0

⑤ Plane equation of image plane of light source 1 (C): A ₁ x + y + C ₁ z = 0

⑥ Image plane planar equation of light source 2 (D): A ₂ x + y + C ₂ z = 0

⑦ Plane equation of object surface: a _m x + b _m y + c _m z + 1 = 0

The algorithm goes through the following process.

Step 1. Find the straight line where the image plane of light source 1 meets the slit surface of light source 1.

-Substituting ⑤ into ③, the following equation 37 is derived.

Step 2. Find the straight line where the image plane and the object plane of the light source 1 meet.

-Substituting ⑤ into ⑦, the following equation 38 is derived.

Step 3. Since the two lines obtained in Steps 1 and 2 must coincide, it can be expressed as Equation 39 below.

Step 4. Repeating the above process for the light source 2, equations 40, 41, and 42 are obtained.

Step 5. The system of equations obtained from Equations 37, 38, and 42 is summarized as Equation 43 below.

Step 6. In Equation 43 Since the coefficient matrix of the matrix is not square, using Pseudo Inverse Obtain

Step 7. The normal vector to be.

With the above algorithm, we can geometrically analyze the process of shooting two line lasers (slit beams) on the object and obtaining the light again, and obtain the plane equation and normal vector of the object lying in space.

5. Experimental Results

FIG. 2A is a structural diagram of a measurement system using a three-dimensional position information measuring apparatus for an object using a CD camera and a laser beam according to an embodiment of the present invention.

As shown in FIG. 2A, two laser slit beam generators (line lasers) 120 and 122 are fixed to the sides of the CD camera 110 for two-dimensional images. And the x, y, z stage (102) that can change the position of the object to be measured under 10 degrees of freedom with 5 degrees of freedom, and the α, β stage that can rotate on the x-axis y-axis of the CD camera on it ( 100) was installed. The CD camera 130 for laser measurement rotated by 45 ° on the x-axis was installed at a position 10 mm away from the two-dimensional image CD camera 110. The heights of the cameras 110 and 130 and the lasers 120 and 122 were measured by a height gauge that can be measured in units of 20 μm. The floor on which the stage is to be placed lies in a fully-planned surface.

Each device used in the experiment is as follows.

-. CD camera (110, 130): 640 × 480 pixel, 0.117mm resolution, 2EA

-. Lasers 120 and 122: Class IIIb type, 655 wavelength, 2EA

-. Optical filter: 640 ~ 660 wavelength, 2EA

-. Complete Flat Plate: 550mm × 310mm × 300mm

-. x, y, z stage (102): 10mm shift in each axis, precision of 10

-. α, β stage 100: axis ± 10 °, axis ± 10 ° movement, straightness 0.05mm / stroke

(5.1) 3D information acquisition experiment using line laser (slit beam)

9A is an image obtained by the image of the laser reflected by the CD camera. As described above, images were acquired with an inclination of 45 ° with the object plane. 9B is an image obtained by acquiring only a laser line to be measured by mounting an optical filter.

When two straight lines are extracted from the image of FIG. 9B and found to meet 120 pixels and 360 pixels on the x-axis in the image, they are as follows.

Laser 1: P1 = (120, 173) P2 = (360, 171)

Laser No. 2: P3 = (120, 412) P4 = (360, 419)

Step 1: Using the fact that the four points obtained above are included in one plane, the plane equation of light source 1 can be obtained.

General form of plane equation:

0.089x-0.0125-0.097z + 1 = 0

Step 2: In the same way as in Step 1, the plane equation of the plane equation of the light source 2 is obtained as follows.

0.20x-0.115y-0.055 + 1 = 0

Step 3: Image plane of light source 1:

-0.7125x + y-0.1632 = 0

Step 4: Image plane of light source 2:

-1.745x + y-0.382z = 0

Step 5: Plane Equation of Object Surface:

Thus, the plane equation of the object plane is given by

The normal vector of the plane is as follows.

(5.2) Coordinate transformation algorithm experiment

In this experiment, an experiment was performed to convert two-dimensional coordinates to three-dimensional coordinates under the assumption that plane equations and normal vectors were obtained.

The normal vector used in the experiment was set to N = (0.001, 0.001, 106.55).

And since we do not know the absolute position of the measured hole center point, we conducted an error test for the relative amount of variation in each axis. The experiment was also conducted when the center of the hole was in each area by dividing the image into nine equal parts.

The experimental value is obtained by moving the center of the hole of the object with the normal vector fixed. The three-dimensional coordinates of the center of the hole were measured ten times while the center of the hole to be initially measured was set as a reference value and moved in the x- and y-axis directions by +0.5 mm.

FIG. 10 is a diagram illustrating a movement error value according to the movement amount on the x-axis and a y-axis and an average value at the moved position, and FIGS. 11A to 11B are graphs of the movement error occurring when moving on the x-axis and the y-axis One drawing.

Referring to FIGS. 10 and 11, as a result of the above experiment, some errors occurred in both the movement on the x-axis and the movement on the y-axis. This is because the accuracy of the focal length is inferior and the plane of the CD camera 110 for the two-dimensional image is inclined so that the normal vector does not coincide with the normal vector of the plane, and the movement in each axis is not performed correctly. Error. Also, the use of analog instruments, rather than digital ones, can make it difficult to read the correct scale. However, despite this error, the degree of the error is constant in μm and it can be seen that the reference value is well followed. This can reduce errors if the correct camera calibration is applied and the correct normal vector is calculated. In addition, it is possible to eliminate the cause of the error (axis distortion, target recognition error due to ambient brightness) through the manufacture of the actual experimental module.

Preferred embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art will be able to make various modifications, changes and additions within the spirit and scope of the present invention. Additions should be considered to be within the scope of the following claims.

Claims (8)

(a) acquiring a two-dimensional image of the object by using a two-dimensional image CD camera and calculating a two-dimensional (x, y) coordinate value for the measurement point of the object; (b) calculating and calibrating an internal factor representing a characteristic of the laser measuring CD camera installed at a predetermined angle with respect to the laser slit beams irradiated from the two laser slit beam generators; (c) irradiating two slit laser beams to an object to be measured using two laser slit beam generators and acquiring a beam image using the laser measuring CD camera; (d) calculating a plane equation and a normal vector of the plane on which the object is placed from the obtained beam image; And (e) converting a two-dimensional coordinate value of a measurement point into a three-dimensional coordinate value by using the inner factor, the plane equation, and a normal vector. Dimensional location information measurement method. The method of claim 1, In step (a), (a1) extracting an image of an object by applying a morphology technique to the input two-dimensional image; (a2) binarizing an image of the extracted object; And (a3) A method of measuring three-dimensional position information of an object using a CD camera and a laser beam, comprising: calculating a center coordinate (measurement point) of the object by applying a cross-sectional primary moment to the binarized image. The method of claim 1, In step (d), (d1) binarizing the beam image; (d2) thinning the binarized beam image; (d3) calculating an image plane linear equation for light sources 1 and 2 by applying a Hough Transform algorithm to the thinned beam image; And (d4) Image plane linear equation of light source 1, image plane linear equation of light source 2, slit plane equation of light source 1, slit plane equation of light source 2, image plane plane equation of light source 1, image plane plane equation of light source 2 And calculating a plane equation and a normal vector of the object plane by using a plane equation of the object plane. 3D position information measuring method of an object using a CD camera and a laser beam. The method of claim 3, In the step (d4), the three-dimensional position of the object using a CD camera and a laser beam is calculated by calculating a plane equation and a normal vector of the object plane by using the slit plane, the image plane, and the object plane meet in a straight line. How information is measured. The method of claim 1, In step (e), (e1) calculating the angle rotated on each axis by orthogonally projecting the calculated normal vector to the XZ plane and the YZ plane in the CD camera coordinate system; (e2) converting two-dimensional (x, y) coordinate values for the measurement point into three-dimensional (X, Y) coordinate values using the calculated rotation angle and the internal factor; And (e3) calculating the three-dimensional Z coordinate value by substituting the converted three-dimensional (X, Y) coordinate values into the calculated plane equation of the calculated object plane. 3D location information measurement method. The method of claim 5, The three-dimensional (X, Y) coordinate values of the step (e2) is calculated by applying two-dimensional (x, y) coordinate values independently to the conversion algorithm, If the coefficient of x or y of the normal vector is negative, the three-dimensional (X or Y) coordinate value is or Is calculated as If the coefficient of x or y of the normal vector is positive, the three-dimensional (X or Y) coordinate value is or Calculated as Θ ₂ is a rotation angle between the image plane of the CD camera and an object plane (XZ plane or YZ plane), and X 'or Y' is a spatial coordinate value of x or y when there is no rotation angle θ _2. 3D position information measurement method of an object using a CD camera and a laser beam. In the three-dimensional position information measuring device of the object using a CD camera and a laser beam, A two-dimensional CD camera for photographing and outputting a two-dimensional image of an object to be measured; A laser slit beam generator for a first light source installed on the two-dimensional image CD camera at a predetermined distance on a horizontal line to irradiate a slit beam to an object to be measured; A laser slit beam generator for a second light source installed at a predetermined distance on a horizontal line of the two-dimensional image CD camera to irradiate a slit beam to a measurement target object; A CD camera for laser measurement, which is installed to have a predetermined angle with respect to the laser slit beam emitted from the laser slit beam generator and outputs a beam image of an object to which the slit beam is irradiated; Analyzing the two-dimensional image output from the two-dimensional image CD camera for calculating the two-dimensional center coordinates of the object to be measured, and performing the camera calibration for the laser measurement CD camera to calculate the internal factor, Analyze the beam image output from the CD camera for laser measurement to calculate the plane equation and the normal vector of the object plane, and use the internal factors and the plane equation and the normal vector of the object plane to determine the two-dimensional center coordinates of the object to be measured. A location information measuring unit converting the coordinates into three-dimensional coordinates and outputting the converted coordinates; And And a controller configured to control the two-dimensional image CD camera, the laser slit beam generator, the laser measurement CD camera, and the position information measuring unit to perform three-dimensional position information measurement of an object. 3D position information measuring device of an object using a camera and a laser beam. The method of claim 7, wherein The location information measuring unit, The image of the object to be measured is extracted by applying a morphology technique to the two-dimensional image output from the two-dimensional image CD camera, the image of the extracted object is binarized, and the cross-sectional first moment is applied to the binarized image. A center coordinate calculation module for calculating a center coordinate of the measurement point; A camera calibration module for calculating an internal factor representing a characteristic of the CD camera for laser measurement; The beam image output from the laser measuring CD camera is binarized / thinned to calculate an image plane linear equation for the light source 1 and the light source 2, an image plane linear equation for the light source 1, an image plane linear equation for the light source 2, Using the slit plane equation of light source 1, the slit plane equation of light source 2, the image plane plane equation of light source 1, the image plane plane equation of light source 2, and the plane equation of object plane, calculate the plane equation and normal vector of the object plane. A plane equation / normal vector calculation module; And Orthogonal projection of the calculated vector of the object plane to the XZ plane and YZ plane in the CD camera coordinate system to calculate the angle rotated on each axis, and using the calculated angle of rotation and the internal factor two-dimensional (x, y) After converting the coordinates to 3D (X, Y) coordinates, the 3D Z coordinates are calculated by substituting the converted 3D (X, Y) coordinates into the plane equation of the calculated object plane. 3D position information measuring apparatus for an object using a CD camera and a laser beam, characterized in that it comprises a conversion module.