WO2002003026A1

WO2002003026A1 - System and method for angle measurement

Info

Publication number: WO2002003026A1
Application number: PCT/CH2001/000419
Authority: WO
Inventors: Sergei Startchik; Mehrdad Agah
Original assignee: Fastcom Technology S.A.
Priority date: 2000-07-04
Filing date: 2001-07-03
Publication date: 2002-01-10

Abstract

This application discloses an Angle Measurement System (AMS) for continuously measuring the angle of a metal sheet that is being bent by a bending machine to a required angle. AMS is composed of two modules (or heads) each containing a light source (laser, LED, etc.), an image sensor, a processing unit and a communication unit. One, two or more modules of the system are mounted above or below the sheet and a geometric pattern is projected by each light source on the sheet. This system automatically performs computations and adjustments and sends the resulting angle to the bending machine for control. The angle measurement method benefits from the following design features of the modules to optimize the speed of the computation: Random pixel access of image sensor is used to rapidly locate the projected pattern and then work only with pattern neighborhood. Immediate processing of the visual information allows to avoid transmission and storage elsewhere.

Description

System and method for angle measurement

References cited U.S. 4,564,765 U.S. 5,531,087 U.S. 5,196,900 U.S. 6,021,222

Keywords: Angle measurement, sheet metal bending , laser pattern, image sensor, camera, digital signal processor (DSP), random pixel access.

11 claims, 19 drawing sheets

I. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a systems for opto-electronic, non-contanct angle measurement methods during the process of sheet metal bending by the bending machine using one or more presses and where the bending occurs along one line. When the metal sheet is being bent a laser pattern is projected onto the surface of the sheet being bent and the displacement and deformation of this pattern in the field of view of the camera is used to derive the angle.

II. BACKGROUND ART

Non-contact measurement systems for measuring the bending angle are based on projecting a certain light pattern onto the metal surface. The following patents were considered relevant to the application of angle measurement U.S.4,564,765, U.S.5,531,087, U.S.5,531,900 and the following patent was relevant to the different parts of the used method U.S.6,021,222. In U.S.4,564,765 (expired), the projected pattern is a set of laser spots. A particular configuration of the spots and the distance between them is used to compute the angle. The main limitation of the method is the speed and precision of point detection.

In U.S.5,531,087 (pending), two lines were projected and their relative rotation was used to compute the angle. This method unless its original design, presents several drawbacks. First, measuring and light emitting devices are spread over the large area along the bending ram thus obstructing the access to the machine for the operator. Second, a change in light emitting device position is required to adapt the system for a new angle measurement. Third, pattern measurements are done without taking into account perspective distortion of the image.

In U.S.5, 196,900 (pending), a set of spots (points) is used as the pattern and as mentioned earlier, their detection requires exhaustive search in the image and not suitable for fast detection.

With respect to the methods to estimate the bending angle from patterns with particular shape, a circle detection algorithm is suggested in U.S.6,021,222. The performance with respect to speed needs, however, to be analyzed.

The herein presented method is suitable for and is efficiently implemented on an image sensor with pixel-based access. This selective access allows to speed up the computation and communication processes so that the angle measurement can be performed in just a few milliseconds. Another advantage of the suggested method is the versatility that allows many possibilities to position the measuring system on the bending machine.

III. DISCLOSURE OF THE INVENTION

The opto-electronic approach to angle measurement consists in projecting a laser pattern onto the sheet metal being bent and measuring the motion and deformation of this pattern during bending. The measured motion and deformation can then be used to directly compute the angle between two positions of the sheet metal.

The general algorithm of the angle measurement is presented on Fig. 2 and comprises several staged explained in the following sections. The stage of rapid localization the laser pattern in the field of view of the camera is explained in section III.1. The stage for computing displacement and deformation of the projected pattern is presented in section 2. Section 3 describes the computation of the angle from the properties of the projected pattern.

Diverse laser patterns composed of several crossing or parallel lines or one or more circles are suitable for measuring the angle. For example, for the cross laser pattern the information that is used to compute the bending angle is the intersection point of two lines of the cross. Detailed angle computations from pattern position are shown for cross laser pattern and the parallel lines.

Realization of the system is described in section V. 1. Fast pattern localization

To rapidly find location of the cross pattern in the field of view, the algorithm described in Fig. 3 is used. It is based on a set of horizontal Scan lines S_i (47) that are evenly spaced in vertical direction as shown in Fig. 4.b,c.d.

Only one line S_t of the image sensor is acquired at a time and its intensity contents is analyzed to find two significant peaks based on intensity maxima (see Fig. 4.a). If two peaks A_t ,B_i are found, their positions are stored and the scanning continues. If no peaks were detected, the next scanline is acquired and analyzed. If three pairs of peaks are found, the scanning stops and an ordering procedure is invoked.

Three pairs of points A₁ ,B_λ ; A₂ ,B₂ ; A₃ ,B₃ , are necessary to uniquely find the approximate position of the crossing lines. However, three pairs of points can correspond to one of the three configurations outlined in Fig. 4(b,c,d). To efficiently recognize which of the three configurations is currenly observed, one need to test the collinearity of point triples and decide which point correspond to which line. To check the collinearity, the following determinants (indicated by shaded areas in Fig. 5) are to be computed first:

and then their sums need to be computed as follows:

Δ₃ - _j£₂.S₃| + |.B_{j 2 3}|

The first configuration (Fig. 4.a) can then be distinguished by the fact that the first sum Δ_j is smaller in absolute value than the sum of other two Δ₂ + Δ₃ :

2Δ_j < Δ₂ + Δ₃ (3)

and at the same time its absolute value is limited by a fixed threshold Δ_j < threshold that defines the limit of collinearity.

Similarly, the second and third configurations can be distinguished with the conditions 2Δ₂ < Δ_j + Δ₃ and 2Δ₃ < Δ₂ + Δ_j respectively and conditions on the absolute values of Δ₂ and Δ₃ . Distinguishing between configurations allows to split the six points A₁ ,B_χ ; A₂ ,B₂ ; A₃ ,B₃ in two groups of three points each, belonging to each line. The three points on each line will thus determine positions of both lines in the image.

This localization operation is very time-efficient since it requires the acquisition and analysis of several lines per image only, as well as evaluation of determinants. The similar procedure can be used to locate the pattern of parallel lines as shown in Fig. 6. A set of maxima points are located on the scan lines. Their order is used to group them for defining where the laser pattern is located.

2. Fast computation of precise pattern position

Once the approximate pattern position was provided by the localization step, we can proceed with the estimation of precise pattern position and properties. The precise position of the cross is the intersection of two lines. The precise position of a line can be obtained by fitting the line equation to a number of precise points that belong to the line. Those precise positions are obtained by analyzing the neighborhood of the approximate line position as shown in Fig. 7.

The Horizontal short scan lines (22) of adjustable pixels length are acquired along the approximate position of each line found previously in the localization step (see Fig. 7.a). These short scanlines are acquired from Upper scanning limit (29) to Upper middle scanning limit (31) and from Lower middle scanning limit (32) to Lower scanning limit (30). The Upper scanning limit (29) and Lower scanning limit (30) are defined by the first and last Scan lines S_t (47) that has two significant peaks (see Fig. 4). The Lower scanning limit (30) is defined by the last Scan lines S_t (47) in Fig. 4 that has two significant peaks. The Upper middle scanning limit (31) and Lower middle scanning limit (32) are defined to avoid acquiring information in the area near the intersection point (45) where interference between two crossing lines can occur (the found peak can belong to any of two lines).

The Horizontal short scan lines (22) are acquired in the image and intensity peak Intensity maximum in short line (48) inside each of them is detected that correspond to precise position of the laser line of the Laser pattern (4) across the short scanline as shown in Fig. 7(b). To obtain a higher precision than a pixel, a subpixel accuracy method is used for each scan line as in Fig. 7(c). A parabola or gauss- ian is used to approximate the maxima pixel (48) and its two adjacent pixels and the coordinate of the parabola maximum Precise maxima position (44) is retained as the precise maximum position for the current short scanline.

The set of computed line points Precise maxima position (44) is used to fit a line Fitted line (43) as shown in Fig. 7(d) in a Least Mean Squares (LMS) sense to these points. The obtained two equations of two crossing lines are used to analytically find the Precise intersection (45) that gives a precise cross position.

3. Geometry of the angle measurements

Precise position, orienation and deformation of the laser pattern can be used to derive the angle of the sheet metal orientation. For the cross pattern, the displacement of the cross center can be used to compute the angle as described in the following section. For a set of parallel lines their combined orientation can be converted into the bending angle. However, to obtain a correct measurement a reference information is required. So, the position and form of the cross f.ex. should be measured for several known (calibration) angles. The number of calibration angles depend on the complexity of the pattern (its number of degrees of freedom). Each pattern has a certain number of degrees of freedom (DOFs) that are necessary to completely define it. More degrees

3.1 Cross pattern

When the metal sheet is being bent, the cross laser pattern changes its position in the field of view of the camera as in Fig. 8(a).

Let us assume that position of the cross center for the bending angles of 180, 120 and 90 degrees are 180 degrees calibration point (39), 120 degrees calibration point (39) and 90 degrees calibration point (40) respectively and are defined by ₁₈₀, P_12Q and P₉₀. Position of these three points should be found prior to any angle measurement and corresponds to the angle calibration.

Let us denote now by P_m the Current point (41) that corresponds to the current measurement of the cross center position. The bending angle can then computed from this position by using its relative position with respect to the calibration point positions. With the use of a narrow angle objective, the radial distortion is low and the trajectory of the cross center during bending can be considered linear.

To establish the exact position of the P_m with respect to P_m , P₁₂₀ and P_go , we project P_m onto the line ₁₈₀ P₉₀ (cf. Fig. 8(b)) and obtain the point P_c . Four points are then mapped to the normalized (barycentric) coordinates as shown in Fig. 8 so that ₁₈₀ is mapped to 0 and P_gQ to 1.

3.2 Computation of the angle from three point calibration and cross pattern

For simplicity, let us consider that laser pattern is a point ray that passes through the center of the cross and we observe the displacement of one point only in the image. In Fig. 9, a side view of the camera and the sheet is presented. Optical ray of the laser and optical axis of the camera are overlappting. Using the definitions of lengths in this figure we first find that:

where α is the half of the bending angle. Then, in Fig. 10 a top view of the camera and sheet metal at different positions is presented where the center of coordinates is the camera optical center and β is the angle between the optical axis of the camera and that of the laser. From that setup one can find:

-^Ho--^Di2θ^tanβ 120

(5) //₀-D_gotanβ = P' 90

/₀-£»_αtanβ = P'

Projecting points P'_m , P'_12Q , P'₉₀ and P'_a on one line Image plane (49) (that could be selected arbitrarily, but parallel to sheet being bent) we then obtain :

₀-/J₁₈₀tanβ = P 180

D ff₀-.D₁₂₀tanβ = P_no- ¹²⁰ ^UD 180

7⁾, (6) ^■/o-^tanβ = P₉ ^180

/7₀-/J_αtanβ = P_α-^ ^180

Eliminating i₀ and tanβ we obtain:

Use of barycentric coordinates defined as follows (see mapping on Fig.8.c):

^120 ^"" ^90 ⁺ (-^180 ~ ^90)^120

(8) ^Pa ^{= •P}90⁺(-^P180^_-^P9θ)*α the two equations are then reduced to the following pair:

D_{90 60}t₆₀ + 15) = (D_ω( ^D6 (-^t+t6θ)-^D9 -^t+ 1))

substitung D_a from the equation (4) and optimizing, we obtain:

Eliminating the ratio of unknowns (substituting the first equation into the second) we finally obtain the formula for the angle from the coordinate of the cross position

To compute the angle one needs to know the positions of calibration points P₁₈₀ , P₁₂₀ , P₉₀ and that of the current measurement position P_m . The barycentric coordinates r₆₀ , r_α of the P₁₂₀ and P_c are then computed with respect to P₁₈₀ and P₉₀. Substituting those barycentric coordinates into the previous forumula, gives the tangent of the bending angle. The mapping function position of the projected point P_c would then be mapped to the bending angle according to the function in Fig. 11(b).

This method requires three calibration angles 180° , 120° , 90° . Using the orientation of the cross, the number of calibration points might be reduced to one. All the stages of the angle computation from cross position are presented in Fig. 19.

3.3 Computation of the angle from several parallel lines

Taking a more complex pattern as a set of parallel lines allows to have more degrees of freedom (DOFs) per measurement and thus less calibration angles. For a pattern of more than 5 parallel lines the only calibration that is required is the 180 angle. Moreover, the position of the calibration plane is not constrained to the bending matrix and can occur anywhere on the optical path of the camera.

IV. BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 Side view of one module.

Fig. 14 Configuration where two modules are placed above the sheet metal being bent.

Fig. 12 Configuration where two modules are placed below the sheet metal being bent.

Fig. 15 Configuration where two modules are placed above the sheet metal being bent. Mirrors are used to mount modules closer to the bending ram.

Fig. 16 External connections of two modules.

Fig. 2 General algorithm of bending angle calculation.

Fig. 3 Algorithm for localization of the "cross" pattern.

Fig. 4 The stage of the fast cross pattern localization corresponding to acquiring scan lines and finding three pairs of points, (a) Example of the profile. (b,c,d) Three possible configurations.

Fig. 5 Second stage of localization where pairs of points are sorted and attributed to two lines according to their collinearity.

Fig. 7 Computing precise position of cross pattern, (a) Acquisition of the short scan lines in pattern neighbourhood, (b) Maxima detection, (c) Subpixel maxima detection, (d) Fitting lines to the detected points and the definition of the intersection. Fig. 8 Cross displacement during bending, (a) Calibration positions of the cross, (b) Computation of the relative cross center position, (b) Mapping crosss position to relative (barycentric) coordinates.

Fig. 9 Side view of viewing geometry.

Fig. 11 Mapping between position of the cross point and the (half) bending angle.

Fig. 18 Algorithm for calibrating the system.

V. BEST MODE FOR CARRYING OUT THE INVENTION

This section, describes the implementation of the described invention as it was realized in the Smart Bending Angle Calculator SBAC-1135.

1. System architecture

The camera and laser can be mounted in one single casing as shown in Fig. 1. Each module has the "all-in-one" casing structure that is illustrated on Fig. 1. The main casing contains the Motherboard (23) with the processing unit and communication interfaces connected to External connectors (36), the VGA board (24). The Sensor controller (26) is connected by a Flexible connector (46) to the Sensor board (51) hosting the CMOS sensor (50). The camera Objective (33) is mounted on this board. The DAC Card (25) is connected to the Laser (7) and can control its power and intensity through the Laser power cable (27) and Laser intensity control cable (28).

External connectors of one module are presented on Fig. 16. The master module contains RS232 port (10), RS485 port (11), Digital port (12), VGA port (14), Master network or CAN-bus connector (38). The slave module has only.

2. Flexible geometric setup

To have a precise measurement of the the bending angle, one need to make measurements on both sides of the press. This requires the use of two modules (master and slave) on both sides of the press that are simultaneously measuring the angle and communicating between themselves. The mounting of the modules can be performed in several ways shown in Fig. 12, Fig. 13, Fig. 14 and in Fig. 15. In the first "standard" configuration, two modules Master module (5) and Slave module (6) are positioned below the metal sheet being bent on both sides of the Bending ram (2) and pointing upward to the Bended metal sheet (3). The number of master/slave pairs (that measure the angle) along the benging ram can be more than one.

None of the modules require a specific position as long as the pattern is visible at the starting bending angle and at the ending angle. Therefore, modules can be positioned with respect to the sheet being bent according to the particular requirements of the user. 3. Fast pattern localization

The fast pattern localization method described in Section III can be efficiently implemented on a CMOS sensor that allows pixel-based access. A pixel-based image sensor is an image sensor which allows for accessing each individual pixel in any chronological or spatial order at any point in time. Each module is equipped with CMOS sensor and with a controller that allows a random-pixel access.

Moreover, the sensor can be used in a special mode that gives priority to the horizontal direction of sensor access. Doing so, the acquisition of horizontal line segments is even faster which suits perfectly the scanline approach.

4. Calibration

The calibration operation is performed according to the algorithm described in Fig. 18. One needs to provide the sheet metal in the correct angle orientation with respect to the bending matrix as shown in Fig. 17. This is achieved with specially designed calibration tools for 180° , 120° and 90° degrees. Their three-point contact provides an easy use.

5. Communication

Two modules are connected by a RS485 or fieldbus interface and the computed angle and other commands are communicated between them over a specially designed protocol.

The resulting angle is then communicated by the master module to the device that controls the bending machine over the RS232 or fieldbus interface with proprietary protocol or standard.

Claims

VI. WE CLAIM:

1. An orientation measurement apparatus and method comprising a high-dynamic sensor capable of reading single pixels, lenses, a processing unit, communication interfaces and a means for projecting a light pattern onto the surface of a planar object. The said apparatus can be integrated in a single casing. The light pattern is a grid of more than two crossing lines allowing for the unconstrained measurement of the surface orientation of the planar object in the three-dimensional space.

2. An orientation measurement apparatus and method as in claim 1 for the measurement of the orientation of sheet metal in metal processing and metal bending applications.

3. An orientation measurement apparatus as in claim 1, whereas the means for projecting a light pattern is a laser with a pattern generator.

4. An orientation measurement apparatus as in claim 1, whereas the light pattern is a set of more than two parallel lines, allowing for unconstrained measure of the surface orientation of the planar object in the three-dimensional space.

5. An orientation measurement apparatus as in claim 1, whereas the light pattern is a circle, an ellipse, a set of circles or ellipses, a matrix of dots allowing for unconstrained measure of the surface orientation of the planar object in the three-dimensional space.

6. An angle measurement method as in claim 1, whereas a line-based access of the sensor is used to find the position of the light pattern in the image. Several lines are acquired by the image sensor to find the intersection of the intensity maxima of the pattern with the scanning lines. The search lines can be horizontal, vertical, in non-consecutive order or diagonal with arbitrary angles.

7. An angle measurement method as in claim 1, whereas a pixel-based access of the sensor is used to find the position of the projected light patterns in the image. Intensity maxima of the projected light patterns are found using pixels that are accessed in a non-consecutive and non-continuous way.

8. The angle measurement system as in claims 1 comprising two or more angle measurement apparatuses providing the possibility of exchanging data.

9. The angle measurement system as in claims 1 that needs less than three reference angle measurements to be fully calibrated if the cross pattern is used. In the case of the pattern of more than five parallel lines, only one reference angle measurement is required for calibration. The module position and orientation with respect to the bending ram in the latter case is not constrained.

lO.The angle measurement system as in claims 1 that uses a calibration tools with three points of contact with the matrix of the bending machine.

11. The angle measurement system as in claims 1 whereas the light pattern is detected by computing the difference between an image taken with the light pattern projected onto the planar object and another image taken without the light pattern projected onto the planar object.