JP4406796B2 - Non-contact three-dimensional object shape measuring method and apparatus - Google Patents

Description

[0001]
[Industrial application fields]
The present invention is used in the fields of various monitoring systems, traffic measurement systems, automatic traveling vehicles, robots that perform operations such as welding and sealing, devices for automatically inspecting the appearance of products, etc. The present invention relates to a method and apparatus for measuring the position, distance, or shape of an object in a non-contact manner by measuring the three-dimensional position coordinates of each point on the target object of the apparatus.
[0002]
[Prior art]
Conventional methods for measuring three-dimensional position coordinates or distance by non-contact include triangulation and focusing methods. The former projects laser light on the surface of the object to be measured, and the reflected light generated from the surface of the object is received by the TV camera from a direction inclined at a certain angle from the projection direction, and the principle of triangulation from the change in the position of the bright spot image, It measures the three-dimensional position coordinates of a target or the distance from a reference point in a reference coordinate system installed in association with the camera. The triangulation method is also called a light cutting method because it measures a contour line represented by a bright line formed by crossing a plate-like projection laser beam and the surface of a measurement object. In the latter, in order to form an image of a surface portion of an object whose distance is to be measured on a light receiving surface (photoelectric conversion surface) of a television camera composed of a two-dimensional array of photoelectric conversion elements such as a CCD image pickup element, The distance is measured from the amount of movement of the focus adjustment mechanism using a focus adjustment mechanism that adjusts the position of the imaging lens to focus the image. In order to obtain the distance of the entire target surface, the focus adjustment mechanism is continuously moved, the surface portion in focus at each moving position of the mechanism is detected by image processing, and the distance of each portion of the surface is obtained. Also, the three-dimensional position coordinates of each point on the object surface can be obtained by setting a reference coordinate system. In the following description, obtaining the 3D position coordinates of each point and obtaining the distance between different 3D position coordinates are based on the same measurement method, and technically distinguish between the two because they are in the same category. We will use without.
[0003]
[Problems to be solved by the invention]
In the triangulation method, in principle, in order to receive the reflected scattered light from the object, it is necessary to place the light receiving optical system tilted with respect to the measuring light projection optical system. For this reason, there is a problem that a blind spot is generated in the measurement part of the object, and in particular, when the object is a cluster of many components, the triangulation method is not practically applicable. Moreover, in order to project a laser beam from the side, it is necessary to arrange a projection optical system in the measurement space, and there is a problem that the measurement space is blocked. Furthermore, since the projection optical system projects sideways, the measuring apparatus tends to be large as a whole, which is disadvantageous for the purpose of mounting it on a robot as a visual sensor.
[0004]
On the other hand, in the conventional focusing method, the blind spot is small, but since the focus adjustment mechanism that moves along the optical axis of the imaging optical system, that is, toward the target, is used, it takes time to operate the focus adjustment mechanism. In this case, the object cannot be measured instantaneously, and the object cannot be moved in the direction perpendicular to the optical axis, that is, in the lateral direction during the measurement period. Therefore, during the measurement period, the object is measured during the measurement period, such as measuring the shape of a long object, continuously measuring products carried by a conveyor belt in a factory, distance sensors used for robot scanning control, or measuring outdoor moving objects. The conventional focusing method is not suitable for measurement when moving. _
[0005]
The present invention moves in a horizontal direction with respect to an optical axis at high speed by using an imaging apparatus that does not require a focus adjustment mechanism in the optical axis direction in a focusing method with few problems of blind spots in distance and three-dimensional position measurement. The object is to enable measurement of the distance of an object or measurement of three-dimensional position coordinates.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, in the non-contact three-dimensional object shape measurement method based on the focusing method of the present invention, the imaging device assumes a measurement cross section that obliquely intersects with the optical axis of the imaging lens, and performs imaging. A light-receiving surface of a two-dimensional image sensor in which photoelectric conversion elements such as CCD image sensors are two-dimensionally arranged is placed on the surface that connects the images of the measurement cross-section with the lens, and the measurement cross-section that extends obliquely in the depth direction intersects the object surface. The object contour is detected as a focused image area from the object image, and the object surface is scanned laterally by the measurement cross section to detect the contour lines at different positions on the object surface continuously. Try to get the position and shape of the surface.
[0007]
Therefore, in the image pickup apparatus of the present invention, unlike a normal television camera, the light receiving surface of the image pickup element that receives and photoelectrically converts an optical image is not arranged perpendicular to the optical axis of the image pickup lens. Depending on the inclination with respect to the axis, the angle of the light receiving surface with respect to the optical axis of the lens is shifted from 90 °. As a result, the target surface in the vicinity of the measurement cross section extending in the oblique depth direction will be focused without adjusting the focus of the lens, and image information from the image signal obtained from the output of the image sensor By extracting the in-focus area by the processing, it is possible to obtain the outline of the cross section that cuts the target obliquely from the in-focus area.
[0008]
[Action]
By adopting the arrangement of the imaging optical system as described above, the contour shape of the cross section of the target can be obtained simultaneously with the image acquisition without adjusting the focus of the optical system, and high-speed measurement is possible. In addition, since the surface of the object is scanned by scanning in the horizontal direction using the measurement cross section, the measurement can be performed when the object moves laterally. Further, by actively utilizing the lateral movement of the object for scanning the surface of the object, it can be easily applied to the shape measurement of the product on the conveyor belt. Therefore, it is possible to measure a long object.
[0009]
【Example】
An embodiment will be described with reference to the drawings. FIG. 1 is a cross-sectional view of an imaging optical system showing a basic configuration of a measuring apparatus that implements a three-dimensional object shape measuring method according to the present invention. The optical axis 4 is on the paper surface of the cross-sectional view, the two-dimensional image sensor 3 in which photoelectric conversion elements such as a CCD image sensor are two-dimensionally arranged is arranged perpendicular to the paper surface, and its light receiving surface (ab) 7 is on the optical axis. On the other hand, it is inclined by an angle φ as shown in the figure, and is also an imaging plane for the imaging lens 2 having a measurement section (AB) 8 perpendicular to the paper surface. That is, the measurement cross section (AB) 8 is a focusing surface of the imaging optical system having the light receiving surface (ab) 7 as an imaging surface, and the ends a and b of the light receiving surface (ab) 7 are respectively in the measurement cross section AB. It is an image of ends A and B. At this time, the inclination angle of the measurement cross section (AB) 8 with respect to the optical axis 4 of the imaging lens is defined as an angle θ as shown in the figure, and the measurement cross section (AB) from the lens center (generally the object side principal point of the lens) O of the imaging lens 2. ) Where H is the vertical leg standing on the plane including 8 and OH is the distance d from the imaging lens 2 of the measurement cross section (AB) 8 and f is the focal length of the imaging lens 2. In the meantime, the relational expression d = f × cos θ × (tan θ + tan φ) needs to be established.
[0010]
In particular, when the above-mentioned various amounts are set so that the relationship of f = d × sin θ holds when the angle θ is 45 °, the angle φ is also 45 °, so that it is possible to avoid a steep inclination and to the light receiving surface (ab) 7. The incident angle of the image is not extremely large (incident light is not incident so as to be parallel to the light receiving surface), and the present invention can be implemented without deteriorating the light receiving conditions. At this time, the distance from the center O of the imaging lens 2 to the center of the measurement cross section (AB) 8 (a point on the optical axis) is 2f, the imaging magnification near the optical axis 4 of the imaging lens is 1, and the light receiving surface. (Ab) 7 and measurement cross section (AB) 8 have a substantially identical dimension and correspond one-to-one. Strictly speaking, the magnification is 1 on the optical axis 4 of the imaging lens, the magnification gradually increases from 1 as the end A of the measurement cross section (AB) 8 is approached, and the magnification is from 1 as the end B is approached. It gets smaller. Under the implementation conditions in which this change can be ignored, the magnification may be 1 over the entire measurement cross section (AB) 8, and as described later, at each point of the image of the measurement object 1 obtained on the light receiving surface (ab) 7. It becomes easy to associate the three-dimensional space coordinates on the measurement cross section (AB) 8, and the calculation process for obtaining the target three-dimensional shape is simplified. FIG. 1 shows an embodiment in which the angles θ and φ are both 45 °. In the conventional triangulation method (light cutting method), an optical system for projecting laser light is arranged on the side of the imaging optical system, on the extension of the measurement cross section. Since there is no need to arrange the optical system to the side, the structure is small and simple.
[0011]
An outline 9 (indicated by parallel oblique lines) of the measurement object 1 is determined as a three-dimensional curve or straight line formed by the surface of the measurement object 1 and the measurement cross section (AB) 8 intersecting. In FIG. 1, a measurement object 1 is placed on a carrier 6 such as a conveyor belt, and as indicated by an arrow (which can also be implemented in the opposite direction to FIG. 1) at a constant speed in the horizontal direction, generally light. When transported in a direction perpendicular to or intersecting the axis 4, the surface of the measurement object 1 is scanned at a constant speed by the measurement cross section (AB) 8. The object 1 is illuminated from the surroundings by an annular light source 5 (in the case of a bundle of optical fibers, a lamp house and an optical fiber guide connecting the annular light source and the lamp house are required but not shown in the figure). While giving the image of the measuring object 1 at a constant time interval by the two-dimensional imaging device 3, it is possible to capture an image of the cross-sectional outline 9 at different positions of the measuring object 1 at a constant interval. If the conveying speed is not constant, the intervals between the contour lines 9 of the cross section are naturally unequal intervals. Since the image of the contour line 9 is on the measurement cross section (AB) 8, it is always in focus, and the conventional image processing technique is applied from the image of the measurement object 1 obtained as the output of the two-dimensional image sensor 3. It can be easily extracted as an in-focus image area.
[0012]
FIG. 2 shows N measurement cross-sections 8 at different positions on the measurement target 1 obtained when the measurement target 1 placed on a plane on the transport body 6 is scanned by the measurement cross-section 8, respectively (1), (2 ), (3),..., (N-2), (N-1), (N), and N pieces detected as each of the measurement cross sections 8 and the surface of the measurement object 1 intersect. The contour line 9 is [1], [2], [3],..., [N-2], [N-1], and [N], respectively. These are respectively displayed in (b), and the plan view (b) represents the three-dimensional shape of the measuring object 1 by a series of contour lines 9. At this time, a contour line 9 formed on a plane on the carrier 6 is also displayed as a continuation of the contour line of the measuring object 1. In the side view (a), the measurement sites indicated by D1, D2, and D3 of the (N-2) -th, (N-1) -th, and (N) -th measurement sections 8, respectively, are obtained by the conventional light cutting method. Is a part that becomes a blind spot for measurement, and is a part where plate-shaped measurement light projected from the direction corresponding to the measurement cross section 8 does not reach. In the present invention, such a blind spot in measurement does not occur, and can be measured without being missing as seen in the corresponding contour lines [N-2], [N-1], and [N] in the plan view (b). .
[0013]
Various methods are known for extracting a focused area. For example, Eric Krockov's research paper “Focusing” (International Journal of Computer Vision, Volume 1, pages 223-237, published in 1987, original title: ERIC KROTKOF, “Focusing,” International Journal of Computer. Various methods are shown in comparison in Vision, 1, pp. 223-237 (1987), Kluwer Academic Publishers, Boston. In general, the in-focus area is an image area characterized by including a high spatial frequency component having a large change in brightness compared to the out-of-focus area. In many in-focus area extraction methods, an in-focus level indicating the degree of in-focus is defined using an image feature amount that reflects such characteristics, and the in-focus area is determined based on the in-focus level. Broadly speaking, a method for evaluating the in-focus degree by directly obtaining a high-frequency component of the spatial frequency distribution of the image region by Fourier transform, and an image feature amount that indirectly reflects the high-frequency component in order to speed up the calculation process. There is a method of defining the degree of focus using. As the latter method, the magnitude of the primary difference of the light / dark distribution or the edge intensity of the light / dark distribution is summed up for all the pixels in the neighboring area in the vicinity area of the target pixel, and the total amount is used as the focus degree of the target pixel. Represents the degree of variation in brightness (density) of pixels in the neighboring region, the sum of the magnitude of the secondary difference amount of the light / dark distribution over all neighboring pixels as the focus degree of the pixel of interest There is a method in which a variance relating to brightness (density), which is a statistic, is used as a focus degree of a pixel of interest.
[0014]
There are various other methods for evaluating the degree of focus. All of them evaluate the magnitude of brightness (density) fluctuation in the area near the pixel of interest. The out-of-range area uses small fluctuations, is suitable for the characteristics of the surface of the measurement target and the purpose of the measurement, has a high discrimination capability, is easy to find, and has a wide range of image processing. It is defined using image feature values obtained by a combination of methods. Even when the present invention is implemented, an appropriate degree of focus may be defined and used according to the measurement object and the purpose of measurement.
[0015]
Since the extracted in-focus area includes a contour line and pixels in the vicinity of the contour line, it generally has a band shape and a width. In order to obtain the position coordinates of the contour line with higher accuracy, the center of the in-focus area is obtained. There is a need. Focusing is a method of determining the center of the area, in the image processing technique generally methods and each pixel by binarizing method and simply focus area known as thinning process takes the center of the width of the area if Use an appropriate method in terms of the amount of image processing and the accuracy of the results obtained by using existing image processing methods such as the method of obtaining the center of gravity in the width direction of the in-focus area using the value of the degree of focus as a weight, and their modifications. be able to. In addition, before the contour detection process, it is possible to smooth the extracted focus area or perform a process that eliminates the influence of noise such as removing isolated noise. Depending on the measurement accuracy to be performed.
[0016]
The conventional technology can be applied to the configuration of the image processing apparatus as hardware technology for speeding up the image processing technique and processing described above, and is not particularly described in the embodiment shown in FIG. Also, the overall configuration of a so-called imaging camera that accommodates and controls the drive of a two-dimensional imaging device and outputs an image signal generated from the device as an appropriate electrical signal is not shown because it is not directly related to the content of the present invention. Furthermore, in order to input image data to a data processing device (not shown) such as a computer that performs image processing, a conventional signal conversion device that receives an image signal from an imaging camera and converts it into an appropriate digital signal is also used. It can be easily configured by technology and knowledge and is not shown. Further, a device for driving and controlling the transport body 6 and a control system for operating the above devices as a whole are not shown because they are not directly related to the description of the embodiments of the present invention. It can be implemented by technology.
[0017]
Since the light receiving surface (ab) 3 is an imaging surface of the measurement cross section (AB) 8, each pixel in the focus area of the image received by the light receiving surface 3 is in one-to-one correspondence with each point on the measurement cross section 8. Correspond. Therefore, each pixel of the image of the contour line 9 obtained as the in-focus region has a one-to-one correspondence with each point of the contour line 9 on the measurement section 8. Accordingly, reference points whose three-dimensional coordinates are known are set in a lattice shape on the measurement cross section 8, and the coordinates of the corresponding pixels are obtained by imaging these reference points, and the tertiary corresponding to the pixel coordinates in advance. An original coordinate conversion table can be created. At this time, the optical aberration of the imaging lens 2 is also corrected.
[0018]
Specifically, for example, a plurality of white disks having a black background as reference points are arranged at lattice points on the measurement cross section (AB) 8, and images of these disks are picked up. Obtain a digitized image and calculate the center position of each disk. Then, a correspondence table between the coordinates of these center positions on the image and the three-dimensional coordinates of the center of the reference disk on the measurement section (AB) 8 is first created. Since the center position of each disk on the obtained image does not always match the integer coordinate value of the pixel or the coordinate value represented by the binary number of finite digits (the grid point of the pixel coordinate), the correspondence table obtained earlier Then, a correspondence table of three-dimensional coordinate values with respect to integer coordinate values of pixels or binary coordinate values of finite digits (lattice points of pixel coordinates) is created again by interpolation. This functions as a coordinate conversion table that can know the three-dimensional coordinate values (x, y, z) of the corresponding contour line using the coordinate values (m, n) of the pixels of the contour image as arguments. By creating a coordinate conversion table in advance, the three-dimensional coordinates of the contour line 9 on the measurement object 1 can be obtained easily and at high speed from the contour image, and the measurement object 1 is scanned by the measurement section 8. Thus, a series of three-dimensional coordinates of the contour line representing the surface shape of the measuring object 1 can be efficiently obtained. At this time, if the brightness (density) values of the pixels corresponding to the respective points on the contour line 9 are stored together with the three-dimensional coordinates, the geometric three-dimensional shape is displayed when the target three-dimensional shape is displayed on various display devices or printed materials. In addition to the geometric shape, information on the brightness (density) of the surface of the object can be displayed at the same time, so that a realistic or realistic display can be achieved.
[0019]
As described above, when the inclination angle of the light receiving surface 7 and the measurement section 8 is both 45 ° with respect to the optical axis 4 of the imaging lens, the imaging magnification in the measurement section is strictly equal to the optical axis. If it can be ignored but it can be ignored, or if sufficient accuracy can be achieved by correcting with the theoretical formula of the magnification based on the lens imaging formula, the two-dimensional image sensor Since the arrangement interval p of the photoelectric conversion elements 3 is known, the three-dimensional coordinate value can be obtained from the pixel coordinates of the contour line 9 by calculation using the magnification without imaging the reference grid point.
[0020]
For example, here, the case where the photoelectric conversion element and the pixel of the two-dimensional imaging element 3 correspond one-to-one is handled for the sake of simplicity. The pixel coordinates of the image of the contour line 9 are (m, n), and the intersection of the measurement section 8 and the optical axis 4 of the imaging lens is the origin of the three-dimensional coordinate axis. The x axis is perpendicular to the scanning direction of the measuring object 1 by the measurement cross section 8, the y axis is along the scanning direction, and the z axis is along the optical axis 4. Furthermore, if the m-axis and the n-axis are parallel to the x-axis and the y-axis, respectively, and the orientation of each coordinate axis is set appropriately, the three-dimensional coordinates (x, y, z) of each point on the contour 9 are From the coordinates (m, n), the calculation formula x = p × (m−mo) {1 + δ (n−no)}, y = p × (n−no) {1 + δ (n−no)} × sin 45 °, z = p × (n−no) {1 + δ (n−no)} × sin 45 °. However, (mo, no) is the pixel coordinate of the optical axis 4, and δ (n-no) is a correction amount indicating the degree to which the imaging magnification deviates from 1 as the pixel (m, n) moves away from the optical axis 4. Thus, except for the optical aberration of the lens, it can be easily obtained from the lens imaging formula as a function of the pixel coordinate value n-no with the optical axis as the origin. Similarly, if the magnification of the optical axis point is known in advance even if the magnification is not equal, it is natural that the corresponding three-dimensional coordinates can be obtained by calculation from the pixel coordinates. In addition, when the measurement object 1 is continuously scanned by the measurement cross section 8 and N contour lines 9 are obtained, in order for the N contour lines 9 to express the three-dimensional coordinate data of the entire measurement object 1, For example, for the N contour lines 9, coordinate offset values n × Δy may be added to the respective y coordinates in the order in which the contour lines 9 are detected. However, (DELTA) y is a constant which shows the measurement space | interval of the outline 9, and n is a detection order number 1,2, ..., N-1, N.
[0021]
FIG. 3 shows a flow of data processing from the measurement cross-section imaging performed in the embodiment of the present invention described above to obtaining the shape measurement value of the measurement object.
【The invention's effect】
[0022]
Since the present invention is configured as described above, the following effects can be obtained.
[0023]
Unlike the conventional focusing method, the focal plane is set obliquely with respect to the optical axis of the imaging optical system, the focal plane is the measurement cross section, and the measurement target is perpendicular to or intersects the optical axis by the measurement cross section. Since the three-dimensional shape of the measurement object is obtained by scanning in the horizontal and horizontal directions, the scanning distance is not limited by the size of the measurement space in the optical axis direction and can be measured even for long objects. In addition, since there is no need to limit the movement of the measurement object in the direction of the optical axis of the imaging optical system, the degree of freedom in the configuration of the measurement system is great. Production lines in the manufacturing industry, various monitoring systems, traffic measurement systems, automated vehicles, It can be widely applied to robots that perform copying operations such as welding and sealing. In addition, since a focus adjustment mechanism is not required, it is possible to measure at high speed while moving the measurement object.
[0024]
Unlike the light cutting method, it is not necessary to irradiate the measurement portion with laser light from an oblique side of the imaging optical system, so that a blind spot that cannot be measured does not occur in the shaded portion of the irradiation light. For this reason, it is excellent in the shape measurement of a complex measurement object with unevenness such as electronic parts integrated with high density.
[0025]
In addition, the optical projection optical system for irradiating the laser beam from the oblique side of the imaging optical system is not required unlike the light cutting method, so that the measuring device can be made compact, and a tertiary to follow the work path. It is suitable for mounting on a robot as a visual sensor for original shape measurement.
[Brief description of the drawings]
FIG. 1 is a vertical sectional view showing an example of a measurement object and an imaging optical system according to the present invention.
FIG. 2 is a side view (a) showing a relationship between a measurement target and a series of measurement cross sections, and a plan view (b) showing a relationship between the measurement target and a series of detected contour lines.
FIG. 3 is a flowchart showing a flow of data processing from imaging of a measurement cross section to obtaining a shape measurement value of a measurement object.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Measurement object 2 Imaging lens 3 Two-dimensional imaging device 4 Optical axis 5 of imaging lens 5 Circular light source 6 Carrier 7 Light-receiving surface 8 Measurement cross section 9 Contour line

Claims (2)

In a 3D object shape measurement method that measures the surface shape of an object in a non-contact manner using an image of the object that exists in the 3D space, the plane that connects the images of the measurement cross section that obliquely intersects the optical axis of the imaging lens In addition, a light receiving surface of a two-dimensional image sensor in which the photoelectric conversion elements are arranged close to each other and are arranged in two dimensions with approximately equal pinches in each dimension is used to obtain an image of the object, and the focused image is obtained from the object image. Non-contact three-dimensional measurement, wherein a three-dimensional shape of an object is measured by extracting a focus image area and detecting a contour line of the object formed by intersecting the measurement cross section and the object surface from the focused image area Object shape measurement method In a three-dimensional object shape measurement device that measures the surface shape of an object in a non-contact manner using an image of the object existing in the three-dimensional space, the surface that connects the images of the measurement cross section that obliquely intersects the optical axis of the imaging lens in, and a photoelectric conversion element adjacent to each other, crossing the optical axis of each dimension with an imaging device having a light receiving surface of the two-dimensional imaging elements arranged two-dimensionally on a large substantially constant pitch, the imaging lens of the object surface by the measuring section A non-contact three-dimensional object shape measurement comprising: a scanning mechanism for scanning in a moving direction; and an image processing device that detects an outline of an object formed by intersecting a measurement cross section and an object surface from an image of the object apparatus

Images