JP2000074643A - Method and device for converting three-dimensional distance data and storing medium storing program for data conversion - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and device for converting three-dimensional distance data by which measuring point data composed of the arrangement of distances to an object to be measured are converted into curved surface data defined by a curved surface. SOLUTION: A device for converting three-dimensional distance data is provided with a picture inputting section 11 to which three-dimensional distance data 18 composed of the distance information to an object to be measured from the position corresponding to each picture element of a picture and stored in each picture element and a picture 19 taken from the object at the same visual point as the data 18 are inputted and a picture processing section 12, 14, and 16 which converts the data 18 into a plurality of surface data based on the picture 19 inputted to the inputted section 11. The picture processing section is provided with a dividing means 12 which divides the picture 19 into each characteristic area in accordance with the features of the information in the picture elements, an assigning means 12 which assigns the picture elements of the three-dimensional distance data corresponding to the picture elements of the picture 19 divided by means of the dividing means 12 to each characteristic area, and a generating means 16 which generates surface data at every characteristic area based of the distance information of the three-dimensional distance data assigned by means of the assigning means 14.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a method and an apparatus for converting three-dimensional distance data, and more particularly, to a method and an apparatus for converting distance data into plane data. The invention further relates to a software product for performing such a data conversion.

[0002]

2. Description of the Related Art Conventionally, a non-contact three-dimensional measuring machine has been developed and widely used. The non-contact coordinate measuring machine includes a laser light source and a CCD camera, and obtains a linear distance from a viewpoint, a vertical distance from a focal plane, and the like. This non-contact CMM can almost automatically measure the distance to the object in a short period of time, and has many points to measure. Measurement can be performed.

[0003]

However, CAD data is obtained from three-dimensional distance data which is an array of distances to an object plane.
There is an inconvenience that it is difficult to create curved surface data. Further, in order to perform the FEM analysis, it is necessary to divide the CAD data into meshes. Therefore, the CAD data must be created first. However, if a prototype already exists as a real product, it is preferable to obtain shape data using a three-dimensional scanner rather than inputting the data using CAD software. However, it is difficult to obtain CAD data of an object to be measured using a coordinate measuring machine.

[0004]

An object of the present invention is to improve the disadvantages of the prior art and, in particular, to convert three-dimensional measurement point data consisting of an array of distances to a measuring object into curved surface data defined by a curved surface. It is an object of the present invention to provide a method and apparatus for converting distance data.

[0005]

Therefore, according to the present invention, a first input to which three-dimensional distance data in which the distance information to the object to be measured at a position corresponding to each pixel of an image is stored in each pixel is input. A second input step of inputting, before and after the first input step, the three-dimensional distance data and a captured image of an object to be measured with the same viewpoint. Moreover, a division step of dividing the captured image input in the second input step into regions for each characteristic according to the characteristics of information in the pixels of the captured image, and a captured image divided by the division step Allocating pixels of three-dimensional distance data corresponding to the pixels to regions of each feature, and generating surface data for each of the characteristic regions based on the distance information of the three-dimensional distance data allocated by the allocating process And a generation step. This aims to achieve the above-mentioned object.

In the present invention, first, in a dividing step, each pixel of a captured image is grouped for each characteristic region. The captured image may be a grayscale image captured by a monochrome CCD camera or a color image captured by a color CCD camera. In the case of a grayscale image, a characteristic region is defined for each density width. In the case of a color image, a characteristic region is generated for each color, and pixels are preferably divided into the respective regions. Also,
The shading and the color may be separately colored in advance for each surface of the object to be separated.

When the division of the gray-scale image is completed, the same division is performed on the three-dimensional distance data according to the relationship between the position of each pixel of the gray-scale image and the characteristic region to which it belongs. That is, in the assigning step, each pixel of the three-dimensional distance data is assigned to the characteristic region to which each pixel of the corresponding grayscale image belongs. Then, by making certain characteristics such as shading and color of the measurement object different for each surface, the distance information of each pixel of the three-dimensional shape data can be divided and assigned to each surface of the measurement object. it can. And the generation process is
The distance information grouped for each surface is output as one surface.

[0008]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a three-dimensional distance data converter (image processing computer) according to the present invention. As shown in FIG. 1, the three-dimensional distance data conversion device includes, as shown in FIG. 1, three-dimensional distance data stored in each pixel and distance information to a measurement target at a position corresponding to each pixel of the image, An image input unit 11 to which a captured image 19 obtained by capturing an object to be measured with the same viewpoint is input.
And image processing units 12, 14, and 16 for converting three-dimensional distance data into a plurality of plane data based on a captured image input to the image input means.

The image processing unit includes a dividing unit 12 for dividing the captured image 19 into regions for each feature in accordance with the characteristics of information in the pixels, and corresponding to the pixels of the captured image divided by the dividing unit 12. Allocating means 14 for allocating the pixels of the three-dimensional distance data to the area for each feature, and generating means for generating surface data for each of the characteristic areas based on the distance information of the three-dimensional distance data allocated by the allocating means 14 16 are provided.

This will be described in detail. A configuration for inputting the captured image and the distance image to the image processing computer 10 is provided at a stage preceding the image processing computer 10. In the embodiment shown in FIG. 1, a light source 6 that irradiates a laser beam to a measurement target, a CCD camera 5 that captures an image of the measurement target 1, and measurement at each pixel based on an image captured by the CCD camera 5 A distance calculation computer 9 for calculating a distance to an object, a CCD camera 5 and a light source 6
And a control computer 7 for controlling

In order to obtain a distance image, for example, a method of calculating a distance to a measurement object based on an image obtained by irradiating the measurement object with slit light has already been realized. The control computer 7 controls, for example, blinking of the light source 6 and scanning of the laser beam. The distance calculation computer 9 calculates the distance from the image of the measurement target imaged by the CCD camera, for example, a plurality of slit images to the measurement target, for each pixel.

In the example shown in FIG. 1, the surface of the object to be measured is color-coded for each surface. In this method, when the CAD data is used, each face constituting the face data is painted separately. Then, the captured image and the distance image input to the image processing computer shown in FIG. 1 are as shown in FIG. As shown in FIG. 2B, here, the captured image is an RGB image.
Is provided for each pixel. For this reason,
The captured image is a color image 19.

As shown in FIG. 3, the distance image 18 includes distance data r ₁₁ from the object to be measured to each pixel of the CCD camera.
~ R _nm is stored. This distance data r ₁₁ ~r _nm is,
The distance from the focal point C of the light beam (straight line) passing through each pixel (pixel) to the object. Therefore, if the angle of view of the camera and the position and orientation of the camera in the world coordinate system are known, the three-dimensional coordinate value of the object in the world coordinate system can be calculated from the distance data. Let the position vector of the camera in the world coordinate system be C
_Assuming that the direction of the light beam passing through the designated pixel r _ij is vector V and the distance is r, the coordinate value P of the measurement point data in the world coordinate system is expressed by the following equation.

P = C + V · r

Therefore, the position of the surface of the object in the world coordinate system can be calculated from the distance coordinates. If a continuous surface is cut out in this coordinate system, the data can be used as CAD data, but it is difficult to determine how far the surface should be. FIG.
In the example shown in (1), portions to be divided as planes in the CAD data are colored in different colors in advance, and pixels belonging to each plane are first grouped in a normal captured image. That is, FIG.
May be provided with a function of grouping pixels of a captured image by color of the captured image.

By dividing pixels according to colors in this way,
The division of each surface becomes clear, and the surface to which each pixel belongs is uniquely determined. In the present embodiment, since the three-dimensional distance image and the color image are captured from the same viewpoint and at the same size, the correspondence between each pixel and the position of the surface of the measurement target is determined by the three-dimensional distance image. Match between image and color image. Therefore, the surface to which each pixel belongs is determined using a color image, and each pixel of the three-dimensional distance image is grouped using this relationship. Can be split. When a surface that connects the coordinates of the divided pixels in the world coordinates is generated, the three-dimensional range image of the measurement target can be converted into CAD data.

When a color image is grouped for each color, the area of the non-measured object painted with the same color does not always have the same RGB value on the color image due to the influence of illumination unevenness or the like. Therefore, it is preferable to perform chromaticity conversion on a color image. In this case, the dividing means 12
The function of normalizing the brightness of the captured image and converting it to an image with only a constant shade of light, the function of mapping the image converted by this function to the color feature space, and the position in this feature space as the feature area And a setting function. With these functions, if the brightness is normalized to make the image only a hue of a constant brightness, the grouping can be performed more accurately.

As shown in FIG. 5, the grouping of the color image is performed by mapping each pixel of the color image to the color feature space shown in FIG. 5B. By performing grouping with a certain width in the feature space, color unevenness due to a change in the imaging environment can be absorbed, and pixel grouping can be performed with high accuracy.

In addition to the method shown in FIG. 5, there are various methods for grouping color images, such as color distance conversion and binarization, multidimensional slicing, maximum likelihood method, and clustering (K-means algorithm). The captured images may be grouped by using these color extraction techniques.

In the example shown in FIG. 1, the measurement object is colored in advance. However, if the measurement object is relatively simple in shape than the measurement object shown in FIG. Pixels appearing on the surface of the target object may be classified to group the pixels.

After each pixel of the captured image is mapped to the feature space, the same mapping as this mapping is performed for the three-dimensional range image. Thus, the three-dimensional range image is grouped into each surface of the measurement target. The three-dimensional distance image and the captured image
As long as the relationship between the focus and the size of imaging is determined, the resolution may be different or the imaging range may be different. The relationship between the captured image and the three-dimensional distance image may be any as long as the grouping having the same contents as the pixel grouping in the captured image can be performed.

FIG. 6 is an explanatory diagram showing an example in which each surface of the object to be measured is grouped by extracting a boundary line of the object to be measured. In this embodiment, the dividing means 12 has a function of extracting a boundary line of a captured image and a function of setting a region surrounded by the boundary line extracted by this function as a characteristic region. The boundary of the captured image is, for example, a solid line 21 shown in FIG. This boundary line is a line where the surfaces of the object to be measured are in contact with each other. Although there are various methods for extracting such a boundary line, in the example shown in FIG. 6, the object to be measured is colored with a different color for each surface, and this is imaged with a color CCD camera, and the color CCD camera is used. A boundary line of the measurement object is extracted by extracting pixels having a difference. The relationship between colors and hatching in the diagram shown in FIG. 6A is the same as that shown in FIG.

In order to extract a boundary line from a color image, the color image may be grouped for all pixels and binarized for each group to extract the boundary line. in this case,
The pixels included in the red group are binarized as 1 and the others are binarized as 0, and an outline is extracted, and this is defined as a boundary line. Alternatively, a boundary line may be extracted by performing an edge detection process called a Sobel operator. This is to focus on a certain pixel, differentiate it from its neighborhood, and extract the edge where the color changes rapidly. Further, in addition to the method of differentiating a color image, a method combining the differentiation of an image with lightness and the differentiation of a saturation / hue image may be employed.

FIG. 7 shows that when the distance image is coarser than the captured image, the distance image can be corrected with the accuracy of the boundary of the captured image. The extraction of this profile is shown in FIG. Generally, a non-contact three-dimensional measuring machine measures three-dimensional coordinates at discrete positions. However, the measurement points do not exist at all at random. For example, when slit light is used as a light source, the measurement points exist substantially on a straight line when viewed from the direction of the slit light, that is, the projection direction. Therefore, a three-dimensional boundary line can be generated from a two-dimensional boundary line using measurement point data that is substantially on a straight line when viewed from such a projection direction.

As shown in FIG. 7, first, a projection point is defined on a two-dimensional boundary line. The distance data corresponding to this projection point is extracted, and this is set as a measurement point. Subsequently, an extrapolation curve connecting the measurement points is generated. The point at which the intersection of the projection point and the boundary line is projected onto the extrapolation curve is defined as a point on the three-dimensional boundary line. By repeating this processing, coarse distance information is corrected at the resolution of the two-dimensional boundary line, and CAD with high resolution is used.
Data can be generated.

Next, an image processing method for three-dimensional distance data according to the present embodiment will be described with reference to the flowchart of FIG. 8 or FIG. In the example shown in FIG. 8, the three-dimensional distance data conversion method is a first method in which three-dimensional distance data in which distance information to a measurement target at a position corresponding to each pixel of an image is stored in each pixel is input. A second input step in which an input step S1 and a captured image of the object to be measured in the same viewpoint as the three-dimensional distance data before and after the first input step S1 are input;
And an input step S2. The term “same viewpoint relationship” refers to a relationship in which the relationship between the position of the measurement object and the position of the pixel of the image is associated without any conversion or conversion. This relationship is established when the focus position and the range of photographing are the same. However, the relationship is not limited to this, and a case where one of the images enlarges the other image may be used.

The method according to the present embodiment further includes a dividing step S3 of dividing the captured image input in the second input step S2 into regions for each characteristic according to characteristics of information in pixels of the captured image. An allocating step S4 of allocating pixels of the three-dimensional distance data corresponding to the pixels of the captured image divided in the dividing step S3 to an area for each feature, and distance information of the three-dimensional distance data allocated in the allocating step S4 And a generation step S5 of generating plane data for each of the characteristic regions based on the

In the example shown in FIG. 8, when a captured image is divided, a characteristic region is defined for each pixel from the grayscale information as image information in the pixel or grayscale information for each RGB.
Then, the destination of the image information of each pixel is determined. On the other hand, in the example shown in FIG. 9, a boundary line that sorts each surface of the captured image is first extracted. Then, a characteristic region is defined based on this boundary line.

The example shown in FIG. 9 includes first and second input steps S1 and S2, and a boundary between pixels whose colors change in the captured image input in the second input step is defined as a boundary. A boundary line extraction step S13 for extracting as a line, and respective pixels of the three-dimensional distance data respectively corresponding to the characteristic regions divided by the boundary line extracted in the boundary line extraction step S13, are respectively classified according to the division of the boundary line. The method includes allocation steps S14 and S15 for allocating to a characteristic region, and a generation step S16 for generating plane data for each of the characteristic regions based on the distance information of the three-dimensional distance data allocated in the allocation step S15.

By converting the three-dimensional distance data into surface data for each surface of the object to be measured by the method shown in FIG. 8 or FIG. 9, the conversion process of the distance data into CAD data can be automated. The processing shown in FIGS. 8 and 9 can be realized by the image processing computer shown in FIG. In this case, the image processing computer includes a central processing unit, a main storage device, an auxiliary storage device,
8 and 9 in the auxiliary storage device.
And a data conversion program for executing the processing shown in FIG.

The image processing computer is an engineering workstation or a personal computer. In such a computer, an operating system that controls input / output and job of a program operates. The data conversion program is a program for executing each processing shown in FIG. 8 or FIG. 9 depending on various functions of the operating system. Therefore,
Each command included in the data conversion program includes an instruction for operating the central processing unit independently and an instruction for operating the central processing unit depending on other software such as an operating system.

Such a data conversion program is C
It can be supplied by being stored in a portable storage medium such as a D-ROM. Further, the data conversion program may be stored in the auxiliary storage device via a communication line such as a network.

[0034]

Since the present invention is constructed and functions as described above, according to this, in the allocating step, each pixel of the three-dimensional distance data is converted to a characteristic region to which each pixel of the corresponding captured image belongs. By assigning some characteristics such as shading and color of the measurement object to each surface, the distance information of each pixel of the tertiary distance data can be grouped for each surface of the measurement object. And the generating step outputs the distance information grouped for each surface as one surface, so that the surface data can be generated from the three-dimensional distance data using the captured image, 3D distance data obtained in a few seconds can be automatically converted to CAD data consisting of surface data, which eliminates the need to create CAD data and makes the measurement target complex CAD of good approximation even in some cases
Data can be generated, and then various applications based on the CAD data, such as analysis of strength by the finite element method, control of machine robots, manufacturing using light-cured resin, etc., are compared with those in the past. Converts unprecedented three-dimensional distance data that can be performed more easily and at high speed, and even if the measurement target has a complicated shape, the characteristics of the shape can be directly input as CAD data. A method and apparatus can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an example of an image input to the image processing computer shown in FIG. 1; FIG. 2 (A) is a diagram showing an example of a three-dimensional distance image; FIG. FIG. 3 is a diagram illustrating an example of an image (color image).

3A and 3B are explanatory diagrams showing the contents of the distance image shown in FIG. 2; FIG. 3A is a diagram exemplifying a distance r _ij from a focal point to a measurement object; FIG. 3B is distance information; FIG. 6 is a diagram illustrating a distance image which is a matrix of FIG.

4 is an explanatory diagram showing a relationship between the distance image shown in FIG. 3 and coordinates in a world coordinate system.

5A and 5B are explanatory diagrams illustrating an example in which captured images of a measurement target are grouped in a feature space. FIG. 5A is a diagram illustrating each pixel of the measurement target, and FIG. 5B is a diagram illustrating RGB. FIG. 5 is a diagram illustrating a feature space of FIG.

FIG. 6 is an explanatory diagram illustrating an example of an image when a boundary line is extracted from a captured image, and FIG. 6A is a diagram illustrating an example of a captured image that is colored in a different color for each surface in advance. FIG. 6 (B)
FIG. 7 is a diagram showing an example of extracting a boundary line from the image shown in FIG.

7A and 7B are diagrams illustrating an example in which a distance image is corrected based on a boundary line obtained from a captured image when the distance image is coarser than the captured image, and FIG. 7A illustrates a projection point on a two-dimensional boundary line; FIG. 7B is a diagram exemplifying a measurement point based on distance data corresponding to the three-dimensional boundary line projected from the intersection of the boundary line and the projection point by extrapolating a curve connecting the measurement points. It is a figure which illustrates a point (Z value).

FIG. 8 is a flowchart showing a first operation example of the configuration shown in FIG. 1;

FIG. 9 is a flowchart showing a second operation example of the configuration shown in FIG. 1;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Measurement object 3 Measurement table 5 CCD camera 6 Light source 7 Control computer 9 Distance calculation computer 10 Image processing computer (three-dimensional distance data conversion device) 11 Input unit 12 Division means 14 Assignment means 16 Generation means

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) 2F065 AA04 AA06 AA11 AA53 BB05 EE05 FF01 FF02 FF09 GG04 HH05 HH12 JJ03 JJ09 JJ26 NN02 QQ00 QQ03 QQ04 QQ13 QQ23 QQ24 QQ42 5B046 EA09 CB10 CA18 DB06 DB09 DC16

Claims (9)

[Claims] 1. A first input step in which three-dimensional distance data in which distance information to a measurement target at a position corresponding to each pixel of an image is stored in each pixel is input, and the first input step includes: A second input step of inputting before and after the three-dimensional distance data and a captured image of the object to be measured in the same viewpoint, and the imaging input in the second input step. A dividing step of dividing the image into regions for each characteristic in accordance with the characteristics of information in the pixels of the captured image, and a pixel of the three-dimensional distance data corresponding to a pixel of the captured image divided by the dividing step. Tertiary characterized by comprising: an allocating step of allocating to an area for each feature; and a generating step of generating plane data for each of the characteristic areas based on distance information of the three-dimensional distance data allocated by the allocating step. Former distance Over data conversion method. 2. A first input step in which three-dimensional distance data in which distance information to a measurement target at a position corresponding to each pixel of an image is stored in each pixel is input, and the first input step includes: A second input step of inputting before and after the three-dimensional distance data and a captured image of the object to be measured in the same viewpoint, and the imaging input in the second input step. A boundary line extraction step of extracting a connection of pixels whose colors in the image change as a boundary line, and the three-dimensional distance data respectively corresponding to the characteristic regions divided by the boundary line extracted in the boundary line extraction step Allocating each pixel to each characteristic region according to the division of the boundary line, and generating surface data for each of the characteristic regions based on the distance information of the three-dimensional distance data allocated in the allocating step. Three-dimensional distance data conversion method characterized by comprising a generation step of. 3. The three-dimensional distance data stored in each pixel with distance information to a measurement target at a position corresponding to each pixel of an image, and the measurement target is imaged in the same viewpoint as the three-dimensional distance data. An image input unit to which the captured image is input, and an image processing unit that converts the three-dimensional distance data into a plurality of plane data based on the captured image input to the image input unit. Dividing means for dividing the captured image into regions for each characteristic according to the characteristics of information in the pixels, and dividing the pixels of the three-dimensional distance data corresponding to the pixels of the captured image divided by the dividing means into Allocating means for allocating to an area for each feature, and generating means for generating plane data for each of the characteristic areas based on distance information of the three-dimensional distance data allocated by the allocating means. That the three-dimensional distance data conversion apparatus. 4. The three-dimensional distance data conversion device according to claim 3, wherein said dividing means has a function of grouping pixels of the captured image according to colors of the captured image. 5. A function for normalizing the brightness of the captured image to convert the captured image into an image having only a constant lightness, and a function for mapping the image converted by this function to a color feature space. And a function of setting a position in the feature space to the feature region.
The three-dimensional distance data conversion device as described in the above. 6. The dividing means has a function of extracting a boundary line of the captured image and a function of setting a region surrounded by the boundary line extracted by this function as the characteristic region. The three-dimensional distance data conversion device according to claim 3. 7. The three-dimensional distance data conversion device according to claim 6, wherein the dividing unit extracts a connection of pixels having a color difference in the captured image as the boundary line. 8. A storage medium storing a data conversion program for converting three-dimensional distance data using an arithmetic device, the data conversion program comprising: a command for operating the arithmetic device; A first reception command for receiving the three-dimensional distance data stored in each pixel with the distance information to the measurement target at the position corresponding to each pixel, and before and after this command, the same viewpoint as the three-dimensional distance data And a second reception command for receiving a captured image of the measurement object in relation to the image. The captured image input in response to the second reception command is provided with each characteristic according to the characteristic of information in a pixel. A division command for dividing the three-dimensional distance data corresponding to the pixels of the captured image divided by the division command into a region for each feature; A storage medium storing a data conversion program, comprising: a generation command for generating plane data for each characteristic region based on distance information of the three-dimensional distance data assigned by the command. 9. A storage medium storing a data conversion program for converting three-dimensional distance data using an arithmetic device, the data conversion program comprising: a command for operating the arithmetic device; A first reception command for receiving the three-dimensional distance data stored in each pixel with the distance information to the measurement target at the position corresponding to each pixel, and before and after this command, the same viewpoint as the three-dimensional distance data And a second reception command for receiving a captured image of the object to be measured in the relation, and a boundary line indicating a connection of pixels whose color changes in the captured image input by the second reception command. And extracting each pixel of the three-dimensional distance data respectively corresponding to the feature region divided by the boundary line extracted by the boundary line extraction command. And a generation command for generating surface data for each of the characteristic regions based on the distance information of the three-dimensional distance data allocated by the allocation command. A storage medium storing a data conversion program.