JP2000146546A - Method and apparatus for forming three-dimensional model - Google Patents

Abstract

PROBLEM TO BE SOLVED: To expedite work of grouping a large number of data as three- dimensional information by grouping distance data for each structure, on the basis of color data and the distance data. SOLUTION: An apparatus 1 which forms a three-dimensional model from distance data obtained by measuring a plant structure, etc., is provided with a matching processing part 4, a color data discriminating part 5, etc. A distance data obtained by measurement by a laser scanning range finder, and a color picture image by a digital camera, for example, are sent to the processing part 4, and stored in a data memory 7 as distance data which correspond to the addresses of the pixels of the picture image. While, color picture images of a frame memory 3 are grouped by color differences and brightnesses by the discriminating part 5. Pixel addresses and distance data are arranged in order in each group by collating this data with the distance data being stored in the data memory 7, and modeling into three-dimensional solids is performed on the basis of the arranged distance data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for measuring a plant structure or the like with a distance measuring device such as a laser scan distance meter, and generating a three-dimensional model from the measured distance data, and an apparatus therefor.

[0002]

2. Description of the Related Art Three-dimensional information of a shape of a measurement object is obtained by contacting and non-contacting various shapes of the measurement object. The object to be measured is photographed from multiple viewpoints, and each 3
By appropriately combining the dimensional information, a three-dimensional model of the entire measurement target can be generated. In the measurement using a laser scan distance meter, a large amount of data of tens of thousands to several millions or more is obtained as three-dimensional information, and the work of grouping data for each structure to be measured is manually performed in an interactive manner with a computer. Going by work.

[0003]

The above-described grouping of distance data is performed by the operator manually specifying a group for each structure on the monitor for each distance data. However, the distance data to be handled requires a large number of measurement points when trying to express a wide range in detail, and the work requires a great deal of labor and time.

Accordingly, an object of the present invention is to quickly perform a work of grouping a large amount of data as three-dimensional information.

[0005]

In order to achieve the object of the present invention, an apparatus according to the present invention is a three-dimensional model generating apparatus, comprising: means for measuring distance data of a measuring object; means for photographing color data; Means for associating data with color data, means for grouping distance data by color data,
Means for deleting distance data that cannot be grouped by color data, or adding distance data to groups that match the grouped distance data in the vicinity, and means for fitting to a three-dimensional basic solid such as a plane or a cylinder for each group And is grouped by color data and distance data for each structure.

[0006]

Embodiments of the present invention will be described below with reference to the drawings.

The first embodiment is as follows.

FIG. 1 shows a three-dimensional model generating apparatus 1 according to the present invention.
FIG. FIG. 2 shows a conceptual diagram of the three-dimensional model generation processing.

A laser scanning rangefinder 20 for obtaining distance data necessary for generating a three-dimensional model is combined with a digital camera 21 for obtaining color data. A polarizing filter may be attached to the camera lens in order to avoid input of reflected light from the metal part.

A digital camera having a viewing angle wider than that of a laser scanning distance meter and having a number of pixels equal to or greater than the number of measuring points of the laser scanning distance meter is used.

At the time of measurement, a frame in which the laser scan distance meter and the digital camera are combined is fixed by a tripod 22 or the like so that the laser scan distance meter and the digital camera do not move during the measurement.

When the object to be measured is, for example, a wall and a pipe, the distance data 2 measured by the laser scan distance meter 20 is used.
3 and a color image 24 taken by the digital camera 21
Is grouped by color data for each structure such as a wall and a pipe.

For each grouped distance data, a cylinder 2
5. A three-dimensional solid that best fits by fitting to a three-dimensional solid such as a rectangular parallelepiped 26 and a conical column 27 is selected and modeled. FIG. 3 shows a three-dimensional model generation processing flow. First, distance data is acquired by a laser scan distance meter to measure the relative position of the plant structure (Step 10),
A color image is photographed by a digital camera or the like (step 11).

The acquired data of the laser scanning distance meter is stored in a distance data storage unit 2, and the color image is stored in a frame memory 3.

Before the grouping of the distance data by the color data, the stored distance data is associated with the color data of the color image by the matching processing unit 4 (step 1).
2).

FIG. 4 shows the optical axis center P of the laser scanning distance meter.
_O , viewing angle θY in Y direction, optical camera center C of digital camera
The relationship between _O and the viewing angle θcY in the camera Y direction is shown.

The association of data is performed based on the relationship between the number of pixels of the image, the length per pixel, and the distance data in the X, Y, and Z directions.

First, the length Lc per pixel of the color image represented by Formula 1 is measured by the laser scan distance meter shown in FIG. 4, the distance data L in the Z direction at the center P, the viewing angle θcY in the camera Y direction, and the image. From the number n of pixels in the Y direction.

[0019]

(Equation 1)

Lc: Length per pixel (mm) L: Distance of center point P in Z direction (mm) θcY: Vertical angle of view of digital camera n: Number of pixels of color image in vertical direction as a pretreatment correspondence of each data, align the optical axis center C _O of the optical axis center P _O and the digital camera of the laser scanning rangefinder.

The number of pixels on the image between C _{O and} P _O corresponds to the distance L _{O of the} center of the optical axis when the laser scan rangefinder 20 and the digital camera 21 are combined, and the length Lc per pixel. And adjust the optical axis. If the optical axes are aligned, the association of each data can be performed based on the X and Y distance coordinates of each distance data and the length per pixel of the image.

The distance data corresponding to the color data Pi is obtained by Expression 2, and the distance data corresponding to the distance coordinates obtained by the mathematical expression can be extracted from all the distance data to associate the data.

[0023]

(Equation 2)

Pi: image address (Pi (X), Pi)
(Y)) Color information distance data L: distance in the Z direction of center point P (mm) θcX: horizontal angle of view of digital camera θcY: vertical angle of view of digital camera m: horizontal direction of color image N: the number of pixels in the vertical direction of the color image Pi (X): the address in the horizontal direction of the color image Pi (Y): the address in the vertical direction of the color image The corresponding distance data is stored in the data storage unit 7.

Next, the distance data is grouped using the color image stored in the frame memory 3 and the distance data stored in the data storage unit 7 (step 13).

FIG. 5 shows a processing flow for grouping distance data. The area is divided by setting a grouping range based on the color difference of the color data (step 20), and the divided areas are further subdivided by luminance (step 21).

First, image data is grouped by color difference. The grouping process based on the color difference is performed according to the procedure shown in FIG. The color image in the frame memory 3 is stored in the color data
To extract color difference information from the color image using a YUV signal which is a video format such as a TV or VTR (step 31).

The color difference can be represented by the difference between the U density value and the V density value. The difference between all U density values and V density values in the image is calculated (step 32), the vertical axis represents the number of pixels, and the horizontal axis represents the U density value and V density.
A histogram is created as the difference between the density values (step 3
3).

If the structure has three colors, for example, the histogram forms a peak for each color, as shown in FIG.

As a method of dividing data, U density value and V
The number of pixels is counted in the direction from the minimum value of the density value to the maximum value, and the range in which the number of pixels to be counted becomes 0 is regarded as one group, and the counted range is set as a grouping range by color.

Next, the movement is performed until the number of pixels to be counted becomes 0 or more, and similarly, a grouping range by color is set (step 34).

This process is repeated until the position of the maximum value is reached.

In the range of the counting, the difference between the U density value and the V density value is considered to have a small width and the number of pixels is assumed to be small. If the number of counted pixels in the set grouping range is extremely small, it is not determined as a plant structure and is not set as a grouping range.

The area of the color image is divided according to the set grouping range (step 35), and each area is stored in the frame memory 3.

The difference in brightness and the monochrome color (white to gray to black) due to the difference in the arrangement of the structures cannot be distinguished from the color difference. Since the brightness and the monochrome color can be determined by the luminance, the groups are further subdivided by the luminance in each area. The subdivision processing based on the luminance is performed according to the procedure shown in FIG.

First, Y corresponding to luminance information from the YUV signal
A luminance value is extracted (step 40), and a histogram is created with the number of pixels on the vertical axis and the Y luminance value on the horizontal axis as in the case of grouping by color difference (step 41).

When the color of the structure is, for example, white and gray,
The histogram forms a peak as shown in FIG. Gray has a lower luminance value than white, so white has a higher luminance value, and white has a higher luminance value. The grouping range of the data is set in the same manner as the grouping based on the color difference (step 42).

When the number of counted pixels in the set grouping range is extremely small, it is not determined as a plant structure and is not set as a grouping range.

The image of each area is subdivided according to the set grouping range (step 43), and each area is numbered.
The addresses of all the pixels in the area are stored in the group data storage unit 6.

In the data storage unit 7, the stored pixel data and the distance data are arranged for each group by associating the stored distance data with the group data.

When matching the distance data with the group data, a distance interval between a plurality of nearby distance data is calculated, compared with other nearby distance data intervals, and eccentric distance data is deleted. The distance data that is not grouped is also deleted.

The rearranged distance data is transmitted to the fitting processing unit 8 as distance data of each group, and fitting to a three-dimensional solid is performed based on the distance data (step 1).
4).

The three-dimensional solid is a three-dimensional shape such as a cylinder or a rectangular parallelepiped. FIG. 10 shows the fitting process.

For example, in the case of the distance data 29 of the pipe 28 as shown in FIG. 11, distance data of a small small area such as the distance data 30 on the front left and the distance data 31 on the right back is extracted (step 60). Check if it fits in a 3D solid.

The fitting was performed as shown in FIG.
While changing the size, position, etc. of the three-dimensional solid 32, the least-squares difference method is used to minimize the sum of the squares of the distance d between each distance data and the three-dimensional solid position corresponding to the position, and the size, position, etc. Is determined (step 61).

The axial direction is obtained from the center of the three-dimensional solid determined by fitting in a small area at a distance (step 62). The information of the determined fitting is transmitted to the model generating unit 9, and as shown in FIG. 13, a triangle is formed from the distance data to create a surface, and modeled for each group (step 15). The method of generating the Delaunay net may be applied to the creation of the surface.

Next, a second embodiment will be described below.

The second embodiment of the present invention is an example in which the first embodiment is partially changed. The changed portions will be described below, and the same portions as those in the first embodiment will be omitted.

FIG. 14 shows a flow chart of the distance data grouping process. An area is divided by setting a grouping range based on the color difference of the color data (step 70), and the divided areas are further subdivided by luminance (step 71).

The distance data that is not grouped by the color data is added to a group to which it belongs based on the distance from the nearby grouped distance data.

FIG. 15 shows a procedure for determining to which group the distance data not grouped by the color difference and the luminance belong.

The group data in the vicinity of the ungrouped distance data is searched (step 50), and the distance data between the ungrouped pixel address and the grouped distance data in the vicinity is used to determine the distance data. Is calculated (step 51), and the ungrouped data is added to the shortest distance data group (step 52).

The range around the pixel address used here is:
Since it is conceivable that a large amount of ungrouped data is distributed in the vicinity, the range is increased from 3 × 3 pixels to 5 × 5 pixels in order to confirm that there are at least two or more grouped data. And

In order to facilitate visual judgment of the structure, the following contents are desirably provided.

That is, a triangular surface is generated from the measured distance data by connecting the distance data as shown in FIG. 13, and the surface is colored with color data corresponding to the three vertex distance data.

Although it is difficult to visually judge a structure with plane data simply generated from distance data, it is possible to generate a model in which a structure can be easily visually judged by coloring the plane data.

Further, preferably, when synthesizing each data, it is preferable that the self-position of the measuring device of the data can be recognized.

FIG. 16 shows the measuring device 33. When measuring color images and distance data from multiple different viewpoints,
Each measurement data must be synthesized.

At the time of synthesis, information on the position and angle of the measuring device 33 is required, and a self-position detecting sensor 34 is attached to the measuring device 33 in order to obtain information on the position and angle of the measuring device 33.

The provision of the self-position detecting sensor 34 enables the automatic synthesis of data.

The distance data, the image data, and the self-position information are recorded in the memory 35, and each data is synthesized.

When the accuracy of the self-position sensor 34 is not the accuracy required for the synthesis, the measurement ranges are overlapped, the information of the self-position sensor is used for rough positioning, and the positioning of the details is not overlapped. Optimal fitting is performed based on the range color data and distance data, and the respective data are synthesized. The self-position distance may be obtained from a signal from an acceleration sensor, a gyro, or a GPS.

The third embodiment will be described below.

This embodiment is an example in which the first embodiment is partially changed. The changed portions will be described below, and the same portions as the first embodiment will be omitted. FIG. 1 shows the structure of the apparatus of this embodiment.
FIG.

The same filter as that of the light receiving section of the laser scan distance meter is mounted on the lens of the digital camera. For example, when the wavelength of a laser beam is 680 nm, only a laser beam is displayed as an image of a digital camera by attaching a filter that passes only that wavelength.

FIG. 18 shows a processing procedure for detecting the measuring point and the color data of the color image in correspondence with each other. First, an image is captured by a digital camera without installing a filter, and stored in the frame memory 3 (step 80).

Next, a filter is mounted, and distance data is measured by a laser scanning range finder. At the same time when a laser beam hits a measuring point, an image is taken in from a digital camera.

The speed of capturing the image of the digital camera and the speed of capturing the distance data of the laser scan rangefinder are adjusted in advance so as to be linked. The captured distance data is
The color image is transmitted to the distance data storage unit 2, and the color image is transmitted to the image processing unit 10 (step 81).

The color image is binarized by the image processing unit 10 at a predetermined threshold value (step 82), the binarized image is temporarily stored, and the next transmitted image is binarized with the binarized image. OR the temporarily stored image (step 8
3), temporarily store the result.

In the binarization, the threshold value is set in advance so that the laser beam becomes white and the background becomes black. This process is repeated until the measurement by the laser scanning rangefinder is completed (step 8).
4).

Since the image of the logical sum is an image in which the measurement points of the laser scan distance meter are plotted, the color image stored in the frame memory 3 is read (step 85), and the two images are operated between the images. Thus, the color data corresponding to the measuring point can be obtained (step 8).
6).

The color data and the distance data corresponding to the measuring point are transmitted to the matching processing unit 4 to associate the distance data with the color data.

The subsequent processing is the same as in the first embodiment.

[0074]

According to the present invention, grouping a large amount of distance data conventionally requires an operator to manually specify a group for each structure on a monitor for each distance data, which requires labor and time. In addition, it is possible to provide a method and apparatus for generating a three-dimensional model capable of reducing work by automatically grouping distance data based on distance data and color data. Also, in the invention in which the distance data is colored,
The effect of being able to display visually intelligibly is obtained.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing a device configuration of the present invention.

FIG. 2 is an explanatory diagram illustrating the concept of a three-dimensional model generation process according to the present invention.

FIG. 3 is an explanatory diagram showing a three-dimensional model generation processing flow of the present invention.

FIG. 4 is an explanatory diagram showing correspondence between distance data and color data.

FIG. 5 is an explanatory diagram showing a grouping process using color data according to the present invention.

FIG. 6 is an explanatory diagram showing a grouping process based on color difference according to the present invention.

FIG. 7 is an explanatory diagram of a graph obtained by the color difference histogram processing of the present invention.

FIG. 8 is an explanatory diagram showing a grouping process based on luminance according to the present invention.

FIG. 9 is an explanatory diagram of a graph obtained by the luminance histogram processing of the present invention.

FIG. 10 is an explanatory diagram showing a fitting process of the present invention.

FIG. 11 is an explanatory diagram showing regions used for fitting processing of the present invention.

FIG. 12 is an explanatory diagram showing details of fitting processing of the present invention.

FIG. 13 is an explanatory diagram showing modeling of the present invention.

FIG. 14 is an explanatory diagram showing a grouping process using color data and distance data according to the second embodiment of the present invention.

FIG. 15 is an explanatory diagram showing a grouping process based on distance data according to the second embodiment of the present invention.

FIG. 16 is an explanatory diagram showing a configuration of a measuring device of the present invention.

FIG. 17 is an explanatory diagram showing a device configuration of a second embodiment of the present invention.

FIG. 18 is an explanatory diagram showing a procedure for detecting the measuring point and the color data of the color image in correspondence with each other according to the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... 3D model generation apparatus, 2 ... distance data storage part, 3
... frame memory, 4 ... matching processing section, 5 ... color data identification section, 6 ... group data storage section, 7 ... data storage section, 8 ... fitting processing section, 9 ... model generation section, 1
0: Image processing unit, 20: Laser scan distance meter, 21:
Digital camera, 22 ... tripod, 23 ... distance data, 24
... Color image, 25 ... Cylinder, 26 ... Rectangle, 27 ... Conical column, 28 ... Pipe, 29 ... Pipe distance data, 30 ... Left front distance data, 31 ... Right back distance data, 32 ... Three-dimensional solid, 33: measuring device, 35: memory.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Koji Kikuchi, Inventor 3-1-1, Sachimachi, Hitachi-City, Ibaraki Prefecture Inside the Hitachi Works, Hitachi, Ltd. (72) Keiji Tanaka 3-1-1, Sachimachi, Hitachi-City, Ibaraki Prefecture No. 1 F-term in Hitachi, Ltd. Hitachi Plant (reference) 2F065 AA04 AA53 BB06 BB07 DD06 JJ03 JJ26 LL04 LL22 MM16 QQ04 QQ24 QQ27 QQ38 QQ43 UU05 5B046 AA02 FA16 5B057 CA01 DA12 DB06 DB08 DC19

Claims (6)

[Claims] 1. An apparatus for measuring the relative distance of an object to be measured and a distance image obtained by an apparatus for photographing a color image of a structure and color data of a color image, based on a positional relationship between the two apparatuses and a viewing angle. The distance data is grouped for each structure based on the color data or the color data and the distance data, and each group is fitted to a three-dimensional basic solid such as a plane or a cylinder, and is modeled for each group 3D model generation method. 2. The method according to claim 1, wherein the distance data of the object to be measured is grouped for each structure, the distance data is grouped for each structure based on the color data, and the distance cannot be grouped by the color data. A method of generating a three-dimensional model, wherein data is deleted or distance data is added to a group having the best distance data by comparison with nearby grouped distance data. 3. The distance data obtained by a device for measuring the relative distance of an object to be measured and the color data of a color image obtained by a device for photographing a color image of a structure, and the color data of the color image are obtained based on the positional relationship between the two devices and the viewing angle. And a three-dimensional model generation method of generating and modeling plane data groups having the colored distance data groups as vertices. 4. A means for measuring distance data of a measuring object, a means for photographing a color image, a means for storing distance data,
Means for storing a color image, means for associating distance data with color data of a color image, means for grouping distance data by color data or color data and distance data,
A three-dimensional model generating apparatus comprising means for fitting each group to a three-dimensional basic solid such as a plane or a cylinder and modeling each grouped data. 5. A three-dimensional model generating apparatus according to claim 4, wherein: means for measuring distance data of a measuring object; means for photographing a color image; means for storing distance data; means for storing a color image; Means for associating color data of a color image; means for grouping distance data by color data; deleting distance data that cannot be grouped by color data; or adding distance data to a group that matches the distance data of a neighboring group 3. A three-dimensional model generating apparatus, comprising: means for fitting a three-dimensional basic solid such as a plane or a cylinder for each group, and modeling each grouped data. 6. A means for measuring distance data of a measurement object, a means for photographing a color image, a means for storing distance data,
A three-dimensional model comprising: means for storing a color image; means for associating distance data with color data of a color image; and means for generating and modeling plane data groups having the colored distance data groups as vertices. Generator.