JPH06258048A - Object input device - Google Patents

Abstract

PURPOSE:To enhance the convenience of an object input device for taking in three-dimensional profile information of an object disposed in an observation space. CONSTITUTION:The objective input device comprises an observation means 1 which can measure at least the background distribution among, distance distribution, mass feeling distribution, and background distribution constituting the spatial information of an observation space, and an operating means 2 for configuring at least three-dimensional profile information of an object to be observed from spatial information. The observation means 1 comprises means for supporting the object and the supporting means is composed of such material as invisible for the observation means 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to CG and CAM, and more particularly to an object input device for automatically observing an existing object using a three-dimensional visual technique.

[0002]

2. Description of the Related Art Conventionally, as a technique for automatically obtaining three-dimensional information of a scene by a computer, a shape-from-
Various methods such as X, depth-from-focus, and stereo method have been proposed. These are called 3D vision techniques.

As the shape-from-X, for example, a method has been proposed in which the normal vector distribution of the surface of the object is obtained from the brightness gradient in the image obtained by photographing the object to restore the three-dimensional shape of the object. ing. However, this kind of method has to assume severe constraints on the target object and observation conditions, and is far from being generally available.

The depth-from-focus is a method of calculating a depth distance of a scene by taking a plurality of images while changing the focus adjustment of the lens for the same scene, obtaining the focus position where the focus is sharpest. Is.

The stereo method is a distance measuring technique to which the principle of triangulation is applied. The stereo method is roughly classified into a passive stereo method and an active stereo method.

The passive stereo method is to detect the same place in two images having different viewpoints taken for the same scene, and to detect the scene of the place based on the position of each of the places in each image. Is a method of calculating the spatial position in. If the same location is successfully detected, the spatial position can be calculated very easily, but it is difficult to automatically detect the same location because the scene is unknown. In particular, it is known that as the scene becomes more complicated, the number of wrong combinations detected as the same location increases, and the accuracy of measurement is significantly impaired. Further, since the detected location is limited to a location having an image feature, it is not possible to measure the distance of a location having a poor feature such as the inside of a surface painted in one color. It is mentioned as a problem.

The active stereo method was devised to solve these problems. In this method, one of the image input means is replaced with an optical pattern projection means, a pattern is forcibly generated on the surface of the object, and the image is captured by the remaining image input means. Since the pattern is known, it can be easily detected from the image, and the spatial position of the pattern can be easily calculated from the relationship between the projected pattern and the captured pattern. Further, since this method forcibly generates features even on a flat surface, it can be expected to function stably without depending on the scene. Therefore, the active stereo method has been put to practical use from an early stage, and many proposals have been made so far (Japanese Patent Laid-Open No. 61-31).
No. 905).

The principle of the simplest active stereo method using a laser spot as an optical pattern will be described with reference to FIG. The laser beam 62 emitted from the light source 61 travels straight until it reaches an object. If the object 63 is present on the optical path of the beam, the beam 62 produces a spot at the point 64 and forms an image at the point 68 on the imaging surface 67. If the object 65 exists instead of the object 63, the beam 6
2 generates a spot at the point 66, and the spot 6 on the imaging surface 67
9 is imaged. At this time, the beam 62 and the straight line 64-6
Obtaining the coordinate value of the intersection point of 8 gives the spatial position of the point 64,
The spatial position of the point 66 is obtained by finding the coordinate value of the intersection of the beam 62 and the line 66-69. This is the principle of coordinate calculation in the active stereo method.

By the way, in the stereo method regardless of whether it is passive or active, two image input means or a pattern projection means and an image input means need to be arranged at different positions. Can only be input at locations on the object that can be seen at the same time, and due to the arrangement of each means and the shape of the object surface, locations at which both can't be seen at the same time by accident remain as uninputtable locations. In the stereo method, such a place is called a blind spot. An example of this is shown in FIG. The laser beam 72 emitted from the light source 71 is a point 74 on the object 73.
Generate spots on. However, the spot is the object 7
3 is blocked by the point 75 and the point 7 on the imaging surface 76
Do not reach 7. As a result, since the image of the point 74 is not detected, the spatial position of the point cannot be calculated.

Further, in the active stereo method, the image input means must detect the pattern image reflected on the object surface, but a pattern image having a detectable brightness is formed on the object surface where the reflection brightness is extremely low. There is a case in which there is a portion that cannot be input even if it does not correspond to the above-mentioned blind spot. An example of this is shown in FIG. The laser beam 82 emitted from the light source 81 reaches the point 85 of the object 83 and the point 86 of the object 84. However, when the inclination of the surface of the object 83 at the point 85 is too large or the reflectance of the object 84 at the point 86 is too low, the brightness of the image formation at the points 88 and 89 of the imaging surface 87 may not be sufficiently high. is there. As a result,
It becomes impossible to find a spot image on the imaging surface,
Active stereo method fails.

Further, even when there is a through hole in the object, a pattern image cannot be obtained at that location because there is no object in the path of the beam, and it is detected as an input impossible location. This example is shown in FIG. The laser beam 92 emitted by the light source 91 does not reach either of the objects 93 and 94, therefore
No spot image is detected on the imaging surface 95.

As described above, the non-inputtable portion (hereinafter referred to as a missing point) is classified into three types, that is, a blind spot, a low reflection luminance portion, and a through hole. Particularly, the blind spot / low reflection luminance portion is the object surface. It can be seen that the through holes are non-object surfaces. At this time, if the missing point is the surface of the object, the input value of the missing point can be estimated by interpolating from the measurement results in the vicinity.
However, if the missing point is not the object surface, it is meaningless to estimate the value of the missing surface as the object surface. The conventionally proposed object input device by the stereo method, when a missing point appears, either estimates the input value as the object surface or does not estimate the input value as the non-object surface. Was only going. The reason why only such uniform processing is performed is that the conventional device cannot determine whether the missing point is the object surface or the non-object surface. That is, the conventional device has no means for correctly classifying missing points. As a result, the observation results were often incorrect.

On the other hand, when observing a real object in a place where gravity exists on the earth, the object must be held by some kind of supporting means.
In addition to the surface shape of the object, at least the shape of the object such as the supporting means that is not the object of measurement is mixed in the observation result. In the conventionally proposed object input device based on the three-dimensional visual technique, the existing range of the object can be set in advance, and only the shape information existing in the range is extracted and output. However, this range must be adjusted every time the setting of the input environment is changed, for example, by adjusting the height of the supporting means, which is a considerable burden on the user. As described above, the conventional device has no means for automatically and adaptively removing unnecessary information mixed in the information of the measurement target object without bothering the user.

Furthermore, it is not possible to input the shape of the entire object only by observing the object from one direction. In order to input the entire object, observation from at least a plurality of directions and integration of the plurality of observation results are required. In order to observe an object from multiple directions, a method of rotating the object to orient the object with respect to the observation means, or a method of observing the object from any direction using an observation means that can freely move around the object Has been proposed in the past. However, as described above, the object must be held by the supporting means at all times, and it is impossible to observe at least the contact portion between the supporting means and the object with the conventionally proposed object input device using the three-dimensional visual technique. there were. In other words, such an invisible part was inevitably present in the observation by the conventional device.

[0015]

As described above, the conventional object input device has problems such as insufficient support for missing points, complexity in eliminating unnecessary information, and existence of invisible parts. The present invention has been made in view of such problems, and an object thereof is to solve the above problems and provide an object input device with improved convenience.

[0016]

According to a first aspect of the present invention, there is provided an object input device in which an object to be observed is arranged in an observation space, and three-dimensional shape information or the like of the object to be observed is taken in. An observation means capable of measuring at least the background distribution among the distance distribution, the texture distribution, and the background distribution, and an operation means for constructing at least three-dimensional shape information of the observed object as object information related to the observed object from the spatial information. The observation means comprises a support means for supporting the object to be observed, and the support means is made of a material or structure invisible to the observation means. .

In a second aspect based on the first aspect, the observing means is capable of observing the object to be observed from a plurality of directions, and the computing means combines the observation results from the plurality of directions into one. The object information is constructed over the entire range of the observed object.

In a third aspect based on the first aspect, the observing means is capable of measuring a distance distribution and a background distribution as the spatial information, and the calculating means is an object region calculated from the background distribution. The three-dimensional shape information is constructed from the distance distribution contained within.

In a fourth aspect based on the first aspect, the observing means is capable of measuring a distance distribution and a background distribution as the spatial information, and the calculating means is the background distribution measured by the observing means. Object regions from a plurality of directions calculated for each are overlapped to generate a closed surface covering the space occupied by the observed object, and the three-dimensional shape information is obtained from the distance distribution included in the surface. To build.

In a fifth aspect based on the first aspect, the illumination means is arranged at a position facing the observation means, and the background distribution is generated by the illumination means illuminating from behind the observed object. The shadow image of the observed object is obtained, and the shadow area located at the center of the image is the object area.

In a sixth aspect based on the first aspect, the observation means can measure a distance distribution and a background distribution as the spatial information, and the distance distribution and the distance distribution are
It is a two-dimensional array image in which the value of each pixel is a three-dimensional coordinate value.

[0022]

[Operation] According to the object input device of the first invention, the observing means obtains at least the background distribution as the spatial information on the observation space, and the computing means at least as the object information on the observed object based on the spatial information. The three-dimensional shape information of the object is calculated, the output means outputs the object information, and in particular, since the supporting means included in the observing means is invisible, the area where the observed object exists can be easily estimated from the background distribution. It is possible. Then, it is possible to determine whether or not the missing point is the surface of the object based on the estimated object area, and to automatically and adaptively eliminate unnecessary information, and the observation means passes through the invisible support means. Then, it is possible to input the contact portion of the observed object with the supporting means.

According to the object input device of the second invention, the observing means is capable of observing the object to be observed from a plurality of directions, and the computing means combines the observation results from the plurality of directions into one. The object information is constructed over the entire range of the observed object. This makes it possible to easily calculate the three-dimensional shape information over the entire area of the observed object.

According to the object input device of the third invention, the observing means measures a distance distribution and a background distribution as the spatial information, and the calculating means puts them within an object area calculated from the background distribution. The three-dimensional shape information is constructed from the included distance distribution. This makes it possible to eliminate noise, which is information other than the object.

According to the object input device of the fourth invention, the observing means measures the distance distribution and the background distribution as the spatial information, and the computing means measures each background distribution measured by the observing means. The calculated object regions from a plurality of directions are overlapped to generate a closed surface that covers the space occupied by the observed object, and the three-dimensional shape information is constructed from the distance distribution included inside the surface. . This makes it possible to eliminate noise, which is information other than the object.

According to the object input device of the fifth invention, the illuminating means is arranged at a position facing the observing means, and the background distribution is generated by the illuminating means illuminating from behind the observed object. A shadow area of the observed object, which is located in the center of the image, is defined as the object area. Thereby, the peripheral shape of the object and the through hole can be accurately determined.

According to the object input device of the sixth aspect of the invention, the observing means can measure a distance distribution and a background distribution as the spatial information, and the distance distribution has three-dimensional coordinate values with each pixel value. Is a two-dimensional array image.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of an object input device according to the present invention will be described with reference to the drawings.

FIG. 1 is an overall configuration diagram of the object input device of this embodiment.

Reference numeral 1 in the figure is an observation unit for acquiring various information inside the observation space covering the observed object as spatial information, and 2 in the figure is a description of the observed object by processing the spatial information. Is a calculation unit for constructing as the object information, 3 in the drawing is a storage unit for storing and storing the spatial information, the object information and other temporary information, and 4 in the drawing is constructed. The object information stored in the storage unit 3 as CG
And an output unit for outputting to CAM. Further, the object information may be directly reproduced in this output unit. In addition,
In the present embodiment, the spatial information is a distance distribution, a texture distribution, and a background distribution, and the object information is mapping information indicating the correspondence between the three-dimensional shape information and the texture information of the observed object.

FIG. 2 shows a processing flow of this embodiment. Reference numeral 11 in the figure is a spatial information acquisition process, which is a process of acquiring the information inside the observation space as the spatial information by the observation unit 1. Reference numeral 12 in the figure is an object information construction process, which is a process for constructing the three-dimensional shape information and the texture information about the object to be observed and the correspondence information between the two as the object information by the calculation unit 2 as the space information. Reference numeral 13 in the drawing denotes an output process, which is a process for outputting the object information by the output unit 3.

FIG. 3 is a detailed configuration diagram of the observation unit 1 in this embodiment. Reference numeral 21 in the figure is an observed object. 2 in the figure
Reference numeral 2 denotes a pattern projection unit for projecting an optical pattern in the observation space. In the present embodiment, a laser beam is moved up and down in the observation space by a mechanism including a laser generation means and a plurality of rotating deflecting means such as mirrors. It adopts a method of scanning left and right. Reference numeral 23 in the figure is an image input unit for acquiring the brightness distribution in the observation space as an image, and is a color CCD.
It is realized by a camera.

Reference numeral 25 in the figure represents the object to be observed in the image input unit 23.
Is a front illuminating unit for illuminating from the direction facing the front side, and 26 is a rear illuminating unit having a screen for uniformly illuminating the observed object from the opposite direction to the image input unit 23.

Reference numeral 24 in the figure is a distance distribution acquisition unit, which analyzes the image obtained by photographing the spot image of the laser beam emitted by the pattern projection unit 22 with the image input unit 23 based on the active stereo method. However, the spatial position of the spot can be calculated.

FIG. 6 shows an operating state of the distance distribution acquisition unit 24. The front illumination 25 and the rear illumination 26 of FIG. 10 are turned off so that the reflected light of the laser can be easily observed. The laser beam is subjected to main scanning 101 and sub-scanning 102, and a quadrangular pyramid 10
A mesh-like sampling is performed inside 3. As a result, the obtained distance distribution becomes a two-dimensional data array having m × n elements. Each element of the array stores the three-dimensional coordinates of the surface of the observed object sampled by the laser beam emitted in a specific known direction corresponding to the element. In this specification, this array is called a distance distribution or a range image,
The elements of the array are called pixels.

Reference numeral 27 in the drawing is a texture distribution acquisition unit, which acquires a color image obtained by photographing the observed object illuminated by the front illumination unit 25 with the image input unit 23 as a texture distribution. Reference numeral 28 in the figure is a background distribution acquisition unit, which acquires a shadow image obtained by photographing the shadow caster of the observed object illuminated by the rear illumination unit 26 with the image input unit 23 as a background distribution. The operation states of the texture distribution acquisition unit 27 and the background distribution acquisition unit 28 are illustrated in FIGS. 7 and 8, respectively.

Reference numeral 29 in the figure denotes a transparent support portion on which an object to be observed can be placed and rotated. Reference numeral 30 in the figure is a control unit for organically connecting and controlling the above respective units. In addition, the acquired distance distribution, texture distribution, background distribution, and supporting unit 29.
Various information such as the rotation angle of the laser beam and the trajectory of the laser beam at the time when each pixel of the distance distribution is acquired are stored in the storage unit 3.

According to the observation unit 1 having the above-described structure, the object to be observed 21 is observed by turning the support unit 29 in various directions with respect to the pattern projection unit 22 and the image input unit 23. As a result, as many sets of distance distributions, texture distributions, and background distributions of the observed object as the number of the observation directions are acquired.

FIG. 14A is a conceptual diagram showing a part of the obtained distance distribution, FIG. 14B is a conceptual diagram showing a part of the texture distribution, and FIG. 14C is a diagram showing a part of the background distribution. It is a conceptual diagram,
(D) shows a part of the object information obtained from them.

FIG. 4 is a detailed block diagram of the arithmetic unit 2 in this embodiment. Reference numeral 41 in the figure denotes an object area extraction unit that extracts an area in which the observed object is reflected from the background distribution as an object area. As described above, since the support portion 29 that supports the object to be observed 21 is transparent, the shadow distribution of the object appears as if it were hollow in the center of the background distribution imaged during rear illumination. The object area extracting unit 41 can easily extract the object area by binarizing the background distribution and then extracting a continuous black pixel area existing in the center of the image. The extracted object area is stored in the storage unit 3 described above.

Reference numeral 42 in the drawing is a texture information extraction unit that extracts information of a portion overlapping with the mask from the texture distribution using the object area as a mask. As described above, since the background distribution and the texture distribution are captured by the same image input unit 23, both images can be superposed exactly on a pixel-by-pixel basis. The texture information extraction unit 42 extracts only the pixels in the texture distribution corresponding to the pixels in the background distribution existing inside the mask to obtain the texture information. As a result, unnecessary information unrelated to the observed object 21 existing in the texture distribution is deleted. The texture information is a color image captured as a texture distribution. In general, a color CCD camera outputs the brightness value of light at three wavelengths of red, blue, and green. Therefore, the texture distribution has a brightness for each wavelength of red, blue, and green of the reflected light from the surface of the observed object under certain illumination. Distribution. The extracted texture information is stored in the storage unit 3.

Reference numeral 43 in the figure denotes a mapping information extraction unit for extracting mapping information that associates each pixel in the distance distribution with a pixel in the background distribution. The background distribution photographed by the image input unit 22 is optically subjected to perspective transformation with the center of the lens as a starting point in the image forming process of the lens. For this reason,
The distance distribution and the background distribution cannot be directly superimposed. Therefore, the same perspective transformation as that performed by the image input unit 22 is performed on the three-dimensional coordinate points represented by each pixel of the distance distribution, and the distance distribution is obtained after the transformation. Of the three-dimensional coordinate values stored by each pixel in the transformed distance distribution, the two-dimensional coordinate points indicated by the two coordinate values excluding the depth coordinate value can be superimposed on the background distribution that is a two-dimensional image. The pixels in the background distribution corresponding to each pixel in the post-conversion distance distribution can be easily obtained. The correspondence relationship between the transformed distance distribution and the background distribution is equivalent to the correspondence relationship between the distance distribution and the background distribution and between the distance distribution and the texture distribution. This correspondence is stored in the storage unit 3 described above as mapping information.

Reference numeral 44 in the figure is a non-object distance in which all pixels in the distance distribution corresponding to the pixels in the background distribution existing outside the object area are deleted as non-object areas based on the mapping information. It is a distribution deletion unit. As a result, unnecessary information unrelated to the observed object 21 existing in the distance distribution is deleted.

Reference numeral 45 in the figure is a shape construction unit for constructing three-dimensional shape information over the entire area of the observed object from spatial information obtained by observing from a plurality of directions. The constructed three-dimensional shape information is stored in the storage unit 3, and is output by the output unit 4 together with the texture information and the mapping information.

FIG. 5 is a detailed configuration diagram of the shape constructing unit 45. Reference numeral 51 in the figure is an object volume extraction unit that calculates the space occupied by the observed object based on the contours of a plurality of object regions extracted from the background distributions observed from a plurality of directions, that is, the object contours. For each pixel of the object contour acquired from a certain viewpoint, there is a possibility that the observed object exists at an arbitrary position inside the conical space surrounded by the set of half-lines extending from the viewpoint. is there. When the common part of such a conical space acquired from a plurality of directions is obtained, the common part gives one closed space in which the observed object can exist. In this specification, this space will be referred to as an object volume. The surface of the observed object always exists on the boundary surface or inside the object volume. The object volume can indicate a through hole opened in the object, for example the handle of a coffee cup. However, it is not possible to extract the hole as the container of the cup itself, that is, the depressed portion that does not penetrate. If the observed object is an object having a shape having no non-penetrating depressions, the object volume correctly reproduces the three-dimensional shape of the observed object.
However, in reality, a large number of non-penetrating depressions appear in the object, so it is not sufficient to adopt the object volume as the three-dimensional shape of the object. In order to reproduce the shape of the non-penetrating depression as well, it is necessary to use the distance distribution to compensate for the defect in the object volume.

Reference numeral 52 in the figure is a spike noise removing section for removing pixels in the distance distribution existing outside the object volume as spike noise. Generally, if the primary reflection spot created by the laser beam directly on the object surface is observed, the spatial coordinate values calculated by the active stereo method correctly give the coordinate values of the sample points on the object surface. However, when the laser beam is further reflected and a secondary or subsequent spot formed by hitting another place on the object surface shows the strongest reflection, the spot may be observed and an incorrect spatial coordinate point may be calculated.

This principle is shown in FIG. Fig. 13-131
The laser beam 132 that emitted the
A primary spot is created on the image plane and an image is formed at a point 135 on the imaging surface 139. On the other hand, beam 132 is reflected at point 134 and
A secondary spot is formed at 6 and an image is formed at a point 137 on the imaging surface 139. The brightness of the point 137 may happen to be brighter than that of the point 135 depending on the shape and surface color of the object 133. At this time, the distance distribution acquisition unit 24 adopts the point 137, and as a result, it is illusion that the object surface is not at the actual point 134 but at the point 138 floating in the air. This is spike noise.

In many cases, spike noise appears in the air floating above the object surface. Moreover, since spike noise often exists inside the object region extracted from the background distribution, it is not completely deleted by the non-object distance distribution deleting unit 44 described above. However, since spike noise protruding from the object surface jumps out of the object volume, almost all spike noise can be detected by detecting the distance distribution existing outside the object volume. Since the spike noise is on the surface of the object, it must be recalculated using the surrounding distance distribution. Therefore, in this embodiment, spike noise is re-registered as a missing point and is interpolated together with missing points that naturally occur for other reasons.

A missing point interpolation unit for interpolating the coordinate values of missing points in each distance distribution 53 in the figure from the coordinate values of non-missing points located in the vicinity of the missing points. In this interpolation, the average depth coordinate values of the non-missing points existing in the vicinity of the eight adjacent points in the distance distribution of the missing points are obtained, and then the depth coordinate values and the trajectory of the laser beam when sampling the missing points are left 2 This is done by calculating one coordinate value.

Reference numeral 54 in the figure denotes a patch generator for generating a triangular patch representing the surface of the observed object from each distance distribution. The distance distribution obtained from a certain direction is an m × n two-dimensional array. The structure of the array is a mesh shape with equal intervals,
The points in the three-dimensional space indicated by the respective elements are also arranged in the space in the same order as the array in two directions except the depth direction, although the intervals are different. At this time, the triangle in the three-dimensional space indicated by the triangle having the apexes of the element of the array, any one element vertically adjacent to the element, and any one element adjacent to the left and right as the vertex is 3 representing a part of the surface of the observed object
It is a square patch. In this way, a large number of triangular patches are generated from one distance distribution. An area in the texture information covered by the patch is given to each triangle patch based on the mapping information.

Reference numeral 55 in the figure denotes an omnidirectional shape constructing unit which constructs three-dimensional shape information over the entire area of the observed object from the triangular patches obtained from the distance distribution in all directions. Although it is possible to output a set of triangular patches generated from a plurality of distance distributions as three-dimensional shape information, the set of triangular patches includes overlapping patches for the same part of the surface of the observed object. Is included. Overlapping patches have a leg of a perpendicular from one apex to the other inside the other patch,
Moreover, the length of the perpendicular line is close within a certain threshold value. In this case, one vertex is moved to the closest vertex to the other. If all vertices of one patch are moved to the vertices of one of the other patches, then the one patch is deleted to reduce duplication. As a result, the triangular patches generated from all distance distributions represent the surface shape over the entire observed object without duplication. A set obtained by removing the overlap of the triangular patches is stored in the storage unit 3 as three-dimensional shape information. This method requires the elimination of redundant redundant triangles, but has the advantage that some of the objects that are not visible from one side, such as the back of the handle of a patterned cup, are observed from the other direction. have. This is because the distance distribution is obtained as a two-dimensional array.

Object information constructed by the above processing,
That is, the output unit 4 outputs the three-dimensional shape information, the texture information, and the mapping information. The output three-dimensional shape information, texture information, and mapping information can be used by a texture mapping method in CG.

Another embodiment of the present invention will be described below.

(1) Although the invisible table is used as the means for supporting the object to be observed, the object to be observed may be hung by a thread or the like instead.

(2) The pattern projected by the pattern projection unit is not limited to the laser spot, and may be a one-dimensional or two-dimensional optical pattern such as a slit or a grid. Further, the method for acquiring the distance distribution is not limited to the active stereo method as an example, and any applicable three-dimensional visual technique may be used. Further, an infrared laser or the like may be used instead of the visible light laser.

(3) The means for making it possible to observe the object to be observed from a plurality of directions is not limited to the rotatable support portion, and for example, a sensor portion including a set of a pattern projection portion and an image input portion may be used. A mechanical mechanism that can move to a position may be used. Alternatively, the number of sensor units is not limited to one set, and a required number may be arranged at a required location to realize simultaneous observation from a plurality of directions.

When a movable sensor unit or a plurality of sets of sensor units are used, if the sensor unit can observe the object to be observed through the support unit from below the transparent support unit,
It is possible to realize observations without invisible parts. In this case, the supporting portion need not be rotatable as long as it is made of a transparent material and has a shape in which optical distortion is eliminated as much as possible, such as a thin glass flat plate. Also, by observing an object whose observation value is known through the support section, the relationship between the refraction amount of the beam emitted by the pattern projection section and the beam incident angle to the support section and the image input section are used to capture the image. If a correction value such as the relationship between the amount of misalignment of each pixel that occurs in the image and the angle of incidence of the optical axis of the image input section on the support section is calculated in advance, the image input section and the distance distribution acquisition section can perform optical It is possible to eliminate distortion computationally. At this time, information necessary for the optical distortion removal process such as the current beam incident angle, the current optical axis incident angle, and the correction value is stored and stored in the storage unit.

(4) The method of constructing the three-dimensional shape information over the entire area of the observed object is not limited to the method of moving the vertices of the illustrated triangular patch. The method of moving the vertices is an excellent method with extremely low calculation cost, but on the other hand,
Occasionally, some of the three vertices of a patch were moved, leaving a triangular patch with the remaining vertices floating in the air.
This is due to the problem of distance measurement accuracy,
Such a triangle becomes noise protruding like a whisker from the surface of the object.

In order to solve this, a line segment that intersects a slice plane generated at equal intervals across an observation space having a triangular patch generated from a certain distance distribution is obtained, and a large number of lines are provided for each slice plane. It is possible to obtain the cross-sectional shape of the observed object that is generated by being surrounded by minutes while removing the whisker noise. When the cross-sectional shapes on all slice planes are put together, information about the entire circumference of the observed object can be generated.

The process of generating the cross-sectional shape on one slice plane is performed as follows. That is, the slice plane is set as a pixel space having a two-dimensional array, the values of pixels through which each line segment passes are black, and the others are white. In this way, a certain cross-section contour image is constructed in the pixel space. This cross-section contour image has a closed curve representing the object cross-section and many whisker noises due to distance measurement errors. Next, the black pixels in the image are searched and tracked, all the pixel rows that have no end points and are closed are left as closed curves, and the others are white. As a result, whisker noise is removed. Next, a closed curve having an inclusion relation is detected, and all closed curves included in other closed curves are removed. As a result, only the closed curve representing the outer cross section of the object is extracted.

The process of collecting the object cross-sections between the slice planes to form three-dimensional shape information is performed as follows. That is,
Among the black pixels between the adjacent slice planes, the one having the shortest distance is linked. With respect to the black pixels between the two adjacent links connecting the two adjacent slice planes, this processing is continued until there are no consecutive black pixels on both slice planes. As a result, a triangular patch that connects the slice planes and represents the surface shape of the object is generated.

It is possible to reduce the calculation cost if the closed curve is not left as a pixel row but is expressed as a polygonal line approximation and the bending points of the polygonal line approximation are linked.

A voxel space having a three-dimensional array generated by cutting the observation space into a three-dimensional mesh may be used. The voxels that intersect the triangular patches generated from the distance distribution are black, and the others are white. The voxel space is a stack of pixel spaces, and there are three possible stacking methods, for example, the x direction, the y direction, and the z direction. The voxel space is regarded as a pixel space that is multiplexed in a certain direction, and whisker noise is eliminated in the same manner as the above slice method. The slice method was a method that can evaluate the possibility of whisker noise only in one direction, but in the voxel method, by repeating this operation in three directions of x, y, and z, the whisker noise can be reliably removed. it can. A closed surface can be extracted by tracking continuous black voxels in the voxel space from which whiskers have been removed. If there is one object, the closed curved surface composed of the most black voxels is the object surface. The process of constructing the three-dimensional shape information of the observed object from the closed curved surface is performed as follows. That is, all the triangular patches generated by connecting three mutually adjacent voxels forming the closed curved surface are extracted. As a result, the perimeter shape information of the object is constructed.

If the sampling density of the distance distribution is higher than the voxel density, the coordinate points in the distance distribution can be directly written in the voxel space without using the triangular patch generated from the distance distribution. Is. In this case, if the coordinates represented by a certain voxel are not the center of the voxel but the average of the coordinate values of all the coordinate points included in the voxel, the shape information with a small sampling error can be constructed.

As a method for directly constructing the all-round shape from the coordinate points in the distance distribution without generating a triangle from the distance distribution,
The energy minimization and relaxation method described below can also be used. This method is called Snake. This is a method of repeatedly calculating the positions and distributions of control points that minimize the sum of internal energy and external energy of a closed surface expressed by connecting many control points in a net. By covering all spatial coordinate points in the total distance distribution and calculating the distribution of control points that fit to this,
This is to construct the entire circumference shape of the observed object. Moreover,
Since the net is originally a closed curved surface, the advantage is that the result can be obtained as a closed curved surface even if there is a gap in the observation data. Although the initial value is important for the snake's convergence time and the quality of the result, good results can be expected by using the shape of the object volume obtained from the object region as the initial distribution of control points.

In constructing the three-dimensional shape information of the object, various construction methods may be manually or automatically selected and used by the user or the device.

Alternatively, the shape information may be expressed as a free-form surface by mathematical description instead of being described as a set of triangular patches.

As described above, in the object input device according to the present invention, as long as the three-dimensional shape information over the entire area of the observed object can be constructed, the difference in the construction method and the description format does not pose a problem.

(5) As described above, the background distribution is obtained by photographing the shadow image of the object illuminated by the rear illumination unit arranged behind the object. In the shadow image, since the object and the background generate areas having extremely different brightness from each other, it is important that the area occupied by the object and the area occupied by the background can be easily separated by simple image processing.

In order to extract the object area, several methods can be considered in addition to the method using a shadow image. For example,
By removing the rear illumination part and preparing a background plate having a color that does not exist in the object, and removing the region having the same color as the background plate from the image taken under the front illumination, the remaining region is the object region. . This method is effective when the color appearing on the object can be predicted to some extent in advance. Moreover, since the rear illumination unit can be eliminated, the device configuration can be reduced, and it is not necessary to switch the illumination.

Even when the color appearing on the object cannot be predicted,
For example, prepare a background plate with a pattern that changes with time,
It is possible to capture a plurality of images in a time series and extract a region in which luminance changes with time as a background region and the remaining region as an object region. The background plate can be easily realized by vibrating a pattern-printed plate.

As described above, in the object input apparatus according to the present invention, as long as the object area can be extracted from the background distribution, the difference between the means for photographing the background distribution and the background distribution itself does not pose a problem.

(6) Considering application to CG, CAD / CAM, etc., it is practically sufficient if three-dimensional shape information, texture information, and mapping information can be obtained as object information. The convenience of information is further improved.

As described above, the three-dimensional shape information and the texture information are information obtained at mutually different sampling densities, such as triangular patches, pixel densities and slice densities, voxel densities, snake curved control point densities, and the like. The unit for dividing the three-dimensional shape information does not match the unit for dividing the texture information of pixels. Therefore, mapping information is needed to associate them. At this time, the three-dimensional shape information is divided into smoothly continuous partial curved surfaces described by the same expression by using the shape processing method used in CAD or the like, and the texture information is obtained by using the image processing method. When the surface of the object to be observed is divided into monochromatic regions having the same color and the surface of the object to be observed is divided into partial curved surfaces having the same expression and the same color, each partial surface is trimmed to show the boundary between the curved surface expression and the partial surface. It has a set of information consisting of a curve formula and the color of the partial surface, and the object information is not a collection of individual information such as three-dimensional shape information, texture information, mapping information, but an entire object area having a shape and color. Given as a set of subsurfaces to cover. Since each partial surface serves as a part of the object surface and all necessary information is given thereto, convenience is improved in that it is easy to change and correct the object information after input.

(7) As described above, the texture information is the luminance distribution for each wavelength of red, blue and green of the reflected light from the object surface obtained as a color image of the surface of the observed object illuminated by the front illumination unit. In order to obtain the texture information of the entire area of the object, the object to be observed is oriented in various directions by the rotating support, and a plurality of texture distributions are photographed. At this time,
Depending on how the front illumination part is arranged, a step of reflected brightness may be confirmed at a joint of a plurality of texture distributions attached to the surface of the circumferential shape after integration. This occurs when the illumination light hits the object to be observed with rotation. Such a step significantly deteriorates the quality of the object information to be output. At this time, by fixing and equally arranging the plurality of illumination light sources on the rotating support portion, it is possible to realize a sufficiently bright illumination environment without a change in illuminance due to rotation and to obtain a good texture distribution. In addition, the illumination light source is not fixed to the support, but fixed in the environment so that a uniform illumination state is realized on the surface of the observed object.
It is also possible to arrange them in equal parts. Alternatively, the texture information may not be the luminance distribution by wavelength of the reflected light that is directly influenced by the illumination state, but may be the reflectance distribution by wavelength showing how much light of which wavelength the object reflects. The reflectance is also called an object color and is a color expression form that does not depend on the illumination condition. The reflectance can be calculated from the intensity distribution of the illumination light by spatial wavelength and the brightness distribution of the reflected light if the position of the light source is known, and these information can be obtained in advance.

As described above, in the object input device according to the present invention, the difference in the illumination method and the texture information does not pose a problem as long as the texture information over the entire area of the observed object does not have unnecessary steps. .

(8) Object information considering application to CG requires a description of the three-dimensional shape and color of the observed object, but when considering input to CAM, CAD, etc.,
Only the three-dimensional shape information of the object needs to be obtained. That is,
Convenience is improved in that the device scale can be reduced by eliminating the function relating to the texture information in the above-described embodiments and modifications.

As described above, in the object input device according to the present invention, if the three-dimensional shape information of the observed object can be output, the presence or absence of the additional information and the difference in the type do not pose a problem.

As described above, according to the present invention, by using the background distribution and the invisible supporting means, at least the unnecessary information irrelevant to the observed object is automatically and adaptively removed. If at least three-dimensional shape information of the observed object can be constructed while enabling discrimination, interpolation, and observation with no invisible portion over the entire observed object,
The present invention can be implemented in various modifications and combinations irrespective of differences in the configuration, means, processing procedures, etc. for realization.

[0080]

According to the object input device of the present invention, since there are no invisible parts, three-dimensional shape information over the entire area of the observed object can be easily calculated.

In addition, insufficient support for missing points, which has not been solved in the past, and complexity of eliminating unnecessary information are eliminated.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram of an embodiment of an object input device according to the present invention.

FIG. 2 is a processing configuration diagram of an embodiment of an object input device according to the present invention.

FIG. 3 is a configuration diagram of an observation unit in an embodiment of the object input device according to the present invention.

FIG. 4 is a configuration diagram of a calculation unit in an embodiment of the object input device according to the invention.

FIG. 5 is a configuration diagram of a shape constructing unit in a computing unit in an embodiment of the object input device according to the present invention.

FIG. 6 is a diagram showing an operation at the time of acquiring a distance distribution in one embodiment of the object input device according to the present invention.

FIG. 7 is a diagram showing an operation at the time of acquiring a texture distribution in one embodiment of the object input device according to the present invention.

FIG. 8 is a diagram showing an operation at the time of acquiring a background distribution in one embodiment of the object input device according to the present invention.

FIG. 9 is a principle diagram of an active stereo method.

FIG. 10 is a diagram illustrating a blind spot in an active stereo method.

FIG. 11 is a diagram illustrating the influence of a low reflectance surface in the active stereo method.

FIG. 12 is a diagram illustrating an effect of a through hole in the active stereo method.

FIG. 13 is a diagram showing the principle of spike noise generation in the active stereo method.

14A is a conceptual diagram showing a part of the acquired distance distribution, FIG. 14B is a conceptual diagram showing a part of the texture distribution, FIG.
(C) is a conceptual diagram showing a part of the background distribution, and (d) shows a part of the object information obtained from them.

[Explanation of symbols]

 1 ... Observing means 2 ... Computing means 3 ... Storage means 4 ... Output section

Claims (6)

[Claims] 1. An object input device in which an object to be observed is placed in an observation space, and three-dimensional shape information of the object to be observed and the like are input, and a distance distribution and a texture distribution which are spatial information in the observation space. The observation means is capable of measuring at least the background distribution of the background distribution, and an arithmetic means for constructing at least three-dimensional shape information of the observed object from the spatial information as object information relating to the observed object. An object input device comprising support means for supporting the object to be observed, wherein the support means is made of a material or structure that is invisible to the observation means. 2. The object information over the entire area of the observed object, wherein the observing means can observe the observed object from a plurality of directions, and the computing means integrates observation results from the plurality of directions into one. The object input device according to claim 1, wherein 3. The observing means is capable of measuring a distance distribution and a background distribution as the spatial information, and the calculating means uses the distance distribution included in an object region calculated from the background distribution to calculate the three-dimensional distribution. The object input device according to claim 1, wherein the shape information is constructed. 4. The observation means is capable of measuring a distance distribution and a background distribution as the spatial information, and the arithmetic means is an object from a plurality of directions calculated for each background distribution measured by the observation means. 3. The region is overlapped to generate a closed surface that covers the space occupied by the observed object, and the three-dimensional shape information is constructed from the distance distribution included in the inside of the surface. The described object input device. 5. An illuminating means is arranged at a position facing the observing means, and the background distribution is a shadow image of the observed object generated by the illuminating means illuminating from behind the observed object. The object input device according to claim 1, wherein the calculation unit sets the shadow area located at the center of the image as the object area. 6. The observing means is capable of measuring a distance distribution and a background distribution as the spatial information, and the distance distribution has a value of each pixel consisting of three-dimensional coordinate values of 2
2. A structure arranged in a three-dimensional array.
The described object input device.