JPH0981788A - Environment model input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate an accurate environment model by integrating the environment model without depending upon a frequency determining probability. SOLUTION: This device is equipped with a three-dimensional attribute information input part 1 which inputs three-dimensional attribute information on a body in environment, an observation control part 8 which controls the input of the three-dimensional attribute information input part 1, a data generation part 2 which generates shape data on one viewpoint from the obtained three- dimensional attribute information, a model storage part 5 which stores the integrated environment model, a model integration part 4 which inputs and integrated the environment model integrated with the shape data generated at the one viewpoint, a video generation part 6 which generates an artificial image at a specific position in the environment according to the description of the environment model stored in the model storage part 5, and an analysis and verification part 7 which corrects the description of the environment model by comparing the generated artificial image with an actual image at the corresponding position.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to, for example, a three-dimensional C
The present invention relates to an environment model input device for creating an environment model by acquiring information such as the position and surface of an object in the environment required for AD, three-dimensional computer graphics, mobile robot control, and the like.

[0002]

2. Description of the Related Art In recent years, needs for three-dimensional CAD for supporting the design of industrial parts, image creation using three-dimensional computer graphics, and intelligent mobile robots are rapidly increasing. In these techniques, it is necessary to input into the computer the geometrical shape, surface attributes, and if necessary, motion data of the object to be designed or displayed by computer graphics, or the environment in which the robot moves. The process of this work is called modeling, and the numerical data expressed inside the computer is called a model.
However, at present, modeling is generally not automated, and humans do much labor by hand, so automation is strongly desired.

Recently, for application to three-dimensional CAD, there has been proposed a system for automatically inputting the shape of a presented target by using a range image device called a range finder.

Further, conventionally, as a method for directly extracting the target three-dimensional information from an actual television image, a stereo method utilizing the principle of triangulation with a plurality of cameras, or changing the focal length with a single camera. Active research is being conducted on techniques for obtaining three-dimensional information by analyzing the obtained image series.

As an example in which these techniques are applied to modeling, studies have been conducted to automatically input the shape of a human face by using a stereoscopic technique, assuming realistic communication or a conference. In addition, research is also being carried out in which robots that move indoors are equipped with ultrasonic sensors and multiple cameras to automatically create a schematic descriptive model of the moving space.

When a model is created by the above-mentioned method, there are cases where a part that cannot be seen from only one viewpoint is generated, or accuracy is insufficient when the object is distant. Therefore, techniques for obtaining target data from a plurality of viewpoints and integrating the obtained target data to create a better model have been studied.

This involves rotating the table by, for example, placing the target on the rotary axis of a rotary table and measuring the three-dimensional coordinates of points on the target surface using a laser range finder that scans the target with laser light. By repeating the measurement, the target data from a plurality of viewpoints is obtained, the shape of the target viewed from each viewpoint is obtained, and these are integrated to obtain the overall shape of the target.

However, in this method, the size of the object is limited by the size of the rotary table and the movable range of the range finder. Also, the subject must be brought to the location of the system, including the turntable and laser range finder.

Therefore, it is difficult to input including the arrangement of the target in a natural environment and to use it as the visual system of the robot. Therefore, a method has been developed in which an object is photographed using a camera and the camera is moved to obtain object data from a plurality of viewpoints.

A stereo method is often used for creating target data at each viewpoint, and a model is obtained by integrating the shapes at each viewpoint as in the case of the laser range finder described above.
The target data created from each viewpoint is described by combining points, contour lines, and surfaces. As these points and contour lines, edges and corner points on the image used when creating the target data are used, and these are combined to generate a surface.

Therefore, in order to perform integration by this expression method, it is necessary to associate edges and corner points between a plurality of viewpoints and surfaces generated from them,
There is a problem that the features obtained from images do not always correspond between viewpoints. Then, as a method of avoiding this correspondence problem between multiple viewpoints and performing integration, integration using voxel space was developed.

That is, in this integration by voxel space, the space containing the target is divided into a grid-like small space (voxel), and it is determined whether or not each voxel becomes a part of the target based on the data at each viewpoint. The target model is the set of all voxels that are part of the target.

This method is not limited to points and contours,
Since only a part of the target is in each voxel, it is not necessary to associate the models close to each other to integrate the models, and it becomes easy to obtain the model from the target data from a plurality of viewpoints.

As a concrete method of integration, a method called a shaving method has been developed. This shaving method is a method of shaving an object by deleting voxels that do not correspond to the object from the voxel space.First, the probability that the object exists in each voxel in the space hidden by the surface of the object from the viewpoint. A value is given and a probability value of 0 is given to voxels in the space between the viewpoint and the target surface. In the integration, the sum of the probability values of each voxel is obtained, and voxels having a probability value equal to or higher than a preset threshold value are extracted.

This shaving method is based on the fact that a voxel having a high sum value has a high probability of being part of the object, but in order to compare the magnitude of the sum between voxels, the sum term The condition is that the number is almost constant. This condition is often satisfied when target data can be stably obtained over the entire target that can be viewed from each viewpoint, such as a laser range finder. Also,
The above condition may be satisfied by setting a large voxel to absorb the error in the position of the target data. This is the case where details of the shape of the voxel region to be extracted are not needed, such as when searching for a space in which the robot can move.

[0016]

However, in the case of obtaining the shape of an object using an image, it is determined whether the frequency at which each voxel is given a probability, that is, the number of terms for summing, actually exists. Variation occurs regardless of. This does not mean that the entire object within the field of view is stably modeled when features in the image cannot be obtained or when an error occurs in the correspondence between the left and right images by the stereo method.
This is because it may be partially missing.

Therefore, in the above-mentioned shaving method, if the voxels are set to be small in order to obtain a fine shape, the voxels which are below the threshold value and are scraped due to this variation occur, and even if the target actually exists. Regardless, there was a problem that the model might be missing.

The present invention has been made in view of the above problems, and an object of the present invention is to integrate environment models without depending on the frequency of giving a probability and to create an accurate environment model. It is to provide a possible environment model input device.

[0019]

Means for Solving the Problems That is, the present invention is as follows: (1) Three-dimensional attribute information input means for inputting three-dimensional attribute information of an object in the environment, and input of three-dimensional attribute information by this three-dimensional attribute information input means. Observation control means for controlling,
The data creating means for creating shape data from one viewpoint based on the three-dimensional attribute information obtained from the three-dimensional attribute information inputting means and the new data storing means for storing the shape data created by the data creating means are integrated. Environment model storage means for storing the environment model, shape data created from one viewpoint stored in the new data storage means and viewpoints other than this viewpoint, and stored in the integrated environment model storage means. Model integration means for inputting and integrating the environment model, an image creation means for creating an artificial image at a specific position of the environment based on the description of the environment model stored in the model storage means, and this image creation means. To provide an analysis verification means for correcting the description of the environment model by comparing the created artificial image with the real image at the position corresponding to the artificial image. Those were.

(2) In the above item (1), the model integrating means carries out normalization of the Gaussian distribution so as to give a probability value 1 when the distance from the surface of the shape data is a predetermined distance. The probability value according to the distribution is given to each voxel space.

(3) In the above item (1), the model integrating means gives the existence probability of the object to a voxel in the voxel space, and from a plurality of probability values given to each voxel, a unique positive value or The environment model input device according to claim 1, wherein a product of positive values is calculated to obtain an integrated probability of each voxel.

(4) In the above item (1), the model integrating means gives the existence probability of the object to voxels in the voxel space, and integrates the probabilities given from the models created from a plurality of viewpoints. The voxels on the probability peak surface are extracted.

(5) In the above item (4), the model integrating means is, for each voxel, a voxel in an area of n × n × n (n: 3, 5, 7, ...) Centering on this voxel. Assuming a plane passing through the center voxel, calculate the average value of the integration probabilities of the voxels on this plane and the average value of the integration probabilities of the voxels on the plane that sandwiches the plane, and compare However, when the former is larger than the latter, it is determined that the voxel at the center is on the plane passing through it as the peak plane and is extracted.

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An environment model input device according to an embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows a schematic structure thereof.
Is an attribute of a target existing in the environment, such as an ultrasonic sensor or a TV camera that measures three-dimensional distance information to the target, and a TV camera that inputs target surface attribute information such as color or pattern. A plurality of types of three-dimensional attribute information input units are provided, and when the three-dimensional attribute information including the above-mentioned three-dimensional distance information and surface attribute information at the current position is input, it is sent to the next-stage data creation unit 2. Here, it is assumed that, for example, two TV cameras are arranged horizontally.

The data creation unit 2 creates the target shape data based on the three-dimensional attribute information which is the one-point viewpoint data input from the three-dimensional attribute information input unit 1, and the created shape data is the new data storage unit. 3 and store it.

The shape data stored in the new data storage unit 3 is read out to the model integration unit 4. Model integration unit 4
Integrates the shape data of the one-point viewpoint read from the new data storage unit 3 with the environment model read from the model storage unit 5 to obtain an environment model description, and stores the obtained environment model description in the model storage unit 5.

The image creation unit 6 reads the environment model description stored in the model storage unit 5 and converts it into an artificial image using computer graphics. The converted artificial image is, for example, a CRT. It is displayed and output on the configured display unit (not shown).

The artificial image created by the image creating unit 6 is also sent to the analyzing / verifying unit 7 via the model storage unit 5. The analysis / verification unit 7 compares the artificial image from the image creation unit 6 with the actual image at the current position obtained by the three-dimensional attribute information input unit 1 to determine whether the three-dimensional distance input is missing or due to an error. The error portion of the environment model description is detected, and the environment model description stored and stored in the model storage unit 5 is corrected.

Further, the analysis / verification unit 7 sends a control signal to the observation control unit 8 at the end of the correction operation. Upon receiving this control signal, the observation control unit 8 controls the movement of the device and sends a control signal to the three-dimensional attribute information input unit 1 to input the next three-dimensional attribute information after the movement.

FIG. 2 shows an example of the internal structure of the data creating section 2. In this embodiment, assuming that the surface of the shape data is represented by triangular patches, the data creating section 2 creates a stereo processing section 2a and a patch. And a section 2b.

The stereo processing unit 2a extracts feature points and edges from the three-dimensional attribute information sent from the three-dimensional attribute information input unit 1, associates the left and right images, and calculates the three-dimensional coordinates of the feature points. By using the three-dimensional coordinates calculated in this way, the patch creation unit 2b sequentially creates triangular patches, outputs them as shape data, and stores them in the new data storage unit 3.

Next, FIG. 3 shows a detailed structure of the model integration section 4. That is, the model integration unit 4 includes a probability generation unit 4a, a probability storage unit 4b, a probability integration unit 4c, a peak extraction unit 4d, and a plane generation unit 4e, although the operation of each unit will be described in detail later. .

The operation of the three-dimensional attribute information input unit 1, the data creation unit 2, the new data storage unit 3, the model integration unit 4, and the model storage unit 5 having the above-mentioned configuration will be described below.

FIG. 4 shows the series of processing contents.
First, an initial model is initially created and stored (steps A1 and A2), and then a new model is input, stored, and integrated to update the model (step A).
3 to A6) operation is repeatedly executed.

That is, in the creation of the initial model, the data creation unit 2 creates an environment model by the stereo method using the three-dimensional attribute information from the three-dimensional attribute information input unit 1 which is an image photographed from one viewpoint. (Step A1), the created model is stored in the new data storage unit 3 and the model storage unit 5 as an initial model (Step A2).

After that, in order to update the model, the data creation unit 2 creates new shape data after moving the viewpoint (step A3), and the new data storage unit 3 stores this data (step A4). Then, the model integration unit 4 reads the newly created shape data and the accumulated environment model from the new data storage unit 3 and the model storage unit 5, respectively, and integrates them (step A5). The specific processing contents related to this integration will be described in detail later. Then, the integrated result is stored in the model storage unit 5 and the environment model is updated (step A6).

Such an updating operation (steps A3 to A)
6) is repeated until the target shape is obtained. Hereinafter, the newly created shape data will be referred to as a new model, and the accumulated model will be referred to as an accumulation model.

FIG. 5 shows the details of the model integration process by the model integration unit 4 in step A5. That is, the model integration unit 4 first inputs the new model and the accumulated model from the new data storage unit 3 and the model storage unit 5 (step B1).

FIG. 6 schematically illustrates a voxel space including a model in which the voxel size is manually set. In the voxel space assumed in this way,
The model integration unit 4 then calculates the shortest distance d _n from the voxel to the surface of the new model and the probability value P _{n according} to the following equation (1) using this, in the probability generation unit 4a, and stores it in the probability storage unit 4b. (Step B2). At the same time, the probability generation unit 4a also calculates the shortest distance d _o from the voxel to the surface of the accumulation model and the probability value P _o according to the following equation (2) using this.
It is stored in the probability storage unit 4b (step B3).

[0041]

[Equation 1]

FIG. 7 exemplifies the distribution of the probability values P _n and P _o , in which the vertical axis represents each probability value P _n and P _o , the horizontal axis represents the calculated distance, specifically, the distance d _n. , D _o are 0 at the surface, and the distance from the surface is shown.

As shown in the figure, the probability values P _n and P _o are both part of the Gaussian distribution, and d _max is the upper limit value of the distance giving a positive probability value, and the probability value is 1 when the distance is d _max. Is standardized to be

The distance d _max is whether or not the established values P _n and P _o are integrated into a single peak distribution as will be described later, that is, whether the surfaces of the two models are integrated into one surface. It has a threshold function that determines whether it should be set manually in advance. Hereinafter, d _max will be referred to as an integrated distance. The deviation σ is also set manually.

When the processing up to step B3 is completed,
Next, the probability value P _n by the new model and the probability value P _o by the accumulation model of each voxel are read from the probability storage unit 4 b, P is calculated according to the following equation (3), and the P obtained by the calculation is re-calculated. (Step B4). Below, P is called an integrated probability value. That is,

[Equation 2]

Here, the relationship between the probability value P _n by the new model, the probability value P _o by the accumulation model, and the integrated probability value P, and the function of the integrated distance d _max are calculated on a straight line with the same distance from each voxel. The case where it is performed will be described.

Since the two established values P _n and P _o are both part of the Gaussian distribution, if the peak-to-peak distance of the established values P _n and P _o is less than or equal to the integrated distance d _max , the integrated established value P is Figure 8
As shown in, the distribution has a single peak.

The peak position of the established value P _{n according to} the new model is the surface of the new model where the distance d _n becomes zero. Further, the peak position of the established value P _o according to the accumulation model is also the surface of the accumulation model as described later.

That is, the peak-to-peak distance is the distance between the surfaces of the model. Therefore, if the distance between the surfaces of the new model and the accumulation model is less than or equal to the integrated distance d _max , the integrated established value P
The probability value P _n given by the two models by the calculation of
From P _o , the two surfaces are integrated by imitating the same surface.

On the other hand, when the peak-to-peak distance is larger than the integrated distance d _max , the peaks are not integrated as shown in FIG. If the straight lines that calculate the distance are not the same,
Although the distribution of the probability value P _{n and} the probability value P _o are widened,
Similarly, if the peak-to-peak distance is less than or equal to the width of the distribution, it is integrated into one peak, and if it is greater than the width of the distribution, it is not integrated.

The steps B2 and B3 may be interchanged in order or may be calculated in parallel. Further, the voxel size and the deviation σ may be calculated and set from an error caused by the pixel size of the image or an error in the position of the viewpoint.

When one of the voxels is called C,
For example, 3 in the voxel space centering on the voxel C
Each voxel in the region of × 3 × 3 is referred to as a sub-voxel for voxel C. FIG. 10 shows a voxel C and its sub-voxels.

Next, using the calculated integrated probability value P, the peak extraction unit 4d determines whether the voxel C is on the peak plane of the integrated probability value P in the sub-voxels (step B5). FIG. 11 shows the direction of the normal line of the peak plane determined by the peak extraction unit 4d and the axes representing the coordinates of sub-voxels.
Each grid point in the figure is the center of the sub-voxel.

That is, the coordinates (x, y,
z) is (0, 0, 0), and the subvoxel probability value P
Let (x, y, z) be {P (x, y, z): x = -1,0,1; y = 1,0,
1; z = -1,0,1}, the determination conditions for the normal directions 1, 4, and 7 in the figure are the following equations (4), (5), and (6). That is, (1) In the case of normal direction 1

(Equation 3)

(2) Case of normal direction 4

(Equation 4)

(3) When the normal direction is 7

(Equation 5)

Similar determination conditions are used for peak planes in other directions. In each of the determination conditions shown in the above equations (4) to (6), the plane having the normal direction passes through the voxel C, and the average value of the probability values of the sub-voxels on the plane sandwiching the plane is the voxel C. It is a condition that it is less than or equal to the average value in the sub-voxels on the passing plane.

12 to 14 show examples of the arrangement of sub-voxels on a plane passing through the voxel C and the arrangement of sub-voxels on two planes sandwiching this plane. That is, FIG. 12 exemplifies the above arrangements with respect to the normal direction 1, FIG. 13 with respect to the normal direction 4, and FIG. 14 with respect to the normal direction 7. In each case, the sub-voxel S1 indicated by a white circle in the drawing is a voxel. Sub-voxels on a plane passing through C, sub-voxels S2 indicated by white circles with hatching are sub-voxels on two planes sandwiching the plane passing through voxel C, and if any of the above determination conditions is satisfied, it is determined that the voxel is a peak. To do.

In FIG. 10, the voxels in the 3 × 3 × 3 area in the voxel space centering on the voxel C are described as sub-voxels for the voxel C.
Not limited to this, for example, 5 × 5 × centered on the voxel C
It is also possible to use a larger area such as 5 and change the judgment condition accordingly to judge whether or not the area is on the peak plane.

Steps B2 to B for all voxels
After the process 5 is performed to calculate and store the normal direction, a triangle patch is generated by connecting the centers of adjacent voxels from the voxels extracted by the surface generation unit 4e, and the generated triangle patch is updated. The model is stored in the model storage unit 5 as a model (step B6), and the series of integration processing shown in FIG. 5 is completed.

By executing the updating of the environment model including the integration processing as described above, the shape data obtained from a plurality of viewpoints can be integrated accurately even if the number of viewpoints is relatively small. You will be able to create various environment models.

The present invention is not limited to the embodiments, and various modifications can be made without departing from the spirit of the invention.

[0063]

As described above in detail, according to the present invention, the environment models are integrated independently of the frequency giving the probability,
An environment model input device capable of creating an accurate environment model can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration inside a data creation unit of FIG.

FIG. 3 is a block diagram showing a configuration inside a model integration unit in FIG.

FIG. 4 is a flowchart showing an operation content according to the same embodiment.

5 is a flowchart showing a subroutine of a model integration process of FIG.

FIG. 6 is a diagram schematically showing a voxel space including an environment model according to the same embodiment.

FIG. 7 is a diagram exemplifying a distribution of established values corresponding to distances according to the same embodiment.

FIG. 8 is a diagram exemplifying a distribution of established values corresponding to distances according to the same embodiment.

FIG. 9 is a diagram exemplifying a distribution of established values corresponding to distances according to the same embodiment.

FIG. 10 is a diagram exemplifying a relationship between voxels and sub-voxels according to the same embodiment.

FIG. 11 is a diagram showing a direction of a peak surface and coordinates of sub-voxels according to the same embodiment;

FIG. 12 is a diagram illustrating an arrangement of voxels and sub-voxels according to the same embodiment.

FIG. 13 is a diagram illustrating an arrangement of voxels and sub-voxels according to the same embodiment.

FIG. 14 is a diagram illustrating an arrangement of voxels and sub-voxels according to the same embodiment.

[Explanation of symbols]

 1 ... Three-dimensional attribute information input unit 2 ... Data creation unit 2a ... Stereo processing unit 2b ... Patch creation unit 3 ... New data storage unit 4 ... Model integration unit 4a ... Probability generation unit 4b ... Probability storage unit 4c ... Probability integration unit 4d ... Peak extraction section 4e ... Surface generation section 5 ... Model storage section 6 ... Video creation section 7 ... Analysis / verification section 8 ... Observation control section

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hiroshi Hattori 1-1-30 Oyodochu, Kita-ku, Osaka City, Osaka Prefecture Toshiba Kansai Branch Office

Claims (5)

[Claims] 1. A three-dimensional attribute information input means for inputting three-dimensional attribute information of an object in the environment, an observation control means for controlling the input of the three-dimensional attribute information by the three-dimensional attribute information input means, and the three-dimensional attribute. A data creating means for creating shape data from one viewpoint based on the three-dimensional attribute information obtained from the information inputting means, a new data storing means for storing the shape data created by this data creating means, and an integrated environment model An environment model storage means to store, shape data created from one viewpoint stored in the new data storage means, and an environment model stored in the integrated environment model storage means created from a viewpoint other than this viewpoint; Video creation that creates an artificial video at a specific position in the environment based on the model integration means that inputs and integrates and the description of the environment model stored in the model storage means Environment model input comprising means and analysis and verification means for correcting the description of the environment model by comparing the artificial image created by the image creating means with the real image at the position corresponding to the artificial image. apparatus. 2. The model integrating means normalizes the Gaussian distribution so as to give a probability value 1 when the distance from the surface of the shape data is a predetermined distance, and assigns a probability value 1 to each voxel space. The environment model input device according to claim 1, wherein a probability value is given. 3. The model integrating means gives the existence probability of an object to a voxel in a voxel space, and calculates a unique positive value or a product of positive values from a plurality of probability values given to each voxel. The environment model input device according to claim 1, wherein the integrated probability of each voxel is set as 4. The voxel on the peak surface of the integration probability obtained by integrating the probabilities given from the models created from a plurality of viewpoints by giving the existence probability of the object to the voxel in the voxel space. The environment model input device according to claim 1, wherein 5. The model integrator means, for each voxel, n × n × n (n: 3, with the voxel as the center.
, Voxels in the area of (5, 7, ...) Are selected and a plane passing through the center voxel is assumed, and the average value of the integration probabilities of the voxels on this plane and the voxels on the plane sandwiching the plane. Compute and compare the average value of the integrated probability of, when the former is larger than the latter, it is determined that the voxel at the center is on the plane passing through it as the peak plane,
The environment model input device according to claim 4, which is extracted.