JPH07146752A - Environmental model preparing device - Google Patents

Abstract

PURPOSE:To provide an environmental model preparing device capable of easily obtaining the positional information within environments of object data such as a desk and a chair, etc., and shapes of large objects such as a ceiling, a wall and a pillar, etc., when an environmental model is prepared. CONSTITUTION:This device is composed of a three-dimensional attribute information input part 1 measusing the three-dimensional attribute information on distance and shape surface attributes, etc., by using visual information from a television camera and an ultrasonic visual sensor, etc., an environmental model processing part 2 controlling the three-dimensional attribute information inputted in the three-dimensional attribute information input part 1 and forming the environmental model, a video preparation part 3 preparing the artificial video of environments based on the model description stored in the environmental model processing part 2, an analysis part 4 verifying the description of the environmental model by comparing the artificial video preparation result of the video preparation part 3 and the visual information at the present location and an observation control part 5 performing the input control of the three-dimensional attribute information input part 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to the environment 3 from image information.
The present invention relates to an environment model creating device that automatically creates an environment model by extracting dimensional shape and surface information.

[0002]

2. Description of the Related Art Conventionally, as a method of constructing an environment model of a room or the like, a person can interactively create it like data of computer graphics (hereinafter referred to as CG), or an environment map of a mobile robot, Some robots move around in the environment and detect free space by an external sensor.

In the method by which a human interactively creates an environment model, the individual shapes of objects in the environment and their positions in the environment must be actually measured.

For example, in an environment such as an office, a plurality of desks, chairs, partitions and the like of the same type are often arranged. Therefore, if object data such as a desk or a chair is individually created, a copy of these objects can be placed at a predetermined position in the environment to create a CG model of the environment.

However, the shape of a small object such as a desk or a chair can be easily measured by a scale or the like.
It is not easy to measure large object data such as columns and walls.

Further, in order to arrange the small object data, it is necessary to obtain the position information in the environment of the small object data such as the position of the desk in the room, which is measured. To do so requires a great deal of effort.
There is also a method of using a drawing to obtain the position information in the environment of the small object data. However, from the drawings, although the position information of the large object that cannot be moved can be obtained, the position information of the small object that frequently moves cannot be obtained. In other words, the drawings only include the initial information when the room layout was formulated, and therefore, there are many cases where the subsequent layout change information cannot be obtained. Furthermore, since information such as the color and pattern of the surface of the object cannot be obtained directly from the drawing, it is necessary to obtain them by some means and add them to the data.

The conventional method in which the robot moves around and acquires the external data by various sensors and constructs the map of the environment can reduce the labor for interactively creating, but the guidance of the mobile robot Since it is a map of purpose, it only obtains a description of free space and outline information of the environment. For this reason, it cannot be applied to the case where detailed shape and surface information of the environment are required like CG data.

[0008]

As described above, although it is not easy to create an environment model by the conventional method, the problems that cannot be created are summarized as follows.

The first problem is that individual object data cannot be accessed because the object data in the environment have not been separated. For example, the obtained three-dimensional shape data is a mere mass of three-dimensional data such as a desk or a chair that cannot be cut, and therefore cannot access the data of the desk to perform processing such as movement. That is, C
It is impossible to directly obtain data that can realize the basic processing required in the field of G processing.

The second problem is that, on the basis of the obtained data, no consideration has been given to how to make observations in order to further increase the reliability. Especially when a TV camera is used as a data input device,
The complexity of the background on which the observed object is placed greatly contributes to the easiness and reliability of the observation, but the conventional method does not consider this point, and it is reliable because the data is acquired under the complicated background. The problem is that it does not increase. This is because the conventional method has the observation procedure for filling the unobserved region, but does not have the observation procedure for improving the reliability of the observation data and obtaining more detailed data. . Therefore, in order to solve this, in the past, only inputting data was fused while adding reliability (probabilistic weighting) (A. Elfes: "UsingOccupancy Grids for M
obile Robot Perception and Navigation ", Computer, Vo
L.22, No.6, pp 46-57 (1989)).

Therefore, according to the conventional method, it is difficult to easily obtain the position information in the environment of the small object data and the shape of the large object when the environment model is created.

In view of this, the present invention provides an environment model creating apparatus that can easily obtain the environmental position information of small object data and the shape of a large object when creating an environment model.

[0013]

An environment model creating apparatus according to a first aspect of the present invention uses three-dimensional attribute information such as distance, shape, and surface attribute by using visual information from a television camera, an ultrasonic visual sensor, or the like. 3D attribute information input section to measure,
An environment model processing unit that forms an environment model from the three-dimensional attribute information input from the three-dimensional attribute information input unit and object data stored in advance, and an environment based on the environment model stored in the environment model processing unit. Of the artificial image, an analysis unit for verifying the environmental model description by comparing the artificial image creation result of the image creating unit with the visual information at the corresponding position, and the three-dimensional attribute information input unit. And an observation control unit that controls the input of.

The environment model creating apparatus of the second invention uses the visual information from a television camera or an ultrasonic visual sensor to measure three-dimensional attribute information such as distance, shape and surface attribute. An input unit, an environment model processing unit that manages the three-dimensional attribute information input from the three-dimensional attribute information input unit to form an environment model, and an artificial image of the environment based on the environment model stored in the environment model processing unit. Input from the three-dimensional attribute information input section, an image creation section that creates an image model, an analysis section that verifies the environmental model description by comparing the artificial image creation result of this image creation section and the visual information at the corresponding position. The outer data and the non-outer data of the target space are identified from the three-dimensional attribute information, and a detailed environment model is sequentially formed from the outer data so that the three-dimensional attribute information input unit It is made of the observation control unit for performing power control.

[0015]

[Operation] The environment model creating apparatus of the first invention will be described.

The three-dimensional attribute information input section is a television camera,
Using visual information from an ultrasonic visual sensor, distance,
Three-dimensional attribute information such as shape surface attribute is measured.

The environment model processing unit forms an environment model from the three-dimensional attribute information input to the three-dimensional attribute information input unit and the object data stored in advance.

The image creating unit creates an artificial image of the environment based on the model description stored in the environment model processing unit.

The analysis unit verifies the environment model description by comparing the artificial image creation result of the image creation unit with the visual information at the current position.

The observation control unit controls the input of the three-dimensional attribute information input unit.

With this, only by inputting visual information,
The environment model processing unit forms an environment model, and the position information in the environment of the small object data and the shape of the large object can be easily obtained.

The environment model creating apparatus of the second invention will be described.

The three-dimensional attribute information input unit is a television camera,
Using visual information from an ultrasonic visual sensor, distance,
Three-dimensional attribute information such as shape surface attribute is measured.

The environment model processing unit manages the three-dimensional attribute information input to the three-dimensional attribute information input unit to form an environment model.

An image creation unit creates an artificial image of the environment based on the model description stored in the environment model processing unit.

The analysis unit verifies the environmental model description by comparing the artificial image creation result of the image creation unit with the visual information at the current position.

The observation control unit identifies the contour data and non-contour data of the target space from the three-dimensional attribute information input from the three-dimensional attribute information input unit, and forms a detailed environment model in order from the contour data. The input control of the three-dimensional attribute information input unit is performed.

As a result, a region having simple surface information can be selected as the background of the object to be observed, which facilitates the separation of the object from the images and the collation between the images.

[0029]

【Example】

(Embodiment 1) Hereinafter, a first embodiment of the present invention will be described with reference to FIGS.

FIG. 1 shows a schematic configuration of an environment model creating apparatus, which is composed of a three-dimensional attribute information input section 1, an environment model processing section 2, an image creating section 3, an analyzing section 4, and an observation control section 5. There is. Hereinafter, each part will be described.

Three-Dimensional Attribute Information Input Unit The three-dimensional attribute information input unit 1 acquires the three-dimensional shape and surface information of the environment by stereoscopic viewing. In this system, stereo images are input by two TV cameras (hereinafter referred to as stereo cameras), and these stereo images are respectively stored in an image memory. Then, the stereo images stored in the two image memories are associated with each other to obtain the three-dimensional coordinate data of the environment in the coordinate system of the stereo camera.
The obtained three-dimensional coordinate data and image data are sent to the environment model processing unit 2.

Environmental Model Processing Unit FIG. 2 shows an example of the configuration of the environmental model processing unit 2. The updating unit 6, the three-dimensional environmental data storage unit 7, the recognition unit 8, the target object data setting unit 9, and the target object data storage unit. 10 and the environment model storage unit 11.

Further, FIG. 3 shows a flow of processing of the environment model processing section 2. Each step will be described in detail below with reference to this flow.

(Step 1) In the updating unit 6, the 3D coordinate data in the environment coordinate system preset in the environment is added to the environment in the coordinate system of the stereo camera obtained in the 3D attribute information inputting unit 1. The three-dimensional coordinate data is converted and registered in the three-dimensional environment data storage unit 7. The initial position in the environment coordinate system of the coordinate system of the stereo camera when the observation is started is measured by a survey instrument in advance.

The updating unit 6 refers to the three-dimensional shape of the environment stored in the three-dimensional environment data storage unit 7 and sets the next image acquisition position. In this case, the image acquisition position is 3
An arbitrary free space position is selected from the three-dimensional shape of the environment accumulated in the three-dimensional environment data storage unit 7. Then, the position information in the environmental coordinate system of the image collection position is sent to the observation control unit 5.

The observation control unit 5 controls the actuator so that the environment model creating device can move to this position.

After moving the environment model creating apparatus, an image input signal is sent to the three-dimensional attribute information input unit 1.

The three-dimensional attribute information input unit 1 again obtains a three-dimensional coordinate value of the environment from a new position from the image stored in the image memory.

The updating unit 6 converts the coordinate values into coordinate values in the environment coordinate system, and integrates the three-dimensional shape and the surface data recorded in the three-dimensional environment data storage unit 7. Integration is an occupancy grid that divides the space on the grid.
This is realized by filling in a grid including the three-dimensional shape and the position of the environment coordinate system from which the surface data is acquired, as described above.

The accumulated data in the three-dimensional environment data storage unit 7 is updated by repeating the processes of this movement, stereo image input, three-dimensional coordinate value calculation, and integration.

(Step 2) The recognition unit 8 uses the object data of the objects in the environment such as desks, chairs, partitions, etc. registered in the object data storage unit 10 in advance, and the three-dimensional environment data storage unit The data of the object part is recognized from the three-dimensional shape of the environment and the surface data stored in 7.

The object data of the in-environment object stored in the object data storage unit 10 is as follows.

(1) Shape Data The surface of the object data is cut into triangular patches, and the vertex coordinate values of each triangle and the normal vector indicating the surface direction of the surface are stored. The coordinate system is arbitrarily set around the object data.

(2) Surface data The image of the object data surface corresponding to each triangle patch is
It is accumulated by texture mapping on triangular patches.

(3) Collation data This is the collation data used in the recognition unit 8, and the adjacent patches in which the normal vectors of the triangular patches are similar to each other are integrated to accumulate a set of planes obtained. A flag that a set of triangular patches not included in this plane set is included in the curved surface is added and accumulated. Each plane holds a normal vector, a three-dimensional coordinate sequence of contour lines approximated to a polygonal line, an area, and a barycentric position on the plane with respect to a coordinate system set for each object data.

The recognition unit 8 performs Hough transform (ho) conversion on the occupied grid data obtained from the three-dimensional environment data storage unit 7.
ugh transformation) is applied to obtain a set of planes. For the obtained plane set Pi, a combination of planes similar to the collation data of each target object data Si stored in the target object data storage unit 10 is searched. That is, for each Si, there is a combination Qi of planes in which the plane lines and contour shapes of the individual planes Fi that are configured are similar, and the three-dimensional positional relationship between Fis is similar, in Pi. Search whether to do or not. If it exists, it is recognized as object data.

(Step 3) In the recognition unit 8, each F of Si is
The amount of rotational movement (θx, between the coordinate system of the primitive and the environment coordinate system such that i most overlaps with the corresponding plane of Qi.
θy, θz) and the parallel movement amount (dx, dy, dz). Then, it is sent to the object data installation unit 9 together with the type of object data.

(Step 4) The object data setting unit 9
The three-dimensional shape and surface data of the recognized portion and the three-dimensional shape and surface data accumulated in the object data storage unit 10 are replaced.

That is, the object data setting unit 9 uses the rotational movement amount and the parallel movement amount sent from the recognition unit 8 to collate with the occupied grid data obtained from the three-dimensional attribute information input unit 1. Incorporate the object data. After the outer surface of the occupied grid data is approximated by a triangular patch, the shape data and surface data of the matched object data are coordinate-converted using the rotation movement amount and the parallel movement amount obtained from the recognition unit 8 and integrated. .

(Step 5) The integrated triangular patch and the surface information texture-mapped on the patch are registered in the environment model storage unit 11.

Video Creation Unit The video creation unit 3 converts the environment model description obtained by the environment model processing unit 2 into a video using computer graphics and displays it.

The analysis unit analysis unit 4 compares the artificial image with the camera image at the current position obtained by the three-dimensional attribute information input unit 1 to detect a missing three-dimensional distance input and a description error due to an error. The part is detected and the environment model description stored in the environment model processing unit 2 is corrected. In addition, the analysis unit 4 detects a region in which model data is not obtained by the above comparison processing, transmits a new observation command to the three-dimensional attribute information input unit 1 or the observation control unit 5, and controls the movement observation. Do.

In the present embodiment, the recognition unit 8 uses a combination of planes to recognize the object data from the surface three-dimensional shape and surface data obtained from the three-dimensional environment data storage unit 7. Other methods such as matching using the volume of the object data can also be applied. Further, the data in the object data storage unit 10 is not limited to that described in the present embodiment, and various data can be used according to the recognition method used in the recognition unit 8.

(Example 2) Next, a second example of the environment model creating apparatus of the present invention will be described with reference to FIGS.
9 will be described.

The schematic configuration of the environment model creating apparatus of this embodiment is the same as that of the first embodiment, and the three-dimensional attribute information input unit 1, environment model processing unit 2, image creation unit 3, analysis unit 4, and observation unit are used. It is composed of the control unit 5. Hereinafter, each part will be described.

Three-dimensional Attribute Information Input Section FIG.
The structural example of the dimension attribute information input part 1 is shown.

The two television cameras 106 and 107 simultaneously input images according to the image input signal sent from the observation control unit 5, and store them in the image memories 108 and 109, respectively. The depth calculation unit 110 uses the image memories 108 and 109.
The depth is calculated by stereoscopic vision from the two input images.

Before observing with this apparatus, a human sets a reference environmental coordinate system in the environment. Then, the initial position and the initial direction of this device in this environment coordinate system are measured and registered in the observation control unit 5.

The coordinate transformation unit 111 transforms the depth data calculated by the depth calculation unit 110 into coordinates using the position and direction in the environment coordinate system of the present apparatus sent from the observation control unit 5. As shown in FIG. 7, the position and direction in the environment coordinate system C of this device are (xv, yv, zv), θ
v, the depth data obtained by the depth calculation unit 110 (the position and direction of this device obtained with reference to the coordinate system C ′ of the stereo camera are the origin position of this coordinate system C ′ and the x-axis). (Using the angle formed by), (xs, ys
, Zs), three-dimensional data (x, y, z) based on the environment coordinate system is obtained by the following equation.
However, in this embodiment, the floor surface around which the present apparatus moves is flat, and rotations of the x axis and the y axis are ignored.

X = xcos θv −ysin θv −xv y = xsin θv + ycos θv −yv z = −zv Observation control unit FIG. 6 shows an example of the configuration of the observation control unit 5, which includes an outer contour data generation unit 112, a storage unit 113, and surface information analysis. The unit 114, the observation position / direction calculation unit 115, the movement control unit 116, and the actuator 117 are included. Hereinafter, each part will be described.

[Storage Unit] The storage unit 113 stores an occupancy grid G which projects the observation space onto the xy plane and divides the plane into uniform size grids (see FIG. 9). Let the (i, j) th grid of this occupied grid G be gij. gij
Holds two state flags ft and fa.

Ft is a target flag and describes whether or not the grid gij is an outer contour. fa is an area flag and describes whether or not gij is an outer area. The relationship will be described below.

Ft sets whether or not the grid gij is a perfect outline, and is 1 when it is not a perfect outline and 2 when it is a perfect outline.

Fa is for setting whether or not gij is an outer region, and is 0 when it is not the outer region and 1 when it is the outer region.

The case of ft = 2 and the complete contour means that the contour is recognized in gij and the three-dimensional data does not exist in the area of gij or more. Then, such a region is the outer region, and fa = 1.

The case where ft = 1 and the contour is not a perfect contour means that the contour is recognized in gij and three-dimensional data exists in the region of gij or more. Then, assuming that such a region is not the outer region, fa =
It becomes 0.

[Outer Contour Data Generation Unit] Outer contour data generation unit 1
The reference numeral 12 obtains contour data, non-contour data, and image data from the three-dimensional data based on the environment coordinate system obtained from the three-dimensional attribute information input unit 1 in accordance with the processing procedure of FIG. The data of the unit 113 is integrated.

Hereinafter, the outer contour data generator 1 will be described with reference to FIG.
The process of 12 will be described.

First, as a premise, ft and fa of each gij are
It is initialized to 0 when the apparatus is started. Then, the occupied lattice G registered in the storage unit 113 is loaded.

(Step 1) For each three-dimensional data (x, y, z) obtained from the three-dimensional attribute information input unit 1,
The target flag ft of gij including the value of (x, y) is set to 1.

(Step 2) For each gij for which ft = 1, ft is set to 2 when fa = 1.

(Step 3) As shown in FIG. 10, each gij for which ft = 1 is labeled with 8 adjacencies to obtain a set L of label areas. In the case of the example of FIG.

[Equation 1] Is labeled as.

(Step 4) It is judged whether L is an empty set. If it is not an empty set, the process proceeds to step 5, and if it is an empty set, the process proceeds to step 10.

(Step 5) Since L is not an empty set, L
One area li is arbitrarily selected from the label areas included in. Then, as shown in FIG. 11, of the areas surrounded by the two tangents p1 and p2 drawn from the position P of the apparatus to the label area li, the area farther from p than li is obtained. All occupied grids gij containing this region
Is set as the outer area A. In the case of the example in FIG. 12, the outer area of the label area 11 is A1, and the outer area of the label area 12 is A2.
Becomes

(Step 6) It is judged whether or not the area of ft = 1 in gij included in the outer area A, that is, the area in which the three-dimensional data exists is included, and the area is included. If so, the process proceeds to step 9, and if not included, the process proceeds to step 7.

(Step 7) Since gij included in the outer area A does not include the area where ft = 1 (that is, the area where the three-dimensional data exists) (in the example of FIG. 12, the outer area A1). ), Gij included in the label area is an outline, and 2 is substituted for ft.

(Step 8) The fa of gij including the outer area is set to 1. In the example of FIG. 12, gij included in l1
Ft is set to 2. Also, gij included in A1
Fa is set to 1. However, in the area of A2, f
Since gij with t = 1 exists, gij included in l2
Of gij contained in A2 also remains 0.

(Step 9) Remove the selected label area li from the set of L, return to step 4, select one label area again, and repeat the above processing until L becomes an empty set.

(Step 10) Since L is an empty set, ft
For each occupied grid gij with 1 or 2 selected, the one having the maximum z coordinate value zmax among the three-dimensional data included in the gij is selected. Then, a three-dimensional occupied lattice V (see FIG. 13) is created by adding zmax to the occupied lattice G.

(Step 11) The image information obtained from the image input section is pasted on the side surface of the three-dimensional occupied lattice by the texture mapping technique in CG. Here, the side surface of the three-dimensional occupied lattice is a side surface of a columnar rectangular parallelepiped obtained by extending the individual lattices of the occupied lattice G upward by zmax, and in the example of the three-dimensional occupied lattice of FIG. 13, S1, S2, S3, S4. Is. As shown in FIG. 14, since the position of this device in the environment coordinate system is known, the position of the camera relative to this device
If the posture and focus position are acquired by calibration, the image part projected on the side surface of the three-dimensional occupied lattice can be cut out. Therefore, the image cut out on each side of the three-dimensional occupied lattice is added as an image file. deep.

(Step 12) After performing the above processing, the outer contour data generation unit 112 updates the data of the storage unit 113 to the new three-dimensional occupied grid V and new occupied grid G.

[Surface Information Analysis Unit] Surface information analysis unit 114
Then, the area division is performed by the edge density of the image added to the side surface of the three-dimensional occupied lattice V of the storage unit 113.

[Observation Position Direction Calculation Unit] The observation position direction calculation unit 115 calculates the observation position and direction such that the background image of the object to be observed has a simple texture area. Based on FIG. 15, the observation position direction calculation unit 115
The processing flow of is described.

(Step 1) It is judged whether or not there is a lattice of ft = 1 in the occupied lattice G, and if it exists, the process proceeds to step 2, and if it does not exist, the process proceeds to step 14 to control the movement of the observation end signal. Send to section 7 and finish.

(Step 2) Since the occupied grid G has a grid of ft = 1, this grid is not an outline, and remeasurement is performed to improve the measurement accuracy. Each g of the occupied grid G
In ij, the region where ft = 1 is labeled with 8 neighbors, and the set is labeled L1.

(Step 3) In each gij of the occupied lattice G, the region where ft = 2 is labeled with 8 neighbors, and the set is labeled as L2. L2 is an outer region.

(Step 4) Let L1i and l2i be the regions in L1 and L2 which are close to each other. Here, being close means that gij of ft = 1 does not exist between l1i and l2i as shown in FIG.

(Step 5) As shown in FIG. 16, l2i
On the other hand, the search area B is set around the point-symmetrical position around the center of gravity of 11i. For example, as shown in FIG. 17, B is set as a triangular region below the point C where the straight line connecting both ends of l1i and l2i is extended and intersects.

(Step 6) If the search area B can be set, the process proceeds to step 7, and if not, the step 12
Proceed to.

(Step 7) When the search area B is set,
An area having a low edge density is obtained as a simple texture area from the area division result of the surface information analysis unit 5 for the side image of the three-dimensional occupied lattice corresponding to l2i.

(Step 8) In the search area B, the position and direction of the apparatus are changed at any time to generate an image when the simple texture area and l1i are projected on the image plane of the apparatus (see FIG. 18).

(Step 9) In this image, as shown in FIG. 19, it is judged whether or not the upper portion of the projected image of l1i is included in the simple texture area, and if it is included, the process proceeds to step 10. If not included, the process proceeds to step 11.

(Step 10) The space in which l1i exists is set as a re-observation space, and (x, y, z) and θ at this time are sent to the movement control unit 116. Then, the process ends.

(Step 11) If the upper portion of the projected image of l1i is not included in the simple texture area, the projected image is created at all locations in the search area B, and the upper portion of the projected image of l1i becomes the simple texture area. It is determined whether or not it is included. If it is included, the process proceeds to step 12, and if it is not included, the process returns to step 8.

(Step 12) If l1i such that the upper portion of the projected image is included in the simple texture region does not exist even if the projected image is created at all locations in the search region B, l1i is removed from the set of L1. To do.

(Step 13) It is judged whether or not L1 is an empty set, and if L1 is an empty set, an observation end signal is sent to the movement control unit 116 to end the processing. If L1 is not an empty set, a pair of adjacent l1i and l2i is selected again to set a search area, and the process returns to step 4.

[Movement Control Unit] In the movement control unit 116, the position (x, y,
z), the actuator 117 is controlled so that the apparatus can move in the direction θ. Then, after moving, an image input signal is output to the three-dimensional attribute information input unit 1. Further, as the current position direction in the environment coordinate system of this device, (x, y, z),
The value of θ is sent to the three-dimensional attribute information input unit 1. When the observation end signal is input from the observation position / direction calculation unit 115, the movement control is stopped and the observation is ended.

Environment model processing unit The environment model processing unit 2 describes the environment model from the three-dimensional data and the image obtained from the three-dimensional attribute information input unit 1.

Video Creation Unit The video creation unit 3 converts the environment model description obtained by the environment model processing unit 2 into a video using CG and displays it.

Analyzing unit The analyzing unit 4 compares the artificial image with the camera image at the current position obtained by the three-dimensional attribute information inputting unit 1 to detect a missing three-dimensional distance input or an error in the description due to an error. Is detected and the environment model description stored in the environment model processing unit 2 is corrected. In addition, the analysis unit 4 detects a region in which model data is not obtained by the above comparison processing, transmits a new observation command to the three-dimensional attribute information input unit 1 or the observation control unit 5, and controls the movement observation. Do.

Here, the case where the number of cameras is two has been described as an example of the configuration of the three-dimensional attribute information input unit, but the number of cameras is not limited, and the same method as described above can be used.

Further, in the surface information analysis unit 114, the image is divided into regions according to the edge density, but a region division method by texture analysis by hue or Fourier transform can be applied. Furthermore, the observation position direction calculation unit 1
The method of setting the search area B in 15 and the method of determining the position direction in which the projection image is generated are not limited to the method described here, and other methods can be applied.

[0103]

According to the environment model creating apparatus of the first invention, instead of the three-dimensional shape and surface information of the environment constructed by the data input means, prestored object data can be individually replaced. , Registered objects can be accessed independently. Therefore, it is possible to directly obtain the information necessary for realizing the basic processing required in the CG processing.

With the environment model creating apparatus according to the second aspect of the present invention, when the three-dimensional data of the environment is acquired, observation is performed in order from the outer contour of the environment, and after the outermost contour is input, the objects inside the outer contour are input. Since an image is input from a position direction in which an area with a simple texture is the background to an object and the 3D data is acquired, the acquisition process of the 3D data is easy, and the reliability of the acquired data is high. Will increase. This allows
Since highly reliable 3D data can be obtained, an environment model can be input.

[Brief description of drawings]

FIG. 1 is a block diagram showing an environment model creating apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram of an environment model processing unit.

FIG. 3 is a flowchart showing a processing flow of an environment model processing unit.

FIG. 4 is a diagram for explaining an occupied grid.

FIG. 5 is a block diagram of a three-dimensional attribute information input unit according to a second embodiment.

FIG. 6 is a block diagram showing a configuration of an observation control unit according to a second embodiment.

FIG. 7 is a diagram for explaining an environmental coordinate system and a coordinate system of a stereo camera of this apparatus.

FIG. 8 is a flowchart showing a flow of an outer contour data generation unit.

FIG. 9 is a diagram for explaining an occupied grid of an outer contour data generation unit.

FIG. 10 is a diagram for explaining labeling of an outer contour data generation unit.

FIG. 11 is a diagram illustrating an outer region of an outer contour data generation unit.

FIG. 12 is a diagram for explaining a method of determining a contour of a contour data generation unit.

FIG. 13 is a diagram for explaining a three-dimensional occupied lattice of an outer contour data generation unit.

FIG. 14 is a diagram for explaining image mapping of an outer contour data generation unit.

FIG. 15 is a flowchart of a processing flow of an observation position / direction calculation unit.

FIG. 16 is a diagram for explaining a method of setting a search area of an observation position / direction calculation unit.

FIG. 17 is a diagram for explaining a method of setting a search area of an observation position / direction calculation unit.

FIG. 18 is a diagram for explaining the projection onto the image plane of the observation position / direction calculation unit.

FIG. 19 is a diagram for explaining the inclusion relation with the simple texture region of the observation position / direction calculation unit.

[Explanation of symbols]

 1 ... 3D attribute information input unit 2 ... Environment model processing unit 3 ... Image creation unit 4 ... Analysis unit 5 ... Observation control unit 6 ... Update unit 7 ... 3D environment data storage unit 8 ... Recognition unit 9 ... Target data installation Part 10 ... Object data storage unit 11 ... Environment model storage unit 110 ... Depth calculation unit 111 ... Coordinate conversion unit 112 ... Outer contour data generation unit 113 ... Accumulation unit 114 ... Surface information analysis unit 115 ... Observation position direction calculation unit 116 ... Move Controller 117 ... Actuator

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 15/00 7/00 H04N 7/18 K // G01B 11/24 Z (72) Inventor Suzuki Kaoru 1-30 Odaidanaka, Kita-ku, Osaka-shi, Osaka Umeda Sky Building Tower West Co., Ltd. Toshiba Kansai Branch (72) Inventor Takashi Wada 1-30-1 Oyodo-naka, Kita-ku, Osaka, Osaka Umeda Sky Building Tower West Co., Ltd. Toshiba Kansai Branch (72) Inventor Minoru Yagi Minoru 1-30, Oyodo Naka, Kita-ku, Osaka-shi, Osaka Umeda Sky Building Tower West Co., Ltd. Toshiba Kansai Branch (72) Inventor Hiroshi Hattori 1-30 Odaidanaka 1-chome, Kita-ku, Osaka City, Osaka Prefecture Umeda Sky Building Tower West Incorporated Toshiba Kansai Branch (72) Inventor Tatsuro Nakamura Osaka 1-30 Oyodo-naka, Kita-ku, Osaka City, Umeda Sky Building Tower West Stock Company Toshiba Kansai Branch Office

Claims (5)

[Claims] 1. A three-dimensional attribute information input section for measuring three-dimensional attribute information such as distance, shape and surface attribute using visual information from a television camera or an ultrasonic visual sensor, and the three-dimensional attribute information. An environment model processing unit that forms an environment model from three-dimensional attribute information input from the input unit and object data stored in advance, and creates an artificial image of the environment based on the environment model stored in this environment model processing unit A video creating unit, an analysis unit for verifying the environment model description by comparing the artificial video creating result of the video creating unit with the visual information at the corresponding position, and the input control of the three-dimensional attribute information input unit. An environment model creating device comprising an observation control unit. 2. The environment model processing unit stores an object data storage unit that stores in advance data about an object in the environment, data stored in the object data storage unit, and the three-dimensional attribute. A recognition unit that recognizes an object in the environment stored in the object data storage unit from the shape and surface data of the environment obtained from the information input unit, and that determines the position of the recognized object in the environment. , At the position in the environment of the object obtained by this recognition unit,
An object data installation unit that replaces the object data recognized by the recognition unit instead of the shape and surface data of the environment obtained from the dimension attribute information input unit, and the object data replaced by this object data installation unit The environment model creating apparatus according to claim 1, further comprising: an environment model storage unit that stores 3. A three-dimensional attribute information input section for measuring three-dimensional attribute information such as distance, shape and surface attribute using visual information from a television camera or an ultrasonic visual sensor, and the three-dimensional attribute information. An environment model processing unit that manages the three-dimensional attribute information input from the input unit to form an environment model, an image creation unit that creates an artificial image of the environment based on the environment model stored in this environment model processing unit, An analysis unit that verifies the environment model description by comparing the artificial image creation result of the image creation unit with the visual information at the corresponding position, and the 3D attribute information input from the 3D attribute information input unit From the observation control unit that controls the input of the three-dimensional attribute information input unit so as to identify the contour data and the non-contour data and form a detailed environment model in order from the contour data. An environment model creating device characterized by: 4. The three-dimensional attribute information input unit, an image input unit for inputting an image, a depth calculation unit for acquiring depth using the image input by the image input unit, and the depth calculation unit. 4. The environment model creation according to claim 3, further comprising: a coordinate conversion unit that converts the depth data obtained by the above into coordinate values in a preset coordinate system to obtain three-dimensional data. apparatus. 5. The observation control unit generates a contour data generation unit that generates contour data, non-contour data, and image data of the target space based on the data from the three-dimensional attribute information input unit, and a contour data generation unit that generates the contour data. The storage unit that stores the contour data, the non-contour data, and the image data, the surface information analysis unit that analyzes the surface information from the image data stored in the storage unit, and the surface information that is analyzed by the surface information analysis unit. And an observation position / direction calculation unit that calculates an observation position and direction from the contour data and non-contour data accumulated in the accumulation unit so that the background of the observed object is simple. The environment model creating device according to claim 3.