JPH11339074A - Scenery modeling device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a scenery modeling device which reproduces a correct scenery efficiently by efficiently setting proper height for a building. SOLUTION: This device is equipped with an image storage means which stores images of an object picked up in different directions, an image pickup information storage means 2 which stores image pickup information on the respective images, a shape storage means 3 which stores outline information on horizontal sections of the object, a feature extracting means 4 which extracts feature parts of the respective images stored in the image storage means 1, a height setting means 6 which finds the height of the object according to the feature parts extracted by the feature part extracting means 4, the image pickup information stored in the image pickup storage means 2, and the outline information stored in the shape storage means 3 and a three dimensional shape forming means 7 which forms a three dimensional shape of the object according to the outline information stored in the shape storage means 3 and the height of the object set by the height setting means 6.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a landscape modeling apparatus for obtaining a three-dimensional shape of a photographed object from an image of the object such as a building.

[0002]

2. Description of the Related Art FIG. 30 shows, for example, Minoru Fujii, Ryosuke Shibasaki,
Junichi Tatemura, "A Trial of Realization of GIS User Interface by Fusion of Real Scene Image and 3D Spatial Data", Photogrammetry and Remote Sensing Vol. 36, no. 3,19
It is a block diagram which shows the conventional landscape modeling device shown by the 1997 issue.

In FIG. 1, reference numeral 101 denotes a building polygon data storage device which stores data of a contour shape of a horizontal section of a building, 102 denotes an elevation data storage device which stores data of a floor number of a building, and 103 denotes building polygon data. An arithmetic unit that calculates the three-dimensional shape of a building from the altitude data
A display device 104 displays the three-dimensional shape of the building calculated by the calculation device 103.

FIG. 31 is a diagram showing a map and various data obtained from the map. FIG. 31 (a) is a map corresponding to a landscape to be modeled, FIG. 31 (b) is building polygon data obtained from the map, and FIG. 31 (c) is a diagram showing the number of floors for each building and the altitude value indicating the altitude at the point where the building is installed. In the figure, 105 is a map, 10
Reference numeral 6 denotes a building drawn on a map 105, which is described as a polygon which is a contour of a horizontal section of the building.

FIG. 32 is a view showing a modeled landscape displayed by the display device 104. In the figure, 1
Reference numeral 07 denotes a landscape composed of a building shape reproduced on a display monitor; 108, a three-dimensional building shape restored from polygons, floor data, and elevation data; 109, a bottom surface of the three-dimensional building shape 108 represented by a polygon; Is the average height per floor of the three-dimensional building shape 108.

In a conventional landscape modeling device, FIG.
As shown in FIG. 31A, as shown in FIG. 31B, the polygon 106 of each building drawn on the map 105 is
The coordinates of the polygon representing the outline of the horizontal cross section of the building are stored as building polygon data in the building polygon data storage device 101. Further, as shown in FIG. The altitude data indicating the altitude of the point being set is stored in the altitude data storage device 102.
In advance.

When reproducing the cityscape shown in the map 105, the arithmetic unit 103 shown in FIG.
For each building, a polygon indicated by the building polygon data stored in the building polygon data storage device 101 is defined as a bottom surface 109 of the three-dimensional building shape 108, and a predetermined height per floor (constant value) Then, a value obtained by multiplying the floor number data of each building stored in the elevation data storage means 102 is defined as a height, and a polygonal pillar having these bottom surfaces and heights is formed.

The bottom surface 109 of the polygonal prism thus formed is drawn so as to be placed at an elevation altitude given by the elevation data of the point where the polygonal prism exists, thereby forming a three-dimensional building shape 108. Thus, the landscape 107 including the building shape 108 is drawn on the display monitor 107.

The three-dimensional building shape 108 can be obtained by retrieving information on a building drawn therefrom from a landscape image, or by inputting a location, as described in the prior art document. It is used for landscape simulation and for determining the outlook of microwave communication in a city, in addition to the purpose of specifying a location.

[0010]

In the conventional landscape modeling device as described above, the height per floor of a building is made uniform and the same value (constant value), and the height per floor is made equal to the height of each building. The height of each building was determined by multiplying the number of floors. However, in general, the height of a building differs for each building or each floor. For this reason, the conventional landscape modeling device has a problem that the accurate height of the building cannot be obtained and the landscape is not correctly reproduced.

In addition, when there are advertisements, water towers, and other installed objects on the roof of the building, the height is not reflected, so that the accurate height of the building cannot be obtained, and the landscape cannot be reproduced correctly. there were.

Conventionally, in order to give an accurate height of a building, a design document indicating the height of each building is obtained and input, or the height of each building is also determined. Has to be measured, which requires a great deal of labor.

Furthermore, since the elevation value of a place where a building is built is usually measured only on a coarse grid point, an accurate setting altitude of the building cannot be obtained in a place with many hills. There was a problem that the landscape would be different.

Furthermore, in the conventional landscape modeling device as described above, since the building shape of the building is represented by a polygonal pillar having a polygon indicating the outer periphery of the building shown on the map as a bottom surface, the polygon is represented by the polygon. When given as data representing the site of a building, there has been a problem that the shape of the polygon is different from the actual contour of the building, and the landscape is not correctly modeled.

The present invention has been made to solve such a problem, and provides a landscape modeling device that efficiently reproduces a correct landscape by efficiently setting an appropriate height for a building. It is intended to be. It is another object of the present invention to provide a landscape modeling device that reproduces a landscape more accurately by obtaining an accurate contour shape.

[0016]

A landscape modeling apparatus according to the present invention stores image storage means for storing at least two images of the same object taken from different directions, and stores imaging information at the time of capturing each image. Imaging information storage means, and shape storage means for storing contour information of a horizontal cross section of the object,
A feature extraction unit for extracting a feature of each image stored in the image storage; a feature extracted by the feature extraction; storage in the imaging information and shape storage stored in the imaging information storage; Height setting means for obtaining the height of the object based on the contour information set, and the object based on the contour information stored in the shape storage means and the height of the object set by the height setting means. And a three-dimensional shape forming means for forming the three-dimensional shape.

Further, the height setting means has a horizontal component and a height component of the characteristic part based on the characteristic part extracted by the characteristic part extracting means and the imaging information stored in the imaging information storage means. A position information calculating means for calculating three-dimensional position information; extracting a characteristic part belonging to the object based on the horizontal component and the contour information of the three-dimensional position information calculated by the position information calculating means; The height direction component of the feature is set to the height of the object.

Further, the height of the highest feature is defined as the height of the object. The characteristic part is a characteristic point. The characteristic part is a characteristic line segment. The characteristic part is a characteristic region.

Further, the landscape modeling device according to the present invention comprises: an image storage means for storing an image of a target object;
An imaging information storage unit for storing imaging information at the time of imaging of the image, a three-dimensional shape forming unit for forming a three-dimensional shape of the object, and an image storage unit based on the imaging information stored in the imaging information storage unit. Display means for displaying the stored object and the three-dimensional shape of the object formed by the three-dimensional shape forming means in association with each other, and receiving the input based on the display result of the display means, the three-dimensional shape of the object Height adjusting means for adjusting the height of the object.

Further, the height adjusting means selects an object whose height is to be adjusted while checking the display result, and adjusts the height. Further, the height adjusting means adjusts the height of the three-dimensional shape selected based on the display result, based on the input information that is input after confirming the display result. Further, the display means displays the three-dimensional shape selected by the height adjusting means and another three-dimensional shape in different modes. Further, the display means displays the object stored in the image storage means and the three-dimensional shape of the object formed by the three-dimensional shape forming means in different formats.

Further, the landscape modeling device according to the present invention includes a map storage means for storing a map corresponding to an object,
Shape storage means for storing outline information of a horizontal cross section of the object; three-dimensional shape forming means for forming a three-dimensional shape of the object based on the outline information stored in the shape storage means;
Based on the imaging information stored in the imaging information storage means, the object stored in the image storage means and the three-dimensional shape of the object formed by the three-dimensional shape forming means are displayed in association with each other, and a map is displayed. Display means for displaying the object on the map stored in the storage means and the contour formed by the contour information stored in the shape storage means in association with each other; Display means for displaying the object and the contour formed by the contour information stored in the shape storage means in association with each other, and correction means for correcting the contour information of the object in response to an input based on the display result of the display means It has.

Further, the correction means corrects the outline information of the outline selected based on the display result, based on the input information input after confirming the display result. Further, the display means includes:
The contour selected by the correction means and another contour are displayed in different modes. The correcting means corrects the position of the contour. The correcting means corrects the shape of the contour. Further, the display means displays the object on the map stored in the map storage means and the outline formed by the outline information stored in the shape storage means in different formats.

The correcting means adds a new contour. Further, the three-dimensional shape forming means forms the three-dimensional shape including the contour information on the new contour.

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 FIG. 1 is a block diagram showing a landscape modeling device according to the first embodiment.
In the figure, reference numeral 1 denotes an image storage device for storing at least two images of an object such as a building taken from different directions;
Reference numeral denotes an imaging information storage device that stores imaging information such as a viewpoint position, an optical axis direction, and angle of view information of a camera when an image stored in the image storage device 1 is captured, corresponding to each image.

Reference numeral 3 denotes a shape information storage device for storing information relating to polygons representing the outline of the horizontal cross section of the object, and reference numeral 4 denotes a feature portion extraction for extracting feature points as feature portions from an image stored in the image storage device 1. The device 5 is a position calculation device that calculates a three-dimensional feature point position of the feature point from the feature point extracted by the feature portion extraction device 4 and the imaging information stored in the imaging information storage device 2.

Reference numeral 6 denotes a height setting device for setting the height of the object based on the three-dimensional feature point positions calculated by the position calculation device 5 and the polygons of the object stored in the shape information storage device 3. Is a three-dimensional shape forming device that reproduces the three-dimensional shape of the object based on the height of the object set by the height setting device 6 and the polygon of the object stored in the shape information storage device 3; This is a display device that displays a three-dimensional shape reproduced by the shape forming device 7.

FIG. 2 is a diagram showing an object to be photographed and an image obtained by photographing the object. FIG. 2 (a) is a diagram showing the object to be photographed, and FIGS. 2 (b) and 2 (c) show this. It is an image when an object is photographed from different directions.

In the figure, reference numeral 9 denotes an object to be photographed, 10a
Is an imaging point of the imaging device when the object 9 is imaged by an imaging device (not shown), and 10b is an imaging point different from the imaging point 10a, and the imaging direction at the imaging point 10b is the same as the imaging direction at the imaging point 10a. In different directions. Arrows indicate the imaging points 10a, 1
0b shows each imaging direction. 11a (11b) is an image of the object 9 taken at the imaging point 10a (10b), and 12a (12b) is an image 11a (11b).
The object above.

It is assumed that a plurality of images obtained by photographing a target real scene are stored in the image storage device 1 in advance. These images 11 are stored by, for example, inputting data from a scanner after photographing or inputting data photographed by a digital still camera. Here, the image 11 is a multi-valued image of light and shade. Note that, for a color image represented by RGB, R
The present invention can be applied by executing restoration processing of three-dimensional coordinates of feature points described below for each component of GB.

The image pickup information storage device 2 stores image pickup information such as the viewpoint position of the camera at the time of image pickup, the optical axis direction, and angle of view information. For the viewpoint position, optical axis direction, and angle of view information of the image, a GPS or an electronic gyro is attached to the camera, and its output is input, or the value at the time of shooting is input by a keyboard and stored. Conversely, it may be obtained by calculating from the position information of the reference point whose coordinates drawn on the image are known.

FIG. 3 is a diagram showing the imaging information stored in the imaging information storage device 2. As shown in FIG. 3, a viewpoint x coordinate, a viewpoint y coordinate, which are three-dimensional position coordinates of an imaging point,
Information such as the viewpoint z coordinate, the azimuth angle of the optical axis, the depression angle, the twist around the optical axis, and the angle of view is recorded for each image.

It is assumed that the shape information storage device 3 stores in advance a polygon indicating the outer circumference of a horizontal section of an object such as a building. Here, the polygon represents the outline of the horizontal section of the object. Hereinafter, in order to simplify the description, the object will be described as a building. The object is not particularly limited to a building, but may be any object as long as a three-dimensional shape of the object can be formed from the polygon and the height.

FIG. 4A is a diagram showing a map corresponding to a target landscape and an object shape on the map. In the figure,
13 is a map corresponding to the target landscape, 14 is a map 13
The upper building shape shows the outer circumference of the horizontal section of the building. FIG. 4B is a diagram showing polygons formed by reading information on the building shape shown in FIG. 4A and forming the information based on the information. In the figure, 15 is a polygon, 15a is a plurality of vertices of the polygon 15, and 15b is a line segment connecting the vertices 15a.

For polygons, a map input by a scanner is automatically read and vectorized, or a building shape drawn on the map is interactively manually input and stored. At this time, if there is a non-flat portion in the target landscape, the elevation data of the corresponding building point is also stored for each polygon. This value is input numerically from a map or calculated from data that has already been digitized.

As shown in FIG. 4B, a polygon is
It is represented by a point sequence on the xy plane and a line segment connecting the point sequence. It is assumed that the polygon has a convex shape. If there is a concave portion, it is processed by dividing it into convex partial polygons, and finally the highest height among them is set as the height of the polygon.

FIG. 5 is a diagram showing information on polygons stored in the shape information storage device 3. As shown in FIG. 5, the number of vertices of the polygon and the x and y coordinates of each vertex of each polygon are stored sequentially in a counterclockwise order from the point at which the x coordinate value is maximum.

Next, the operation of the landscape modeling device shown in FIG. 1 will be described in detail. In the landscape modeling device according to the present embodiment, the height of a building to be reproduced is obtained using a captured image, and a city landscape is reproduced based on the obtained height. Hereinafter, the ground plane is defined as the xy plane, and the z axis is defined in the height direction.

First, the characteristic point extracting device 4 extracts characteristic points using the image 11 stored in the image storage device 1, and the position calculating device 5 obtains three-dimensional position coordinates of the characteristic points. A feature point is a point drawn as a corner or a vertex on the image 11 due to a difference in color or brightness from the surroundings. The corner of the upper surface of the building is a feature point due to the difference in appearance from the surrounding landscape.

Here, for the sake of simplicity, a point where the luminance value changes from the surroundings on the image is used as a feature point, and the extraction is performed by, for example, "Hidden Boundary" by Takeshi Kurata, Yuichi Nakamura, Yuichi Ota. Feature Point Tracking "IEICE Technical Report Vol. 95, No. 470, 55-62, published in 1996.

This corresponds to the pixel (io, i) of the image I (i, j).
jo), consider a region W in the vicinity,

[0041]

(Equation 1)

A pixel having a smaller eigenvalue is a feature point of a pixel having a predetermined threshold value or more. Here, w is a weighting function. It should be noted that when actually extracting the feature points, discretization is considered. For example, the neighboring area W is set to, for example, a 3 × 3 area, w is always set to 1, and

[0043]

(Equation 2)

It is assumed that

Then, the three-dimensional position coordinates of the feature points extracted as described above are
Using the images, the pixel information of the feature points captured there,
It is obtained by the principle of triangulation from imaging information such as position information of the viewpoint at which the image was taken.

Next, the height setting device 6 obtains the height of the target building itself from the three-dimensional position coordinates of the feature points. FIG. 6 is a diagram showing the relationship between the positions of the polygons of the target building and the positions of the feature points. In the figure, 16 is a feature point extracted from the image 11, and 16a is an xy of the feature point.
A feature point whose projection onto a plane does not exist in the target polygon is represented by a triangle. Reference numeral 16b denotes a feature point existing in a polygon whose feature point is projected onto the xy plane. The feature point is represented by a circle, and the highest (z component) feature point is represented by a black circle.

In order to obtain the height of the building itself, it is determined which building the feature point extracted by the characteristic part extracting device 4 and whose three-dimensional position coordinates have been calculated by the position calculating device 5 belongs to, and becomes a target. The feature point having the highest height (z coordinate value) among the feature points belonging to the building is set as the height value of the building.

Here, the determination as to which building the feature point belongs to is made based on which polygon the projection of the feature point on the ground plane (xy plane) falls inside. As shown in FIG. 6, the projection of the feature point onto the xy plane causes the feature point 16 b to enter the interior of the polygon 15 of the target building (in FIG. 6,
The feature point indicated by a circle) is determined to belong to the building, and the feature point 16a that does not enter the interior is regarded as not a feature point of this building. That is, the projection of the feature point 16b on the building indicated by the polygon 15 is naturally included in the polygon 15, and is not included in a different polygon at the same time. Whether or not a feature point falls within the polygon is determined by comparing the xy coordinates of the feature point with the coordinates of the polygon.

At this time, the value of the height (z component) of the highest feature point is defined as the height of the building. This is because there is a possibility that a feature point may be generated even on the wall surface of the building. If the height is set in this way, even when the height is not constant due to the rooftop installation, when performing microwave line-of-sight determination, the height of the highest point can be assumed, and the line of sight is actually invisible. Nevertheless, it is possible to realize a correct landscape without being erroneously determined to be able to see through.

As described above, when the height of the building is set by the height setting device 6, the three-dimensional shape forming device 7 sets the polygon of the building stored in the shape information storage device 3 as the base, and A polygonal prism having the height value set as the height is formed, and these are drawn on the display monitor of the display device 8.

In order to realize more accurate landscape modeling, the bottom of the polygonal pillar is determined by the elevation altitude given by the elevation data of the point where the polygonal pillar exists (the elevation of the point where the building is built is obtained in advance). It is good to draw so that it is placed in

In the present embodiment, the three-dimensional shape of the object is represented by a polygonal prism. However, the present invention is not particularly limited to this. When the object has a curved surface, a cylinder or a combination of a prism and a cylinder is used. The three-dimensional shape of the object may be represented.

FIG. 7 is a flowchart showing the operation of this embodiment. In step ST1, the characteristic portion of each image stored in the image storage device 1 is extracted by the characteristic portion extracting device 4, and the three-dimensional position of the characteristic point of the extracted image is calculated by the position calculating device 5. This is performed based on the principle of triangulation by associating feature points of two images, for example. The details are shown in the flowchart of FIG.

FIG. 8 is a flowchart for explaining the operation in step ST1 shown in FIG. Step S
At T21, the variable N is initialized to 0. Step ST2
In step 2, it is determined whether or not there is an unprocessed image. If it exists, the process proceeds to step ST23. If not (if the process has been completed for all images), the process proceeds to step ST1.
To end. In step ST23, two images to be compared are selected. This is, for example, two unprocessed images in the shooting order. In step ST24, for each selected image, feature points of the selected image are extracted.

In step ST25, the feature points of the two images are associated with those representing the same feature point in space. This involves, for example, calculating the similarity of pixel patterns in the vicinity of a feature point, and associating those that match well. The similarity is calculated, for example, by taking the difference between the pixel values in a nearby 3 × 3 area and calculating the sum of the squares, and the more the value is smaller than a predetermined threshold value, the better the match is made .
The number of feature point sets associated in this manner is M.

At this time, the position of the corresponding point is restricted on the epipolar straight line which is the projection of one line of sight to the other, depending on the position of the viewpoint at the time of photographing the image.
The processing speed can be increased by narrowing the search range of the corresponding point near the epipolar straight line.

In step ST26, a variable n is initialized to 1. In step ST27, from the n-th feature point set of each feature point associated in step ST25, three-dimensional position coordinates of the feature point are obtained. This is performed as follows.

FIG. 9 is a diagram for explaining a method of calculating the three-dimensional position coordinates of a feature point. In the figure, reference numeral 10 denotes an imaging point (hereinafter, sometimes referred to as a viewpoint) of the imaging apparatus when the image 11 is captured by the imaging apparatus;
17 is a feature point on the image 11, 1
8 is a line-of-sight vector for viewing the feature point 17 from the viewpoint 10, 19
Is an optical axis vector of the image 11, 20 is a position corresponding to a feature point on an actual building, and 20a is an epipolar straight line on the image obtained by projecting a line-of-sight vector of the other image forming a pair.

As shown in FIG. 9A, the viewpoint 10
To q (qx, qy, qz) and the optical axis vector 19 to u (c
osαcosβ, sinαcosβ, −sinβ). α is the azimuth of the optical axis vector 19 (optical axis vector 19
Is the angle of depression of the optical axis vector 19 with respect to the xy plane (the angle looking down from the horizontal). Further, assuming that the twist of the image 11 around the optical axis (counterclockwise) is γ, the image size is Nx × Ny, and the focal length is f, the line of sight 18 that looks at the pixel I (i, j) in the image 11 The vector v of

[0060]

(Equation 3)

Is represented by This means that the viewpoint 10 is the origin O '
A camera coordinate system is defined in which the x 'axis is in the direction of the optical axis vector 19, the y' axis is in the opposite direction parallel to the i axis of the image 11, and the z 'axis is in the j axis direction. , I.e., a vector (f, -i + Nx / 2, j-N) from the viewpoint 10 (origin O ') to the characteristic pixel 17
y / 2) is converted to an absolute coordinate system. In addition. Assuming that the horizontal angle of view of the image 11 is φ, the focal length f is

F = (Nx) / {2tan (φ / 2)}

Is obtained. As shown in FIG. If both the pixel I (i, j) in the image 11a and the pixel I ′ (i ′, j ′) in the other image 11b are characteristic pixels and are associated, Since the actual building position t (the three-dimensional position of the feature point) corresponding to the feature point 17 in the image of FIG.

T = av + q t = a'v + q '

Is expressed as follows. Here, a and a ′ are scalar parameters. From this, a and a 'are calculated, and the three-dimensional position t of the feature point 17 is obtained. Then, the three-dimensional coordinates of the feature points 17 obtained as described above are stored in an array. For example, the x, y, and z coordinates of the n-th feature point are stored in X [N + n], Y [N + n], and Z [N + n], respectively.

In step ST28, 1 is added to n.
In step ST29, it is determined whether n is equal to or less than M,
If not more than M, the process proceeds to step ST27, and if not, the process proceeds to step ST30. In step ST30, M is added to a variable N indicating the number of feature points. Then, step ST22
Return to

Subsequently, in steps ST2 to ST11 shown in FIG. 7, the height of the building is obtained by the height setting device 6. In step ST2, a variable K indicating the number of polygons is read from the shape information storage device 3, and the variable k is set to 1
Initialize to In step ST3, a variable h indicating the height of the building is initialized to 0, and a variable n indicating the feature point is initialized to 1. In step ST4, it is determined whether the n-th feature point is a k-th building point. This is performed by determining whether the projection of the n-th feature point on the ground plane (xy plane) is inside the k-th polygon or on the side. This will be described in detail with reference to the flowchart of FIG.

FIG. 10 is a flowchart for explaining the operation in step ST4 shown in FIG. here,
As shown in FIG. 5, the shape information storage device 3 stores the number of vertices of the k-th polygon in P [k] and the x and y coordinates of each vertex of each polygon in the arrays Xp and Yp. ing. Note that Xp [k] [p] and Yp [k] [p] are k
It indicates the coordinates of the p-th vertex of the th polygon. However, for convenience, Xp [k] [P [k] +1], Y
Xp [k] [1] and Yp are also used for p [k] [P [k] +1].
[K] [1] is stored.

In step ST41, a variable p is initialized to 1. In step ST42, the projection of the feature point onto the xy plane starts at the p-th vertex of the k-th polygon and starts at p +
It is determined whether the left side or the right side of the edge vector toward the first vertex exists. this is,

D = (Xp [k] [p + 1] -X [n])
(Yp [k] [p] -Y [n])-(Xp [k] [p]
−X [n]) (Yp [k] [p + 1] −Y [n])

Is determined. If this value D is negative,
On the right. This is because, as shown in FIG. 4B, since the polygon is stored in a convex shape and the row of vertices 15a is counterclockwise, if the value D is negative, the projection on the xy plane is Indicates that it is outside the polygon 15. Therefore, if the value D is negative, the n-th feature point is not the building point corresponding to the k-th polygon, ie,
It is determined that the point is not the k-th building, and the process returns to step ST4 and proceeds to step ST7. Otherwise, if the value is positive or 0, the process proceeds to step ST43, and 1 is added to p.

In step ST44, it is determined whether or not p is equal to or smaller than P [k].
Returning to 42, otherwise, it is determined that the n-th feature point is the point of the building corresponding to the k-th polygon, and the process returns.

In step ST5, z of the n-th feature point
The coordinates, that is, the height is compared with h, and if larger than h, h is updated to that value in step ST6. In this way, the height of the building is set based on the value of the higher feature point. In step ST7, 1 is added to n.

In step ST8, it is determined whether or not n is equal to or smaller than N. If n is equal to or smaller than N, the process returns to step ST4; otherwise, the process proceeds to step ST9. In step ST9,
Let h be the height H [k] of the k-th building. Step ST
At 10, 1 is added to k. In step ST11, k
Is determined to be less than or equal to K, and if less than K, step ST
3; otherwise, the process proceeds to step ST12. The height of the building is determined as described above.

In step ST12, the three-dimensional shape forming device 7 expresses the building by a polygonal pillar-shaped building shape, and draws a landscape on the display device 8. The height H [k] obtained here indicates the altitude of the highest point of the building.
For each building, if the elevation data of the point where it is constructed is obtained in the array B [k], the polygon is set as the base, and the polygonal column having the height H [k] -B [k] is set as the building shape. This is set and drawn such that the bottom surface is z = B [k]. In this case, even if the accuracy of the elevation data B [k] is not good, the height of the upper surface of the building is exactly H [k], so that the rendered scene matches the actual one.

In the present embodiment, the height of the object is set from the feature points of the captured image and the polygon, so that the height of the shape of the object such as a building can be accurately obtained. In addition to accurately reconstructing the shape of the object, the height of the object is determined by focusing only on the feature points of the image, making it much easier than reconstructing the shape of an object such as a building in detail. Can be restored.

Embodiment 2 In the present embodiment, the height of the object is obtained from the feature points of the image in the first embodiment, whereas the height of the object is obtained from the characteristic line segments of the image.

FIG. 11 is a block diagram showing a landscape modeling device according to the second embodiment. In the figure, reference numeral 21 denotes a characteristic portion extraction device for extracting an edge, which is a characteristic line segment, from an image stored in the image storage device 1, and reference numeral 22 denotes a characteristic portion extraction using the imaging information stored in the imaging information storage device 2. The position calculating device 23 calculates the three-dimensional edge position of the edge extracted by the device 21. The position calculating device 23 calculates an object based on the three-dimensional edge position calculated by the position calculating device 22 and the shape information stored in the shape information storage device 3. It is a height setting device for setting the height of an object. Other features are the same as those described in the first embodiment, and thus description thereof will be omitted.

In order to obtain the height of the building itself, it is determined to which building the edge extracted by the characteristic part extracting device 21 and whose three-dimensional position coordinates have been calculated by the position calculating device 22 belongs, and the target building is determined. Height (z
The one with the higher coordinate value is the height value of the building.

In the height setting device 23, the height of the target building itself is obtained from the three-dimensional position coordinates of the edge.
FIG. 12 is a diagram showing the relationship between the position of the polygon of the target object and the position of the edge. In the figure, reference numeral 24 denotes an edge extracted from an image, and reference numeral 24a denotes an edge which does not exist in a polygon to be projected on the xy plane, and is represented by a dotted line. Reference numeral 24b denotes an edge existing in the polygon to be projected on the xy plane, and is represented by a solid line.

In order to obtain the height of the building itself, as shown in FIG. 12, the three-dimensional position coordinates of the edge 24 on the building are obtained using the image, and the height (z coordinate value) of the building is used as the height. Ask for it. An edge is a line segment element drawn as an edge on an image due to a difference in color or brightness from the surroundings. For example, the side of the upper surface of the building is an edge due to the difference in appearance from the surrounding landscape.

The judgment of which building the edge belongs to is made in the same manner as in the first embodiment, and the ground plane (xy plane) of the edge is determined.
It is determined which of the polygons 15 the projection to the inside enters. As shown in FIG. 12, the projection of the edge on the building with respect to the polygon 15 is included in the polygon 15 and is not included in another polygon at the same time. Also, at this time, the height of the building is set to the highest edge.

FIG. 13 is a flowchart showing the operation of the second embodiment. Steps other than step ST101 and steps ST104 to ST106 are the same as the steps shown in FIG. 7 of the first embodiment, and a description thereof will be omitted.

In step ST 101, the edge of each image is extracted by the feature extracting device 21 and the position calculating device 22
Calculates the three-dimensional position of the edge of each of the extracted images. This is also based on, for example, a stereo matching method. The details are shown in the flowchart of FIG.

FIG. 14 shows step ST10 shown in FIG.
3 is a flowchart illustrating the operation of FIG. The flowchart shown in FIG. 14 differs from the flowchart shown in FIG. 8 in steps ST124, ST125, and ST.
Since the other steps are the same except for the 127, the description is omitted.

In step ST124, for each image, the edge of the image is extracted. Here, the edge means a line segment region in which the luminance value is largely different from the surroundings on the image. For the extraction of the edge, for example, a Laplacian filter shown in "Industrial Image Processing" by Shoji Ejiri, published by Shokodo, 1988 is applied. Since the zero-crossing position of the output represents an edge, this is traced, and a line segment is approximated to a line segment. A line segment whose length exceeds a predetermined threshold value among the approximate line segments is extracted as an edge.

In step ST125, an edge corresponding to the same edge is associated with each image edge of the two images. This is, for example, to calculate the similarity of pixel patterns in the vicinity of an edge and associate them when they match well. For example, the similarity is calculated by calculating the sum of squares by taking the difference between neighboring pixel values at both end points of both edges, and it is assumed that when the value is smaller than a predetermined threshold value, the values match well. Let M be the number of sets of associated edges.

In step ST127, the three-dimensional position coordinates of the n-th edge are obtained from the correspondence of the n-th edge. In this method, the three-dimensional positions of the points at both ends of the edge are obtained, and a line segment having the point at both ends is obtained. The method of finding the positions of both end points is the same as that shown in the first embodiment. The coordinates of the three-dimensional edge obtained in this way are stored in an array. The x, y, and z coordinates of the start point of the n-th edge are SX [N + n], SY [N + n], and SZ [N
+ N] and the end point x, y, z coordinates are expressed as EX [N +
n], EY [N + n] and EZ [N + n].

In step ST104 shown in FIG.
It is determined whether the n-th edge is an edge of the k-th building. This is performed by determining whether or not the projection of the n-th edge on the ground plane (xy plane) is inside or on the side of the k-th polygon. If both end points of the projection are inside or on the side of the polygon, the projection is also judged to be inside or on the side. This will be described in detail with reference to the flowchart of FIG.

FIG. 15 shows step ST10 shown in FIG.
6 is a flowchart illustrating the operation of FIG. Step 1
Other than 42 is the same as that shown in FIG. 10 of the first embodiment, and the other description is omitted.

In step ST142, it is determined whether the two end points of the projection are located on the left or right of the k-th polygon from the p-th vertex to the (p + 1) -th vertex. this is,
Similar to the determination method of the first embodiment,

Ds = (Xp [k] [p + 1] -XS
[N]) (Yp [k] [p] -YS [n])-(Xp
[K] [p] -XS [n]) (Yp [k] [p + 1]-
YS [n]) De = (Xp [k] [p + 1] -XE [n]) (Yp
[K] [p] -YE [n])-(Xp [k] [p] -X
E [n]) (Yp [k] [p + 1] -YE [n])

The judgment is made as follows. If either of the values Ds and De becomes negative, the corresponding end point is on the right side,
It is determined that the edge is not the edge of the target building.

In step ST105 shown in FIG.
Comparing the z coordinate, that is, the height with h at both end points of the nth edge, and if at least one is greater than h,
In step ST106, h is updated to the larger value. That is, the height of the higher edge is set as the height of the building.

In the present embodiment, the height of the object is set using the edge to create a landscape.
Compared to a case where a landscape is generated by feature points, it is less susceptible to noise on an image, and a correct landscape can be obtained stably.

Embodiment 3 In the present embodiment, the height of the object is obtained from the feature points of the image in the first embodiment, and the height of the object is obtained from the feature line segments in the second embodiment. It was made.

FIG. 16 is a block diagram showing a landscape modeling device according to the third embodiment. In the figure, reference numeral 31 denotes a characteristic portion extraction device for extracting a characteristic region from an image stored in the image storage device 1, and reference numeral 32 denotes an extraction by the characteristic portion extraction device 31 using the imaging information stored in the imaging information storage device 2. A position calculating device 33 for calculating the three-dimensional region position of the calculated characteristic region, and a height of the object based on the three-dimensional region position calculated by the position calculating device 32 and the shape information stored in the shape information storage device 3. It is a height setting device for setting the height. Other features are the same as those described in the first embodiment, and thus description thereof will be omitted.

In order to obtain the height of the object building itself, it is determined which region the area extracted by the feature extracting device 31 and the three-dimensional position coordinates of which are calculated by the position calculating device 32 belongs to. Height (z
The one with the higher coordinate value is the height value of the building.

In the height setting device 33, the height of the target building itself is obtained from the three-dimensional position coordinates of the area. FIG. 17 is a diagram showing the relationship between the position of the polygon of the target building and the position of the characteristic region. In the figure, 34 is a feature region extracted from the image, 34a is a feature region belonging to the building of the polygon 15, 34b is a feature region not belonging to the building of the polygon 15, 35a is the center of gravity of the feature region 34a, 35
b is the center of gravity of the characteristic region 34b.

To obtain the height of the building itself, as shown in FIG. 17, the three-dimensional position coordinates of the characteristic region 34 on the building are obtained using the image, and the height (z coordinate value) of the building itself is obtained from the height (z coordinate value). Find the height. A characteristic region is a region element drawn as a region on an image due to a difference in color or brightness from the surroundings. For example, the surface of a building is an area because it has an area of substantially the same color and brightness as the surrounding landscape.

The determination as to which building the characteristic region belongs to is made in the same manner as in the first embodiment, the center of gravity of the characteristic region is obtained, and the projection of this center of gravity onto the ground plane (xy plane) is Judge if you go inside. As shown in FIG. 17, the projection of the center of gravity of the feature area onto the xy plane determines that the area 34 a that enters the interior of the polygon 15 of the building belongs to the building of the polygon 14, and the area 34 b that does not enter the interior of the polygon 14 Is not considered to be an area. Then, the height (z component) of the highest region among the regions belonging to this building is defined as the height of the building.

In the present embodiment, it is determined which building the region belongs to based on whether or not the projection of the center of gravity of the region onto the xy plane is inside the polygon. However, this is not particularly limited. Instead, for example, it may be determined which building the area belongs to based on whether or not the projection of the area around the xy plane is inside the polygon.

In the present embodiment, the height of the target object is set using the region to generate the landscape, so that the feature portion is more easily extracted than the feature points and edges are extracted. And a stable and correct landscape can be obtained.

Embodiment 4 In the present embodiment, 3 of the object reproduced based on the height obtained in the first to third embodiments is used.
The three-dimensional building shape is superimposed on the captured image on the display, and when the height of the reproduced building shape is not appropriate, the height of the building shape is corrected.

FIG. 18 is a block diagram showing a landscape modeling device according to the fourth embodiment. In the figure, reference numeral 41 denotes a shape information storage device for storing building shape information such as polygons and heights necessary for forming a three-dimensional shape obtained as shown in the first to third embodiments, and 42 denotes an image pickup device. The images stored in the image storage unit 1 are displayed by matching the viewpoints with the imaging information stored in the information storage device 2, and a three-dimensional image formed from the information stored in the shape information storage device 41. A three-dimensional shape forming device 43 for reproducing a shape is a display device for displaying a three-dimensional shape formed by the three-dimensional shape forming device 42 and an image.

Reference numeral 44 denotes an input device constituted by a mouse or a keyboard, etc., and reference numeral 45 denotes a height adjusting device for adjusting the height of the three-dimensional shape of the object based on input information from the input device. Other features are the same as those described in the first embodiment, and thus description thereof will be omitted.

Next, the operation of the landscape modeling device shown in FIG. 18 will be described. FIG. 19 is an explanatory diagram illustrating the operation of the landscape modeling device according to the fourth embodiment. In the figure, reference numeral 46 denotes a landscape displayed on the display of the display device 43, and 47 denotes an image displayed on the display, which displays an image stored in the image storage device 1. 47a is a building on the image, 47b is a mountain on the image, and 47c is a road on the image. 48 is a three-dimensional shape of a building as an object displayed on the display, and 49 is a cursor.

In the fourth embodiment, when the height of the obtained three-dimensional shape is not correct, the height of the three-dimensional shape is adjusted to the height of the object on the captured image. The object and the obtained three-dimensional shape of the object are collated, and the height value of the three-dimensional shape is adjusted interactively.

As shown in FIG. 19A, the display device 43
Thus, based on the imaging information stored in the imaging information storage unit 2, the viewpoint of the image matches the viewpoint of the three-dimensional shape, and the image stored in the image storage device 1 and the three-dimensional The shape is superimposed and displayed. At this time, a solid line having a color and thickness that can be clearly distinguished from the object drawn on the image so that the object on the image can be easily distinguished from the three-dimensional shape of the object. Alternatively, the three-dimensional shape of the object is displayed by a dotted line.

While checking the collation result displayed in this way, the user can use the mouse cursor 49 displayed on the display device 43 to adjust the height of the three
Select a dimensional shape. At this time, the selected 3
In order to facilitate the distinction between the three-dimensional shape and the three-dimensional shape, a three-dimensional shape selected by a solid line or a dotted line having a color and thickness that can be clearly distinguished is displayed.
Then, as shown in FIG. 19B, the height of the selected three-dimensional shape is adjusted to an appropriate height by using a mouse or numerical input.

FIG. 20 is a flowchart showing the operation of the fourth embodiment. Step ST300 includes steps ST1 to ST1 of the flowchart shown in FIG.
The operation up to step ST11 is performed. However, it is assumed that the obtained information on the three-dimensional shape is stored in the shape information storage device 41.

In step ST301, a variable k indicating the selected building is initialized to 1. In step ST302, a three-dimensional shape is overwritten on the image in accordance with the viewpoint position, the optical axis direction, and the focal length of the image, and the result is displayed on the display device 4.
Draw on the display in 3. At this time, the three-dimensional shape is represented by, for example, sides representing the polygonal prism.
In other words, the three-dimensional shape is perspective-transformed based on the coordinates of each vertex of the polygon and the vertex given the height H [k] in accordance with the viewpoint position of the image, the optical axis direction, and the focal length, and connects them by sides. Draw by doing. The building shape of the k-th building, which is the selected building, is drawn with a different color or a thicker side so as to be distinguished from other building shapes.

In the following steps, the operation of adjusting the height of the building by the height adjusting device 45 will be described. In step ST303, it is determined whether or not to continue the processing. If not, the processing ends. The determination here is made by giving a specific input with the input device 44 such as a mouse or a keyboard at the time of the end.

In step ST304, it is determined whether there is an input for changing the selection of a building. This change is also performed by giving a specific operation using the input device 44 such as a mouse or a keyboard. For example, inputting the number of the corresponding polygon, or moving the cursor 49 drawn on the screen with the mouse onto the three-dimensional shape 48 to be specified, and obtaining the number of the three-dimensional shape 48 by inputting the mouse button. Do so. If there is a change input, in step ST305, the variable k is changed to a number indicating the newly selected building.

In step ST306, it is determined whether or not there is an input for changing the height of the selected three-dimensional shape 48. This is also performed by giving a specific operation using the input device 44 such as a mouse or a keyboard. For example,
If a key on the keyboard corresponds to an increase in height and another key corresponds to a decrease, and that key is pressed,
The height value is increased or decreased by a constant value.

Further, the value of the height may be directly input as a numerical value. If there is an input, the height value H [k] of the k-th building selected in step ST307 is changed, and new drawing is performed in step ST302. If there is no input, since there is no change in the landscape, no drawing is performed and step S
Proceed to T303.

In the present embodiment, a three-dimensional shape having an inappropriate height is selected from the landscape drawn by forming the three-dimensional shape, and the height is adjusted. The three-dimensional shape can be further improved, and a more appropriate three-dimensional shape can be obtained. Further, since the height of the three-dimensional shape can be adjusted while confirming the correspondence with the image, more accurate three-dimensional shape (building) can be reproduced.

Embodiment 5 FIG. In the present embodiment, 3 of the object reproduced based on the height obtained in the first to third embodiments is used.
The three-dimensional building shape is displayed on the display so as to overlap the captured image. If the reproduced building shape (polygon) is not appropriate, the building shape polygon is corrected.

FIG. 21 is a block diagram showing a landscape modeling device according to the fifth embodiment. In the figure, 51 is a map storage device for storing a map, 52 is a map drawing device for drawing a map stored in the map storage device 51 and a polygon stored in the shape information storage device 41, and 53 is a map drawing device. A display device for displaying the map and polygon drawn by 52 and the three-dimensional shape and image drawn by the three-dimensional shape forming device 42; 54 is a position of the three-dimensional shape polygon based on input information from the input device 44; And a shape adjusting device for correcting the shape. Others are described in Embodiment 4.
The description is omitted because it is the same as that described above.

In the fifth embodiment, for example, when the building shape shown on the map is not the shape of the building itself but data representing a site including a building other than the building itself, such as a garden, the shape information If the polygon stored in the storage device 41 is not correct, the position and the shape of the building-shaped polygon are interacted so that the three-dimensional building-shaped polygon becomes the polygon of the object on the captured image. It is intended to be adjusted on a regular basis.

The image obtained by capturing the landscape is stored in the image storage device 1, the image capturing information such as the viewpoint position, the optical axis direction and the angle of view at the time of image capturing is stored in the image storage device 2, and the map is stored in the map storage device 51.
It is assumed that polygons representing buildings and their heights are further stored in the shape information storage device 41. The image may be a grayscale image or a color image, and the height of the building stored in the shape storage device 41 can be obtained by, for example, the method described in the first to third embodiments.

Next, the operation of the landscape modeling device shown in FIG. 21 will be described with reference to the flowchart shown in FIG.
FIG. 22 is a flowchart for explaining the operation of the fifth embodiment. FIG. 23 is a diagram showing a landscape (including a three-dimensional shape) and a map (including polygons) displayed by the display device 53. In the drawing, reference numeral 55 denotes a display unit on the display of a display device 53 including a landscape (including three-dimensional shape) display unit 55a and a map (including polygon) display unit 55b, and 56 denotes a map display unit 55b on the display.
Is the selected polygon displayed in. Others are the same as those described in Embodiment 4 with reference to FIG.

At step ST401 shown in FIG.
As shown in FIG. 23, the three-dimensional shape forming device 42 draws an image 47 in which the three-dimensional shape 48 of the building (hereinafter, also referred to as a building shape) is overwritten on the landscape display unit 55 a of the display monitor. . At this time, the building shape 48
Indicates the viewpoint position, the optical axis direction and the focal length in the image 47.
Draw according to the object.

The map drawing device 52 draws the polygon 56 along with the map on the map display section 55b of the display monitor in accordance with the position of the target building.
At this time, the building shape 48 is represented by a side representing the polygonal pillar, and the polygon 56 is also represented by the side. FIG.
In the example, the building shape 48 and the polygon 56 are represented by thick lines or the like, and are represented in such a manner that they can be distinguished from the building 47a, the mountain 47b, the road 47c on the image, or the polygon on the map. In addition, the three-dimensional shape 48 is rendered by performing hidden surface hidden line elimination processing.

The following steps are performed by the shape adjusting device 54.
Describes the operation when changing the position and shape of the polygon. In step ST402, it is determined whether or not to continue the process, and otherwise, the process ends. The determination here is, for example, that the shape of the building on the image displayed on the drawing display unit 55a is different from the shape of the obtained three-dimensional shape, that is, the shape of the building on the image is the base of the building shape. If the three-dimensional polygon is different from the
When it is determined that the polygon needs to be corrected, the process is continued, and when it is finished, the correction is performed by giving a specific input using the input device 44 such as a mouse or a keyboard.

In step ST403, it is determined whether or not there is an input for correcting the position of the vertex of the polygon 56. This is done as follows. The cursor 49 drawn on the display monitor by the input device 44 is moved to the map display 55
b) is moved to one vertex of the polygon 56 to be corrected shown on b, and the vertex is designated by inputting a button. If there is this input, it is determined that there is a correction input.

In step ST404, the coordinates of the selected vertex on the polygon 56 are changed to the coordinates on the map indicated by the position of the cursor 49. Step ST40
In step 5, a map in which the polygon 56 is overwritten and an image 47 in which the building shape 48 is overwritten are drawn on the display 46 by the same operation as in step ST401. At this time,
In drawing the building shape 48, the selected vertices are distinguished from the other vertices.
Draws the side where the vertices are extended vertically in different colors. Also,
The vertices of the polygon 56 are also displayed in different colors.

In step ST406, it is determined whether or not the position correction processing has been completed.
Go to 401. This is instructed to end by button input or key input by the input device 44. in this way,
In step ST404, the position of the vertex is changed while confirming the building shape 48 with the landscape and the map drawn in step ST405.

FIG. 24 is a diagram for explaining the operation when correcting the positions of the vertices of a polygon. FIG. 24 (a) is a diagram showing a state before the correction, and FIG. 24 (b) is a diagram after the correction. It is a figure showing a state. In the figure, 48a is a side corresponding to the vertex 56b and configuring the building shape 48, 56a is a polygon selected on the map display unit 55b displayed on the display, and 56b is a vertex configuring the polygon 56a. Other points are the same as those described with reference to FIG.

Thus, for example, in step ST401
In FIG. 24, when the landscape shown in FIG. 24A is drawn, the vertex 56b of the polygon 56a corresponding to the building shape 48 shown in FIG. 24A is moved on the map, and the shape of the polygon is appropriately adjusted. To a different shape. With this change, the position of the side 48a corresponding to the vertex 56b is also changed,
The building is corrected to an appropriate building shape shown in FIG.

Here, the position of one vertex is moved, but this is not particularly limited, and all vertices may be moved. For example, if all vertices of a polygon are moved in the same direction by the same distance, the polygon itself can be moved while the shape of the polygon is maintained. Therefore, even if only the position of the polygon of a certain building is different, it can be corrected to the correct position of the polygon.

Next, in step ST403, if there is no input for correcting the position of the vertex of the polygon 56, the process proceeds to step ST407. In step ST407, it is determined whether or not there is an input for adding a vertex of the polygon. This is performed as follows. The cursor 49 drawn on the display monitor by the input device 44 is moved to one side of the polygon 56 to which the vertex shown on the map display section 55b is added, and the vertex is designated by a button or key input. If there is this input, it is determined that there is an additional input. However, the button or key input in this case is
Buttons or keys different from those used in step ST403 are assigned.

In step ST408, a new vertex of the polygon 56 is added on the selected side. At this time, the number of vertices of the polygon 56 is increased by one, and the array is reconfigured so that the polygon 56 is stored counterclockwise starting with the one having the largest x coordinate value. The added vertex is regarded as the selected vertex, and the process proceeds to step ST404. Hereinafter, step ST4
Since the steps after 04 are the same as those described above, the description will be omitted.

FIG. 25 is a diagram for explaining the operation when correcting the shape of a polygon (adding vertices).
(A) is a diagram showing a state before correction, and FIG. 25 (b) is a diagram showing a state after correction. In the figure, 48a is a side corresponding to the vertex 56b and constituting the building shape 48, 56a is a selected polygon on the map display section 55b displayed on the display, and 56b is a polygon 56a
And 56c are sides forming the polygon 56a. Others are the same as those described with reference to FIG.

As described above, for example, in step ST401
In FIG. 25A, when a landscape as shown in FIG. 25A is drawn, as shown in FIG.
A vertex 56b is added to the side 56c of the side 6a and its position is moved on the map, so that the side 4 corresponding to the vertex 56b is
8a is added to the correct position, and the shape is changed to the shape shown in FIG.

Next, in step ST407, if there is no input to add a vertex of the polygon, step ST4
Go to 09. In step ST409, it is determined whether or not there is an input for deleting a vertex of the polygon. This is performed as follows. The cursor 49 drawn on the display monitor by the input device 44 is moved to a vertex to be deleted shown on the map display section 55b, and the vertex is designated by a button or key input. If there is this input, it is determined that there is a deletion input. However, in this case, a button or a key input different from those used in steps ST403 and ST407 is assigned.

In step ST410, step ST4
Similar to 05, a landscape and a map are drawn using different colors for the selected vertex and its side. Step ST41
In step 1, the deletion is confirmed, and the vertex selected in step ST412 is deleted. At this time, the number of vertices of the polygon 56a is reduced by 1, and the array is reconfigured so that the polygon 56a is stored in a counterclockwise direction with the one having the largest x coordinate value at the top.

FIG. 26 is a diagram for explaining the operation when the shape of the polygon is corrected (vertex is deleted).
FIG. 26A is a diagram showing a state before correction, and FIG. 26B is a diagram showing a state after correction. In the figure, 48a is a side corresponding to the vertex 56b and constituting the building shape 48, 56a is a selected polygon on the map display section 55b displayed on the display, and 56b is a polygon 56a
Are vertices. Others are the same as those described with reference to FIG.

As described above, for example, in step ST410
In FIG. 26A, when a landscape as shown in FIG. 26A is drawn, as shown in FIG.
By selecting the vertex 56b to be deleted in 6a and deleting the vertex 56b, the side 48a corresponding to the vertex 56b is deleted, and the shape is changed to the shape shown in FIG.

In this embodiment, the position and the shape of the polygon are adjusted by the shape adjusting device. However, the height value of the three-dimensional shape stored in the shape adjusting device is also adjusted. Then, the three-dimensional shape can be reproduced more accurately.

In the present embodiment, the position of the vertex of the polygon, which is the outline thereof, is adjusted while checking the collation result between the image of the scenery and the three-dimensional shape. And an accurate three-dimensional shape can be formed.

Also, while checking the collation result between the captured image of the landscape and the three-dimensional shape, the vertices are added to the polygon which is the outline, so that the shape of the polygon can be changed, and the accurate three-dimensional shape can be changed. Can be formed.
In addition, since the vertex of the polygon, which is the outline of the image, is deleted while confirming the comparison result between the image of the landscape and the three-dimensional shape, the shape of the polygon can be changed, and an accurate three-dimensional shape can be formed. It becomes possible.

At this time, by specifying and adjusting the polygons on the map, the polygons do not overlap each other, so that the target polygons and vertices can be easily specified, and other polygons such as roads shown on the map can be used. The correct polygon can be obtained intuitively from the positional relationship with the object, and the three-dimensional shape can be easily corrected. Further, since the three-dimensional shape is corrected as needed and drawn on the image, an accurate three-dimensional shape can be easily obtained.

Embodiment 6 FIG. FIG. 27 shows the sixth embodiment.
1 is a block diagram showing a landscape modeling device of FIG. In the figure, reference numeral 61 denotes a shape adding device for adding a building shape.
The other parts are the same as those described in the fifth embodiment, and the description is omitted.

FIG. 28 is an explanatory diagram for explaining the operation of the sixth embodiment. In the figure, 62 is a projecting portion of the building, 62a is a building projecting shape obtained by giving a height value to the polygon 63, 63 is a polygon showing the contour shape of the projecting portion of the added building, 63a is a vertex of the polygon 63 It is.

In the sixth embodiment, the structure of the building is adjusted interactively. Although the shape of the building is expressed as a polygonal pillar formed by giving a height value to the polygon, as shown in FIG. There are many examples. In such a case, when the building shape is determined at the highest point, the difference in height in other portions becomes large and does not match the actual landscape. In addition, there are cases where the projections are formed even on a horizontal plane, and in this case, the landscape is not correctly reproduced. Therefore, in the sixth embodiment, the projection is made to have an independent building shape so as to match the actual scene.

It should be noted that the image of the landscape (either a grayscale image or a color image) is stored in the image storage device 1 and the imaging information such as the viewpoint position, the optical axis direction, and the angle of view at the time of image capturing is stored in the imaging information storage device 2. It is assumed that a map is stored in the map storage device 51, and a polygon representing an object such as a building and its height are stored in the shape information storage device 41.
Also, regarding the height of the building, Embodiments 1 to 1 of the present invention
3 may be obtained.

The operation of the landscape modeling device shown in FIG. 27 will be described below with reference to the flowchart shown in FIG.
FIG. 29 is a flowchart showing the operation of the landscape modeling device shown in FIG.

In step ST501, as in step ST401 of the fifth embodiment, as shown in FIG.
The image 47 in which the building shape 48 is overwritten by the three-dimensional shape forming device 42 is displayed on the landscape monitor 55a of the display monitor.
To draw. At this time, the building shape 48 is drawn such that the viewpoint position, the optical axis direction, and the focal length match those of the image 47. The map drawing device 52 draws the polygon 56 on the map on the map display section 55b of the display monitor in accordance with the position of the building. At this time, the building shape 48 is represented by a side representing the polygonal pillar, and the polygon 56 is also represented by the side.

The following steps are operations controlled by the shape adding device 61. In step ST502, it is determined whether or not to continue the process, and if not, the process ends. The determination here is made by giving a specific input with the input device 44 such as a mouse or a keyboard at the time of the end.

In step ST503, as shown in FIG. 28, a vertex representing the projection 62 is designated for the polygon 56 corresponding to the building having the projection 62. This is performed as follows. The cursor 49 drawn on the display monitor is moved on the map by the input device 44 such as a mouse or a keyboard, and further sequentially moved along the side to a position that is a vertex representing the protruding portion 62. Alternatively, the vertex 63a is designated by a key input, and is simultaneously drawn on the map. The input is also terminated by giving a specific button or key input.

In step ST504, step ST5
Polygon 63 composed of vertex 63a specified by 03
Register additionally. Here, the variable K indicating the number of polygons is increased by 1, and the added polygon 63 is set as the new K-th polygon. The p-th coordinate of the input vertex 63a is stored in the arrays Xp [K] [p] and Yp [K] [p]. Also in this case, the configuration is such that the one having the largest x-coordinate value is in the counterclockwise direction, starting with the one having the largest x-coordinate value. Furthermore, the vertex 63a specified in step ST503 is added to P [K] indicating the number of vertices.
Is stored, and the height H [K] is set to a preset initial value. Elevation B of the installation point of the building protrusion shape 62
[K] uses the value closest to the location. Alternatively, the elevation value of the building closest to the building projection shape 62 may be used.

In step ST505, the height of the building projection shape 62 generated by the polygon 63 is corrected.
This operation is equivalent to the operation from step ST302 to step ST307 of the flowchart shown in FIG. 20 of the fourth embodiment. However, the building projection shape 63 is regarded as a building shape, and in the drawing of step ST302, a map is also drawn in accordance with the scenery as shown in FIG. 23 as in step ST501. Further, in order to correct only the height of the newly added building projection shape 62a, k indicating the selected building is always K.

Thus, for example, the building shape 48 initially drawn as shown in FIG. 28A can be represented by the building protrusion shape 62a and the building shape 48 as shown in FIG. 28B. And it is possible to reproduce the landscape more accurately. At this time, the building projection shape 6
By inputting the polygon 63 indicating 2, the correct polygon shape can be intuitively obtained from the positional relationship with other polygons, roads, and the like shown on the map, and the building shape can be easily corrected. In addition, by overwriting the landscape, while checking the shape of the building protrusions together with the building shape,
The height can be adjusted, and an accurate landscape can be reproduced.

In the first to third embodiments, the measurement of the three-dimensional position coordinates of a feature point, an edge, or an area is performed.
The feature points, edges, or areas are detected and associated with pixel values in the vicinity thereof, but this is not particularly limited, and is directly performed on the image using an input device such as a mouse or a keyboard. The position of the characteristic pixel, the characteristic line segment, or the characteristic region and the correspondence between the two images may be instructed.

In the first to third embodiments, the characteristic points
The measurement of the three-dimensional position coordinates of an edge or a region is performed based on the principle of triangulation using two images. However, the measurement may be performed using a larger number of images. May be performed by another method, for example, a distance measurement method using a laser.

In the present embodiment, the elevation value B [k]
However, if the area is limited, the terrain is considered to be horizontal, and the ground plane is set to the xy plane, and all B [k] =
It may be configured as 0.

In the fourth to sixth embodiments, the building shape selected by the three-dimensional shape forming device and the map drawing device or the color of the side of the polygon is changed and displayed. It may be configured so that it can be identified from a map or a map, and the line type such as the line width or broken line may be changed and displayed.

In the fourth to sixth embodiments, the height of the building shape is obtained by the operation described in the first to third embodiments. However, the height of the building shape is determined by another method, for example, the conventional method. And the following operations may be performed on the obtained building shape.

In the fifth and sixth embodiments, the landscape and the map drawn by the three-dimensional shape forming device and the map drawing device are displayed on the same display monitor. You may comprise.

In the sixth embodiment, the description has been made on the assumption that a building projection shape of a building shape is added. However, a building shape of a building which is not shown on the map but exists on the image is newly added. May be operated.

Further, in the above embodiment, the characteristic part extracting device, the position calculating device, the height setting device, the three-dimensional shape forming device, the height adjusting device, the map drawing device, the shape adjusting device, and the shape adding device are used. However, these operations may be executed by software using a computer.

In the first to sixth embodiments, the image,
Camera viewpoint position, optical axis direction and angle of view information at the time of image shooting,
Although the information on the polygon and the height and the map are stored in different storage devices, they may be stored in the same storage device.

The images used in the first to sixth embodiments may be aerial photographs taken from an aircraft or a helicopter.

Although the images used in the first to third embodiments are multi-valued images of light and shade, color images may be used. In this case, it is also possible to adopt a configuration in which only the red light of the building is extracted as a characteristic portion such as a feature point, or a blue, white, or gray area is regarded as a background sky or cloud, and the extraction of a characteristic figure is omitted. .

[0166]

Since the present invention is configured as described above, it has the following effects.

A landscape modeling device according to the present invention comprises: an image storage means for storing at least two images of the same object taken from different directions; and an imaging information storage means for storing imaging information at the time of capturing each of the images. A shape storage unit that stores outline information of a horizontal cross section of the object; a feature unit extraction unit that extracts a feature unit of each image stored in the image storage unit; The characteristic portion, height setting means for obtaining the height of the object based on the imaging information stored in the imaging information storage means and the contour information stored in the shape storage means, the shape storage means 3 based on the stored contour information and the height of the object set by the height setting means.
Since the three-dimensional shape forming means for forming the three-dimensional shape is provided, the accurate height of the building can be obtained by simple processing, and the correct landscape can be efficiently reproduced.

Further, the height setting means has a horizontal component and a height component of the characteristic part based on the characteristic part extracted by the characteristic part extracting means and the imaging information stored in the imaging information storage means. A position information calculating means for calculating three-dimensional position information;
A feature part belonging to the object is extracted based on the horizontal component of the dimensional position information and the outline information, and a height direction component of the feature part belonging to the object is set to the height of the object. The height of the object can be easily obtained.

Further, since the height of the highest feature portion is taken as the height of the object, the height of the highest point can be assumed even if the height of the upper part of the object is not constant. Even if the line of sight is not clear, it is possible to realize a correct landscape without erroneously determining that the line of sight can be seen.

Further, since the characteristic portion is a characteristic point, the three-dimensional position of the characteristic portion can be easily calculated by the existing image processing technique, and the landscape can be efficiently restored.

Further, since the characteristic portion is a characteristic line segment, the three-dimensional position of the characteristic portion can be easily calculated by the existing image processing method, the scene can be efficiently restored, and the image can be accurately reproduced even in an image having much noise. The scenery can be recreated.

Further, since the characteristic portion is a characteristic region, the extraction of the characteristic portion and the three-dimensional position of the characteristic portion can be easily calculated by the existing image processing technique, and the landscape can be efficiently restored.

Further, the landscape modeling device according to the present invention comprises: an image storage means for storing an image of a target object;
Imaging information storage means for storing imaging information at the time of imaging of the image, three-dimensional shape forming means for forming a three-dimensional shape of the object, and the imaging information storage means based on the imaging information stored in the imaging information storage means Display means for displaying the object stored in the image storage means in association with the three-dimensional shape of the object formed by the three-dimensional shape forming means; and receiving input based on the display result of the display means. And the height of the three-dimensional shape of the object is adjusted, so that it is possible to change the height of the building once set, and to correct the scenery more accurately. it can.

Further, the height adjusting means adjusts the height of the selected three-dimensional shape based on the display result based on the input information which is input after confirming the display result. It can be restored and easily changed to the correct value.

Further, the display means displays the three-dimensional shape selected by the height adjusting means and another three-dimensional shape in different formats, so that the selected three-dimensional shape can be easily identified.

Further, since the display means displays the object stored in the image storage means and the three-dimensional shape of the object formed by the three-dimensional shape forming means in different formats, the display means is easily selected. The building can be identified.

[0177] The landscape modeling device according to the present invention includes a map storage means for storing a map corresponding to an object,
Shape storage means for storing contour information of a horizontal cross section of the object; three-dimensional shape forming means for forming a three-dimensional shape of the object based on the contour information stored in the shape storage means; Displaying the object stored in the image storage unit and the three-dimensional shape of the object formed by the three-dimensional shape forming unit in association with each other based on the imaging information stored in the information storage unit; Display means for displaying the object on the map stored in the map storage means and the contour formed by the contour information stored in the shape storage means in association with each other; Correction means for correcting the contour information of the target object based on the input based on the input information, so that the position and shape of the contour once set can be corrected, so that a more accurate landscape can be corrected. Rukoto can. Further, the positional relationship between the object on the map and the contour becomes clear, and the position and shape of the contour can be easily corrected.

Further, the correction means corrects the outline information of the selected outline based on the display result based on the input information inputted after confirming the display result, so that the user can interactively correct the outline information. It is possible to easily restore an accurate landscape.

Further, the display means displays the outline selected by the correction means and another outline in different modes, so that the selected outline can be easily identified.

Further, since the correcting means corrects the position of the contour, even if the position of the contour according to the contour information is different from the position on the map, it is possible to correct the contour information so as to be an appropriate position. it can.

Further, since the correcting means corrects the shape of the contour, even if the shape of the contour based on the contour information is different from the shape on the map, the correcting means can correct the contour information so as to have an appropriate shape. it can.

Further, the display means displays the object on the map stored in the map storage means and the contour formed by the contour information stored in the shape storage means in different formats, so that the display means can be easily selected. The building that was done can be identified.

Also, since the correction means adds a new contour, a building shape having a plurality of contours can be obtained.
A more accurate landscape can be realized.

Further, since the three-dimensional shape forming means forms the three-dimensional shape including the contour information relating to the new contour, the three-dimensional shape forming means includes, for example, the object such as a water tower of a building or an elevator of an apartment house in the contour information. If there are shapes that are not present, a more accurate building shape can be obtained in consideration of these shapes.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a landscape modeling device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an object and an image obtained by capturing the object.

FIG. 3 is a diagram showing imaging information stored in an imaging information storage device.

FIG. 4 is a diagram showing a map and polygons formed from the map.

FIG. 5 is a diagram showing information on polygons stored in a shape storage device.

FIG. 6 is a diagram for explaining the operation of the first embodiment of the present invention.

FIG. 7 is a flowchart showing the operation of the first embodiment of the present invention.

FIG. 8 is a flowchart showing the operation of the first embodiment of the present invention.

FIG. 9 is a diagram for explaining the operation of the first embodiment of the present invention.

FIG. 10 is a flowchart showing the operation of the first embodiment of the present invention.

FIG. 11 is a diagram illustrating a landscape modeling device according to a second embodiment of the present invention.

FIG. 12 is a diagram for explaining an operation according to the second embodiment of the present invention.

FIG. 13 is a flowchart showing an operation according to the second embodiment of the present invention.

FIG. 14 is a flowchart showing the operation of the second embodiment of the present invention.

FIG. 15 is a flowchart showing the operation of the second embodiment of the present invention.

FIG. 16 is a diagram illustrating a landscape modeling device according to a third embodiment of the present invention.

FIG. 17 is a diagram for explaining the operation of the third embodiment of the present invention.

FIG. 18 is a diagram illustrating a landscape modeling device according to a fourth embodiment of the present invention.

FIG. 19 is a diagram for explaining the operation of the fourth embodiment of the present invention.

FIG. 20 is a flowchart showing the operation of the fourth embodiment of the present invention.

FIG. 21 is a diagram illustrating a landscape modeling device according to a fifth embodiment of the present invention.

FIG. 22 is a flowchart showing an operation according to the fifth embodiment of the present invention.

FIG. 23 is a diagram for explaining the operation of the fifth embodiment of the present invention.

FIG. 24 is a diagram for explaining the operation of the fifth embodiment of the present invention.

FIG. 25 is a diagram for explaining the operation of the fifth embodiment of the present invention.

FIG. 26 is a diagram for explaining the operation of the fifth embodiment of the present invention.

FIG. 27 is a diagram illustrating a landscape modeling device according to a sixth embodiment of the present invention.

FIG. 28 is a diagram for explaining the operation of the sixth embodiment of the present invention.

FIG. 29 is a flowchart showing the operation of the sixth embodiment of the present invention.

FIG. 30 is a block diagram showing a conventional landscape modeling device.

FIG. 31 is a diagram showing a map, polygons, and height information.

FIG. 32 is a diagram for explaining the operation of a conventional landscape modeling device.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 image storage device 2 imaging information storage device 3 shape information storage device 4 feature extraction device 5 position calculation device 6 height setting device 7 three-dimensional shape forming device 8 display device 9 object 10 imaging point 11 image 12 object on image Object 13 Map 14 Object shape on map 15 Polygon 16 Feature point 17 Feature point on image 18 Line of sight vector 19 Optical axis vector 20 Position corresponding to feature point 20a Epipolar straight line 21 Feature extraction unit 22 Position calculation unit 23 Height Setting device 24 Edge 31 Feature extraction device 32 Position calculation device 33 Height setting device 34 Area 35 Center of gravity 41 Shape information storage device 42 Three-dimensional shape forming device 43 Display device 44 Input device 45 Height adjustment device 46 Landscape on display 47 Image 48 3D shape on display 48a Sides 4 that make up building shape Cursor 51 Map storage device 52 Map drawing device 53 Display device 54 Shape adjustment device 55 Display unit 56 Polygon 61 Shape addition device 62 Projection part 63 Polygon 101 Polygon data storage device 102 Elevation data storage device 103 Computing device 104 Display device 105 Map 106 Map Upper building 107 Landscape on display 108 Three-dimensional building shape 109 Bottom of building 110 Average height of building

Claims (18)

[Claims] 1. An image storage means for storing at least two images of the same object taken from different directions, an imaging information storage means for storing imaging information when each of the images is taken, Shape storage means for storing section outline information; feature extraction means for extracting a feature of each image stored in the image storage means; a feature extracted by the feature extraction means; Height setting means for obtaining the height of the object based on the imaging information stored in the storage means and the contour information stored in the shape storage means, and the contour information stored in the shape storage means A landscape modeling device comprising: a three-dimensional shape forming unit that forms a three-dimensional shape of the object based on the height of the object set by the height setting unit. 2. The height setting means has a horizontal direction component and a height direction component of the characteristic part based on the characteristic part extracted by the characteristic part extraction part and the imaging information stored in the imaging information storage part. A position information calculating means for calculating three-dimensional position information; extracting a characteristic part belonging to the object based on the horizontal component and the contour information of the three-dimensional position information calculated by the position information calculating means; 2. The landscape modeling device according to claim 1, wherein a height direction component of a characteristic part belonging to the object is set to a height of the object. 3. The landscape modeling apparatus according to claim 2, wherein the height of the highest feature portion is set as the height of the object. 4. The landscape modeling device according to claim 1, wherein the characteristic part is a characteristic point. 5. The landscape modeling device according to claim 1, wherein the characteristic part is a characteristic line segment. 6. The landscape modeling device according to claim 1, wherein the characteristic portion is a characteristic region. 7. An image storage means for storing an image of an object, an image information storage means for storing imaging information at the time of imaging the image, and an image information storage means for forming a three-dimensional shape of the object.
A three-dimensional shape forming unit, an object stored in the image storage unit based on the imaging information stored in the imaging information storage unit, and a three-dimensional shape of the object formed by the three-dimensional shape forming unit A landscape modeling apparatus comprising: display means for displaying the information in association with each other; and height adjustment means for adjusting the height of the three-dimensional shape of the object in response to an input based on the display result of the display means. 8. The apparatus according to claim 7, wherein the height adjusting means adjusts the height of the three-dimensional shape selected based on the display result based on the input information input after confirming the display result. Landscape modeling equipment. 9. The landscape modeling device according to claim 8, wherein the display means displays the three-dimensional shape selected by the height adjusting means and another three-dimensional shape in different styles. 10. The display unit displays the object stored in the image storage unit and the three-dimensional shape of the object formed by the three-dimensional shape forming unit in different formats. The landscape modeling device according to any one of claims 7 to 9. 11. A map storage means for storing a map corresponding to an object, a shape storage means for storing outline information of a horizontal cross section of the object, and a shape storage means for storing the outline information stored in the shape storage means. Three-dimensional shape forming means for forming a three-dimensional shape of the object; an object stored in the image storage means based on the imaging information stored in the imaging information storage means; and the three-dimensional shape forming means Is displayed in association with the three-dimensional shape of the object formed by the above, and is formed by the object on the map stored in the map storage means and the outline information stored in the shape storage means. Display means for displaying the outline in association with;
A landscape modeling device comprising a correction unit that corrects contour information of the object in response to an input based on a display result of the display unit. 12. The method according to claim 11, wherein the correction unit corrects the outline information of the outline selected based on the display result, based on the input information input after confirming the display result.
A landscape modeling device as described. 13. The landscape modeling apparatus according to claim 12, wherein the display means displays the outline selected by the correction means and another outline in different modes. 14. The landscape modeling device according to claim 11, wherein the correction unit corrects a position of the contour. 15. The landscape modeling device according to claim 11, wherein the correction unit corrects a shape of the contour. 16. The display means displays an object on the map stored in the map storage means and a contour formed by the contour information stored in the shape storage means in different formats. The landscape modeling device according to any one of claims 11 to 15. 17. The landscape modeling apparatus according to claim 11, wherein the correction unit adds a new contour. 18. The landscape modeling apparatus according to claim 17, wherein the three-dimensional shape forming means forms a three-dimensional shape including contour information on a new contour.