JP2000076453A - Three-dimensional data preparing method and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional data preparing device capable of precisely preparing three-dimensional data from a photograph or a perspective with a simple constitution and by a simple processing. SOLUTION: The device is provided with a digital image inputting device 10 and a scanner 11, which fetch a still picture in a space of unknown spatial structure, a means catching the fetched still image as an image projecting the space to identify a view point to the still image based on the positional relation of a straight line in the still picture, a means preparing three-dimensional data of spatial structure in the still image based on information on the still image and the view point and an image processing computer 20 outputting prepared three-dimensional data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The present invention relates to a building, a building,
3D data creation method and apparatus for reading a photograph or a perspective drawn of a space such as a room or a space where an object whose shape is to be measured is placed, and processing image information obtained therefrom to create 3D data of the space. It is about.

[0002]

2. Description of the Related Art Conventionally, there has been a three-dimensional data creating apparatus such as a technique of drawing a perspective from three-dimensional data in a given space or a system for creating a virtual reality space in which the space is viewed while moving by computer graphics. But,
Conversely, there is no system or apparatus that derives three-dimensional data of the target space from an arbitrary space photograph or perspective and creates a layout, plan, elevation, and the like.

[0003]

However, the conventional three-dimensional data creating apparatus derives three-dimensional data of a target space from an arbitrary space photograph or a perspective, and arranges, plans, elevations, etc. Does not create
It is not possible to obtain an image from a single photo when the target space is viewed from a different viewpoint, and also want to see a case where the furniture etc. arranged there is replaced with another object However, there was a problem that a high-quality image could not be obtained.

In conventional computer graphics, enormous work is required to input three-dimensional data, and there is a limit in simplifying the input work using image information such as photographs and perspectives. There is a problem that three-dimensional data cannot be created without a drawing. In addition, since a huge number of polygons are set and calculated in order to enhance the reality, it is necessary to use an extremely high-speed computer system to create a virtual reality space that moves in real time. Although you can see the approximate positional relationship and size of the layout map, plan view,
As an elevation, there is a problem that their positional relationship and size cannot be accurately grasped.

[0005] Further, in the existing three-dimensional measuring device for industrial use, it is difficult to carry the device due to the nature of the device requiring high accuracy, and the cost is also high because the pattern projector itself requires high accuracy. In the method of reading six points on a screen and solving a system of 12-element simultaneous equations to obtain the position of a camera in order to unify the coordinates, the sampling error increases when the number of pixels of the camera is only one hundred and several hundred thousand. There was a problem that would. In addition, even in the case of obtaining topographical data from aerial photographs, similarly complicated calculations result in the disadvantage that point reading errors are large, and there is a problem in measurement accuracy.

SUMMARY OF THE INVENTION The present invention has been made to solve such a problem, and a three-dimensional data creating apparatus capable of creating three-dimensional data from a photograph or a perspective with a simple configuration and a simple process with high accuracy. The purpose is to provide.

[0007]

According to a first aspect of the present invention, there is provided a method for creating three-dimensional data in which a still image in a space whose spatial structure is unknown is captured, and a still image is created based on a positional relationship between straight lines in the captured still image. Is calculated, and three-dimensional data of the spatial structure in the still image is created based on the information on the still image and the viewpoint. The three-dimensional data creation device according to the second invention is a unit that captures a still image in a space whose spatial structure is unknown, captures the captured still image as a projected image in space, and detects the position of a straight line in the still image. Means for calculating a viewpoint for a still image based on the relationship, means for creating three-dimensional data of the spatial structure in the still image based on information on the still image and the viewpoint, and means for outputting the created three-dimensional data. It is provided.

[0008] A three-dimensional data creating apparatus according to a third aspect of the present invention comprises:
The three-dimensional data obtained from the image information of the still images with different viewpoints in the same target space were collated, and the relative positional relationship between the parts arranged in the same target space was calculated and displayed in the target space. It is provided with means for obtaining three-dimensional data of all parts. The three-dimensional data creation device according to the fourth invention classifies the still image into a plurality of parts for each part in the still image, and the three-dimensional data of each part is a plurality of parts-specific three-dimensional data models prepared in advance. It is provided with a means for approximating the closest one from among the three-dimensional data and adding the three-dimensional data to the three-dimensional data created from the still image.

A three-dimensional data creating apparatus according to a fifth aspect of the present invention
It is provided with means for transferring color information from a still image to three-dimensional data and creating three-dimensional data including color information of the still image. The three-dimensional data creation device according to the sixth invention is
A means is provided for replacing the three-dimensional data of an arbitrary part with three-dimensional data of another part selected from a plurality of separately prepared samples, and creating an image when the target space in which the part is replaced is viewed from the viewpoint. Things.

[0010] A three-dimensional data creating apparatus according to a seventh aspect of the present invention comprises:
It is provided with means for synthesizing a plan view, a layout view, an elevation view, and a still image when the viewpoint and the viewing direction are arbitrarily changed from the stereoscopic data created from the still image. The three-dimensional data creating apparatus according to the eighth invention sequentially displays a synthesized viewpoint and a still image when the line-of-sight direction is arbitrarily changed according to the viewpoint that moves continuously, and the line-of-sight direction. It is provided with means for creating a moving image as if viewed while moving.

According to a ninth aspect of the present invention, there is provided a three-dimensional data creating apparatus.
Simultaneously measure multiple cameras and multiple points on the surface of the object to be measured from cameras located at multiple viewpoints with different positions, and measure the vanishing point by placing a reference frame in the same space Means for obtaining the viewpoint of the camera, superimposing and matching the basic composition derived from the images captured by the plurality of cameras, identifying the positions of the many points, and creating three-dimensional data of the object to be captured; It is provided with.

A three-dimensional data creating apparatus according to a tenth aspect of the present invention simultaneously measures a plurality of cameras and a plurality of points on the surface of an object to be measured from a plurality of viewpoints having different positions and has the same function. 2, 3 whose space coordinates are known
Alternatively, the viewpoint of the camera is obtained from the four points, the basic composition derived from the images captured by the plurality of cameras is overlapped and collated, the positions of the many points are specified, and the three-dimensional data of the object to be captured is obtained. It has a means for creating.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the principle of creating stereoscopic data and the outline of data processing according to the present invention will be described. The present invention is characterized in that the spatial structure of the object to be photographed is unknown. Therefore, if there are a plurality of images, the spatial structure is specified even from a photograph that has already been photographed, and the entire target space is identified. Alternatively, various applications can be opened by obtaining some three-dimensional data and using the data.

If three-dimensional data of the space represented by the image information such as a photograph, a perspective, and a digital photograph can be obtained, a layout diagram, a plan view, an elevation view, a top view, and the like can be created. It is possible to create digital photographs and perspectives when the viewpoint and viewing direction are changed.

Normally, in an indoor space, a building, or a building, its walls, ceiling, floor, and the like are arranged vertically or horizontally with respect to the ground plane, and they are also in a vertical or horizontal positional relationship with each other. It appears in the image as a straight line of a book. That is, the angle at which they intersect is known, and the boundary is visible on the image. The reason why a person can know the depth and the positional relationship of a space from a photograph is because it is inferred from the relationship between the known angle and the boundary line. Similarly, by estimating the crossing angle of a part such as a floor, a wall, or a ceiling (usually 90 degrees) and calculating based on a two-point perspective view or a one-point perspective view or a three-point perspective view, each part and viewpoint can be obtained. Can know each other's positional relationship.

Furthermore, if there is a single line whose position and length are known in the image information shown in a photograph or a perspective, the length and size of all parts can be known. In addition, for other parts such as furniture and fences, some information such as the height, the distance from the wall surface, and the angle with the wall surface can usually be estimated and complemented, and the positional relationship based on the estimated and complemented information is used. Can be estimated fairly accurately.
As a result, it is possible to estimate the mutual positional relationship of all the objects shown in the photograph and obtain the three-dimensional data.

[0017] Further, by comparing information of a photograph taken of a same object from a plurality of different points and a perspective drawn, more three-dimensional data can be obtained, and the positional relationship of each part can be specified without inputting estimation information. it can. In addition, it is not possible to extract three-dimensional data such as a sofa or a curtain from a two-dimensional information such as a photograph, which is not formed by a simple plane, but to select a similar one from a number of three-dimensional data models prepared in advance. This makes it possible to obtain three-dimensional data similar to a complicated shape.

Further, if the original image information is divided for each part from the color information of each pixel, each part is replaced with another object and displayed on the image, so that, for example, an indoor sofa can be separated. Can be seen as a composite photo. In this case, as the three-dimensional data of the image data to be replaced, approximate data selected from the preceding three-dimensional data sample is used, and for the color and texture of the material, material sample data prepared separately therefrom is used.

It is to be noted that the material sample data may be both planar data having a color and a pattern and data representing a material feeling by a large number of three-dimensional polygon three-dimensional data.

Further, when the image information is divided for each part as described above, by utilizing the change of the color data for each pixel, the place where the rapid change occurs is regarded as the boundary line of the image of the different part. , The image can be divided. It is possible to arbitrarily determine how much change is to be taken as the boundary of the image of an object with a different part, but even in the same part, the shaded part and the part illuminated have different color data values. On the other hand, when objects of very similar colors appear to overlap, it may be difficult to distinguish the difference.

In such a case, even a small change in the color data is classified into different image parts once, and the image parts regarded as the same part according to the criterion when the user looks at a photograph or the like on a daily basis. It is necessary to input the fact that they are the same part and to make it into data.
Thus, image data in which the same part is composed of a plurality of surfaces, such as a sofa or a cupboard, can be converted into data that all the parts are the same while separately capturing the image data of each surface. .

Further, the direction and the position of the sunlight can be derived from the length and angle of the shadow. When the position of the illumination is changed or the position of the sun is changed based on this information, Can be combined. Then, based on the obtained three-dimensional data, it is possible to create an image when the viewpoint and the viewing direction are changed, and to create an image when the viewpoint and the viewing direction are slightly changed. By continuously displaying, a moving image can be created.

The calculation of polygon three-dimensional data is used to create an image in which a certain part is changed to another object, and the part using the original image information is, of course, a simple plane or curved surface converted to a three-dimensional object. Calculate as data. Therefore, if the viewpoint moves to a place far away from the viewpoint, it is expected that the actual appearance will be different. However, if there are a plurality of original image data, the image data at the position closest to the viewpoint is used. Thus, a composite image can be created with a relatively small amount of calculation.

Since the calculation of the polygon three-dimensional data is small, even if an image is viewed from many viewpoints, the load on a computer is relatively small, and a moving image can be generated in real time. Further, if the image information of each part is based on an actual photograph or the like, it is possible to faithfully express the material feeling.

In an image of an indoor space, the viewpoint of a person is usually located between the ceiling and the floor, so it is considered appropriate to create an image based on a two-point perspective projection. In addition, in the case of outdoor space, the viewpoint of a person is between the surface of the earth, a building, and a structure. In the three-point perspective projection, an object located below the eye level is expressed larger than the actual appearance. Tends to be distorted. Conversely, in the two-point perspective projection, an object located at a position considerably higher than the eye level is greatly expressed, and the image tends to be distorted. Therefore, when creating an image of an outdoor space, the more appropriate one is selected from the height of the building and the distance from the viewpoint.

In the case where the same point is photographed from a plurality of viewpoints as described above, if the basic composition derived from those images can be overlapped and the position of the point can be identified and specified, the number of such points can be set. Thus, the shape of the object to be photographed can be obtained as accurate three-dimensional data.

If such a system is a simple device and the measurement error can be reduced with a small amount of calculation, for example, two cameras can be arbitrarily placed and only a low-power laser beam should be applied, and the system can be carried easily. Then, the shape of a monument or the like that is not portable can be obtained as three-dimensional data, and can be reproduced industrially. Furthermore, if the shape of the target object can be obtained as three-dimensional data with high accuracy, industrial production can be performed using a manufacturing apparatus such as a mold that has already been put into practical use. In addition, if only a small number of reference points are read from the image and multiple images are compared, if the shape of the ground surface can be converted into three-dimensional data from the aerial photograph,
It can also be applied as a method to create ground photographs and terrain models.

Next, a specific process of creating stereoscopic data according to this embodiment will be described. FIG. 1 is a block diagram illustrating a configuration of a three-dimensional data creation device according to an embodiment of the present invention. In the figure, 10 is a digital input device, 1
1 is a scanner, 12 is a camera, 13 is a pattern projector, 20 is an image processing computer, 21 is a three-dimensional data model for each part, 22 is product sample data, 23 is a mouse / keyboard, 30 is a monitor, and 31 is a printer / plotter. . First, the outline of the three-dimensional data creation processing of this embodiment will be described. FIG. 2 is a flowchart showing an outline of the three-dimensional data creation processing according to this embodiment.

First, a photograph or the like is taken of an indoor or outdoor room where stereoscopic data is created (S100), and a perspective graphic method such as a one-point perspective method, a two-point perspective method, or a three-point perspective method is selected (S101). Then, the reference frame is photographed (S102),
Based on the information in S101 and S102, the basic composition,
A viewpoint is determined (S103), pattern light projection and photographing are performed (S104), and three-dimensional data is set for each part based on the information in S103 and S104 (S105). And the position by multiple images,
The height difference is compared (S106), and each part is assigned to the basic composition (S107).

Then, color data is assigned to the three-dimensional data (S108), and the three-dimensional data is corrected (S1).
09), and then determine whether or not to continue (S11).
0), if it is determined in S110 not to continue, the process ends (S110).
111), if it is determined to continue in S110, exchange with different sample data of each part and change of the viewpoint, viewing direction, and focus are performed (S112, S113). Then, a composite image of the layout drawing, the plan view, and the elevation view is created (S114),
Thereafter, the process returns to S110 or a moving image is created (S1).
15). S102, S104, S1 in FIG.
The processing in step 06 is a processing necessary when a three-dimensional measurement is performed to create a three-dimensional model, and is not necessary when a spatial structure is converted into three-dimensional data from a photograph.

Next, details of each processing of the three-dimensional data creation processing of this embodiment will be described. 3 to 14
FIG. 5 is an explanatory diagram for describing details of each process of the three-dimensional data creation process according to the embodiment. First, a photograph taken indoors or outdoors or a perspective drawn indoors or outdoors is read as digital image data by the scanner 11 or the like, or image data of a photograph taken by the digital camera 12 is read by the input device 10. When the coordinates of each pixel of the read image are (Xu, Yu), the color data is CLR (Xu, Yu).

In the first stage, the viewpoint, the position of the image and the spatial configuration such as the floor, the wall, the ceiling, and the ground surface are determined by the image processing computer 20 from the read image (this is called a basic composition). Then, the positions of other parts such as furniture and curtains are dropped, and three-dimensional data is given to each part by selecting an approximate one from a three-dimensional data model, thereby obtaining three-dimensional data of the entire space represented in the image.

In the following, a case where an image photographed indoors is processed in accordance with a two-point perspective projection will be described. In the case of an outdoor, a one-point perspective projection and a three-point perspective projection will be described later. First, from the image data, as shown in FIG. 3, two vanishing points Q and R located at positions horizontal to the viewpoint with respect to the floor, and arbitrary points J and O located on the boundary between the wall and the ceiling. , On the boundary between floor and wall,
K point and P, which are vertically descended from J point and O point
The points, the point M at the position where the two walls and the ceiling intersect, and the position of the point N where the two walls and the floor intersect are obtained.

The coordinates of each of the points J, K, M, N, O, and P are represented by (Xj, Yj), (Xk, Yk), (Xm, Ym), (X
n, Yn), (Xo, Yo), (Xp, Yp). Also, Q
The points and the R point are (Xq, Yq) and (Xr, Yr). Note that FIG.
, The length (height) of point J-K is Hk, the length (height) of point M-N is Hn, and the length (height) of point O-P is Hp.

Here, there are a plurality of methods for reading the coordinates of each point from the image, but the most rudimentary method is a method of obtaining a cursor by moving the cursor to the corresponding point on the monitor. Alternatively, it may be obtained as an intersection of two lines. In the case of reading a line, sampling errors can be reduced by extracting pixels constituting the line and obtaining the pixels by the least square method. If you find a point as the intersection of such lines, you can read it quite accurately. Also, this may be automatically extracted by image processing.

If six points from the point J to the point P can be taken out, the two vanishing points Q (Xq, Yq) and R (Xr, Yr) can be calculated from the following equations. Xq = [(Xo-Xm) * (Yp * Xn-Yn * Xp)-(Xp-Xn) * (Yo * Xm-Ym * X
o)] / [(Xo-Xm) * (Yp-Yn)-(Xp-Xn) * (Yo-Ym)] Yq = [(Yo-Ym) * (Yp * Xn-Yn * Xp)-(Yp -Yn) * (Yo * Xm-Ym * X
o)] / [(Xo-Xm) * (Yp-Yn)-(Xp-Xn) * (Yo-Ym)] Xr = [(Xj-Xm) * (Yk * Xn-Yn * Xk)-(Xk -Xn) * (Yj * Xm-Ym * X
j)] / [(Xj-Xm) * (Yk-Yn)-(Xk-Xn) * (Yj-Ym)] Yr = [(Yj-Ym) * (Yk * Xn-Yn * Xk)-(Yk -Yn) * (Yj * Xm-Ym * X
j)] / [(Xj-Xm) * (Yk-Yn)-(Xk-Xn) * (Yj-Ym)] Since the above equation is a general equation for finding the intersection of two lines,
In the following description, the explanation of such calculation formula is omitted.

Here, since the Q point and the R point are at horizontal positions, Yq and Yr and the Y coordinate Y0 at the center of the image have the same value. From this, if the positions of the six points from the point J to the point P are known, when the values of Yq and Yr are different, it is estimated that the read photograph or the like is inclined, so that Yq = Yr. Rotate the entire image around the center of the image to correct it horizontally. Also, the J point and the K point, the M point and the N point, and the O point and the P point are in a vertical positional relationship, and if the values are not such, a reading error is expected. Fix it. An example of the simplest fix is Xj '= Xk' = (Xj + Xk) / 2,
Xm '= Xn' = (Xm + Xn) / 2, Xo '= Xp' = (Xo + Xp) / 2, Ym '= Ym, Y
n '= Yn, and Yj', Yk ', Yo', Yp 'are each RM
It is a method of finding on a line, an RN line, a QM line, and a QN line.

There are a plurality of methods for finding the positions of the six points from the vanishing points Q, R, and J to P, in addition to the method of directly finding them. I do. Normally, some of the six points are hidden behind furniture or the like, and it is not possible to directly obtain all the points. However, the relationship between 6 points and 2 vanishing points is related by the above-described relational expression. If a photograph is input horizontally and there is no data reading error, 6 points can be obtained even if some data cannot be obtained. It is possible to calculate all points. There are many methods for calculating the six points, and a representative method is shown here.

The first alternative method will be described with reference to FIG. Yq and Yr
Have the same value, so even if any one of the points J, K, O, and P is missing, the coordinates of the vanishing points Q and R can be calculated from five points including the points M and N. . As a result 1
The coordinates of the point can be calculated. In addition, J point and K point,
Alternatively, the point O and the point P do not necessarily have to be at a vertical position. For example, the point K "may be used instead of the point K.

A second alternative method will be described. If the M and N points are hidden, the six points J, K ", P, J ', K', and P 'may be read to calculate the M and N points.
It is desired that the point and the K point are on the same vertical line with respect to the floor, but Xk = Xj on the line connecting the K 'point and the K "point even if the K" point does not lie vertically below the J point. The problem can be solved by obtaining the Y coordinate of the point and setting it as the K point.

A third alternative method will be described. When K "and N points are hidden, five points can be derived also by inputting four points of J, M, O, and P, the ceiling height, and the height of the viewpoint taken. In this case, Yr = Yq = Yp + If (Yo-Yp) × (height of viewpoint) / (ceiling height), points K and N can be derived, and similarly, if the ceiling height is known, the floor (ceiling) whose height is known ) And a line parallel to the wall surface, for example, a line of a sideboard or a window frame, the position of the floor surface can be specified, so that five points can be derived.

Incidentally, Yr and Yq usually have the same value as the Y coordinate Y0 of the image center. When this value is different, it is considered that the line of sight of the image is not horizontal to the floor, and the angle φ is obtained by the following equation. sin φ = (Y0−Yq) / (Yt−Ys) Here, when the line of sight of the image is not horizontal with respect to the floor, the image is regarded as a three-point perspective projection, and is converted into an image of a horizontal line of sight according to a method described later. To find the basic figure.

A in the basic composition (FIG. 3) of the layout diagram
Coordinates of point, point B, point C and viewpoint S (Xa, Ya), (Xb, Yb)
, (Xc, Yc), (Xs, Ys) are calculated by the following formulas. The angle at which the walls intersect is θ ゜. Usually θ = 90
゜, and a layout diagram may be assumed on the premise. here
Yt may be any value, but it is the larger of Yj and Yo in order to make the drawing compact. Yt = MAX (Yj, Y
o)

Xs usually uses the central value of the entire image.
This is based on the assumption that the input image is the entire photograph taken. If the image is only a part of a photograph, it is necessary to obtain data from the image and accurately determine the position of the viewpoint without using the coordinates of the center point of the image.
In this case, it is necessary to know the distance between the point A and the point B and the distance between the point B and the point C, or to know two sides of a horizontal plane of furniture or the like, which are photographed there. For example, when the distance Lab between the points A and B and the distance Lbc between the points B and C are known and the two sides are orthogonal, Xs is calculated by the following equation.

G1 = Xk / Hk-Xn / Hn, G2 = 1 / Hk-1 / Hn, G3 = Xn / Hn
If -Xp / Hp and G4 = 1 / Hn-1 / Hp, the X coordinate Xs of the viewpoint can be derived from the following equation. Xs = ｛(G1 * Lbc) ² − (G3 * Lab) ² + Xq * Xr * [(G4 * Lab) ² − (G
2 * Lbc) ² ]｝ / ｛(G1 * G2 * Lbc ² −G3 * G4 * Lab ² ) * 2 + (Xq +
Xr) * [(G4 * Lab) ² − (G2 * Lbc) ² ] Ｙ The Y coordinate Ys of the viewpoint is given by the following equation. Ys = Yt-√ [-(Xs-X
q) * (Xs-Xr)]

If the angle at which the walls intersect is not 90 °,
The angle must be entered and specified. Assuming that the angle at which the walls intersect is θ ° and β = cotθ, Ys = Yt-｛β * (Xr-Xq) + √ ([β * (Xq-Xr)] ² -4 * (Xs-X
q) * (Xs-Xr))｝ / 2

Obtaining image information of the facade of a building from a plurality of photographs of a street taken while walking, and creating a virtual reality space in which the street can be freely walked as three-dimensional data of the street. Can also.

In an image obtained by photographing a cityscape, adjacent buildings are distant from each other, there is a difference in elevation of the ground, and although the composition is based on the two vanishing point perspective method, only one vanishing point is calculated due to the restriction of data obtained from the image. It seems that there are many cases that cannot be done. In this case, as shown in the example shown in FIG. 11, a virtual line that intersects at a fixed angle with the line on which the one vanishing point is calculated can be dealt with on the image. The angle at which these two lines intersect can usually be determined from a map. The virtual line is selected so that the intersection angle of the line of the basic composition extracted in the image can be known. In the example shown in FIG.
-Find a vanishing point R from the intersection of the P 'line, find a Q point on the extension of the JJ' line, and X = Xs such that the angle between the viewpoint S and the line connecting the Q point and the R point becomes θ ゜. By obtaining the upper viewpoint S, a basic composition of the space can be obtained.

If the viewpoint can be determined,
Even if the cityscape is winding, the degree of bending of the building facade can be calculated as long as each of the buildings is formed of a plane. Although the basic composition of the space targeted by the image can be understood from the map, in addition to the fact that the position with respect to the viewpoint is known, information on each part in the image can be easily given as three-dimensional data.

Any two points E1 (Xe1, Ye1) on the image whose position on the layout diagram shown in FIG. 3 and the distance between the two points are known.
), E2 (Xe2, Ye2), the distance between point A and point B or the distance between point B and point C is calculated as 1/10, 1/20, 1/50, 1 /.
Α is determined so as to be an arbitrary scale desired by the user, such as 100 or 1/200. For example, when the distance between the points A and B is calculated, α is derived by the following equation. α = [(distance between point A and point B) * (scale)] / [√ ｛[(X
k-Xs) / Hk- (Xn-Xs) / Hn] ^2- (Xs-Xq) * (Xs-Xr) * [1 / Hk-1 / Hn]
² ｝] However, Hk = Yj-Yk, Hn = Ym-Yn, Hp = Yo-Yp. As the value of the scale, an appropriate numerical value is given according to an output device such as a monitor and a printer, and the arrangement level of the pixels on the screen.

When the distance between the points B and C is calculated, k is
Substitute p for α and derive α from the same formula. When the ceiling height is known, α = (ceiling height) * (scale) Once α is determined, the coordinates of each point of A, B, and C can be obtained by the following equations. Xa = α * (Xk-Xs) / (Yj-Yk) + Xs, Ya = α * √ [-(Xs-Xq) *
(Xs-Xr) / (Yj-Yk) + Ys] Xb = α * (Xn-Xs) / (Ym-Yn) + Xs, Yb = α * √ [-(Xs-Xq) *
(Xs-Xr) / (Ym-Yn) + Ys] Xc = α * (Xp-Xs) / (Yo-Yp) + Xs, Yc = α * √ [-(Xs-Xq) *
(Xs-Xr) / (Yo-Yp) + Ys] In addition, the height of each point is set above the point A, with the height of the viewpoint being 0.
That is, the height Zaj of the point J above the point A is Zaj = (Yj-Yq) * (Ya-Ys) /
(Yt-Ys), and the height Zak of point K above point A
Is determined by Zak = (Yk-Yq) * (Ya-Ys) / (Yt-Ys). Other points are similarly obtained.

When the line connecting the points J and M and the line connecting the points K and N are parallel, the image has a one-point perspective composition. If the two wall surfaces are orthogonal to each other, in this case, the X coordinate of the intersection of the line connecting the points O and M and the line connecting the points P and N is the X coordinate Xs of the viewpoint. If the distance between the points A and B and the distance between the points C and B are known, Ys is calculated by the following equation.
However, Yt is an arbitrary value, and Xs is a value at the center of the image. Ys = Yt + (distance between points B and C) * (Xk-Xn) * (Xp-Xs) / [(distance between points A and B) * (Xp-Xn)] If the connecting line and the line connecting the points B and C do not cross each other but intersect at θ ゜, Xs is the value at the center of the image, and f = (distance between points A and B) / (points B and C Ys = Yt + (Xp-Xs) * sin θ / (f * (Xp-Xn) / (Xk-Xn) -cos
θ]. Yb-Ys = (Yt-Ys) * (Xb-Xs) / (Xm-Xs), Xc-Xs = Hm * (Xb-Xs) *
(Xp-Xs) / [Hp * (Xm-Xs)], Yc-Ys = (Yt-Ys) * (Xc-Xs) / (Xp-X
s), Ya = Yb, Xa-Xs = (Xk-Xs) * (Yb-Ys) / (Yt-Ys).

Next, the case of the three-point perspective projection will be described. When taking a picture while looking up at a building, the composition of the photograph is a composition narrowed upward as shown in FIG. In such a case, it is appropriate to determine the arrangement by three-point perspective. First, similarly to the two-point perspective projection, two vanishing points Q (Xq, Yq),
R (Xr, Yr) is obtained, and the entire image is rotated around the center point of the image so that Yq = Yr. If the data input from the image is appropriate, the extension of the line connecting the points J and K, the extension of the line connecting the points M and N, and the extension of the line connecting the points O and P should all converge to one point. It is.

However, in practice, it is expected that there will be slight deviations from the input error at each point. Therefore, the average value (Xu, Yu) of the coordinates of the intersection of each two lines is obtained, and this X coordinate is calculated as the coordinate Xs of the viewpoint.
(= Xu), and the Y coordinate Ys of the viewpoint is obtained in the same manner as in the two-point perspective projection. It should be noted that this Xs may be the median value X0 of the image, and a person who seems to have a small input error is selected. here,
If the tilt of the screen, that is, the elevation angle at the time of shooting is φ, sin φ
= (Yt-Ys) / (Yu-Yq). Note that this φ is sinφ = (Y0-Y
q) It may be obtained as / (Yt-Ys), and the person who thinks that the input error is small is selected.

Then, the three-point perspective image is converted into a two-point perspective image by the following processing. Color data CLR (Xw, Yw) of each point (Xw, Yw) of the newly created two-point perspective projection
Is the color data CLR (Xv, Yv) of the coordinates (Xv, Yv) on the screen of the three-point perspective projection based on the following formula. Xw = Xs + (Yt-Ys) * (Xv-Xs) / [(Yt-Ys)-(Yv-Yq) * sinφ] Yw = Yq + (Yt-Ys) * (Yv-Yq) * cos φ / [( Yt-Ys)-(Yv-Yq) * sin
φ] Hereinafter, three-dimensional data of each part is obtained in the same manner as in the case of the two-point perspective projection method.

In the second stage, the spatial data of each part of the read image is dropped on the basic composition of the space obtained earlier. The read image information is divided for each part. The dividing method may be arbitrary, and commercially available software may be used in addition to the following method. The image data is decomposed into a plurality of parts as shown in FIG. 7 using the change in the color data. In this case, each part is a wall AB, BC, ceiling,
It is desirable to disassemble each part such as a floor, a curtain, furniture, and an accessory. However, there are cases in which a change in color information such as a shadow or a small outline of another part dropped on each part cannot be captured. Therefore, the standard of color data change for disassembly and recognition of parts is set to an appropriate value, and when the shaded part and the other part are recognized as different parts, it is determined. Each part is allocated to each part by inputting the same part with a keyboard, a mouse, or the like. As a result, each part is associated with one or more image parts.

Since no three-dimensional information such as furniture is given, the user selects similar ones from the three-dimensional data models for each part, such as furniture, curtains and accessories, which are prepared separately for each part. The shape of each part is determined by associating the selected part-specific three-dimensional data model with each part. However, the image is displayed as it is on the screen as it is, unless it is replaced with another object. There are two types of region-specific three-dimensional data models: a case where a simple plane is combined and a case where the three-dimensional data model is composed of an aggregate of minute polygon three-dimensional data.

The three-dimensional data model for each part represents the relative positional relationship of the three-dimensional data, and does not determine the absolute size and the proportions of the upper, lower, left and right depths. Accordingly, when a site-specific three-dimensional data model of each site is selected, the size, proportions of the top, bottom, left, and right are determined from the data on the layout drawing and the elevation view, and the image is displayed as in the example shown in FIG. It is displayed according to the space of. In addition, temporary data such as the height is attached to the three-dimensional data of each part as an initial value.

Further, if the three-dimensional data model is designed so as to be able to change not only the entire size and the proportion but also a part of the size and the proportion, the versatility can be increased. . For example, in the case of a sofa, it is preferable that only the central portion of the backrest can be made higher than the surroundings, and only the thickness of the armrest portion can be changed. The window frame on the wall surface can be used in the obtained three-dimensional data space so that operations such as pulling out a part of the wall in a rectangular parallelepiped shape or retracting it to the back in accordance with the image can be performed. Can be easily created. Part of the 3D data model is
Those stored in existing three-dimensional CG, CAD software or the like may be used.

The walls, floor, and ceiling are arranged according to the basic composition shown in FIG. At the initial stage, each part is arranged at an appropriate position on the plan view. Since the inclination of the room on the plan view is known, the three-dimensional polygon three-dimensional data of each part is positioned along the inclination, and the proportions of the left, right, up, and down are determined according to the shape of the image. Furthermore, the distance from the viewpoint is provisionally determined according to provisional data such as the height of each part.

When a straight line parallel to the floor or the ground surface can be extracted from an image of a part, the orientation of the arrangement of the part in the space can be determined. That is, the direction of the extracted straight line in the space is parallel to a line connecting the viewpoint and the intersection of the extended line of the straight line on the image and the line represented by Y = Yq. If the extension does not intersect with Y = Yq,
Its orientation is parallel to the image. By utilizing this fact, it can be arranged on the basic composition. Therefore, if the actual length of the extracted straight line, for example, the width of the sofa or the depth of the cupboard, is known, the position of that part can be specified from only one image. By utilizing this, each part can be arranged on the basic composition.

Further, parts such as curtains, furniture, small articles, etc. are shown on the layout drawing according to certain assumed conditions. For example, tall furniture, reception sets, and curtains are arranged in contact with walls, and small items are arranged in the center of tables and flower stands.
Further, the position (distance from the viewpoint) is corrected by input from the cursor and the keyboard. At that time, the size of each part is automatically corrected so as to match the original image according to the position correction.

When synthesizing images from the same viewpoint or from a relatively close viewpoint, it is not necessary to individually give stereoscopic data to each of these small objects and the like. Three-dimensional data may be given as a simple plane. In addition, the operation may be simplified by giving three-dimensional data as simple plane data to a corresponding position for each part. As described above, the coordinates of the three-dimensional polygon or the simple three-dimensional data of each part in the three-dimensional space and the coordinates of its height are determined.

The color data of an arbitrary point (Xw, Yw, Zw) of each of the three-dimensional data whose position is determined as described above is the color data of the intersection of the line connecting the point and the viewpoint and the image of the corresponding part. . That is, the color data of (Xw, Yw, Zw) is Xu = X on the image.
s + (Xw-Xs) * (Yt-Ys) / (Yw-Ys), Yu = Yq + (Zw-Zs) * (Yt-Ys) /
Color data CLR (X) of coordinates (Xu, Yu) obtained by (Yw-Ys)
u, Yu). If the line connecting the point and the viewpoint on the three-dimensional data does not intersect the image area of the part, a transparent color is given for the time being.

In the three-dimensional data, the color data is a transparent color in a portion which is not shown in the image because it cannot be seen from the viewpoint. However, by appropriately diverting the color data of the other portion of the portion, the approximation is performed. Color data. As a result, three-dimensional data of the entire target space, although approximated by the three-dimensional data model, can be obtained from the captured or drawn image of the space read as described above.

Although the wall and the like are often hidden behind furniture, the material and color of the wall are usually the same as those of other places. Color data can be given to all locations by copying the same thing. If there is a pattern on the wallpaper, a continuous pattern can be obtained by selecting the portion to be copied.

In the case of an object having a complicated shape such as a curtain,
The actual image gives stereoscopic data as a square plane as it is. In addition, a portion where the actual shape does not match the rectangle is given a transparent color as color data. When three-dimensional data is obtained by newly selecting from material samples, color data of the material sample is given as color data of a three-dimensional data model of the curtain to obtain three-dimensional data. In the case of furniture, etc., the color data of the original image is given to the approximated three-dimensional data model, and in the places that were not visible behind other furniture, copy the color data appropriately from the place where the color data was obtained. Get it.

Various material sample screens are separately prepared, and the shape of each material sample is given approximately three-dimensional data by designating one of the above-mentioned three-dimensional data models for each part. For example, in the case of a sofa, a sample image of the cloth is given as color data, and in the case of a cupboard, the front side, which has a complicated color scheme, uses the image as it is, and the side repeatedly repeats some color data. By using this, color data is specified for the approximated three-dimensional data. When the user selects a new material and a three-dimensional model for each part associated with the new material from the material sample and replaces it with one part on the screen, the three-dimensional data of the part is replaced with the newly selected three-dimensional data. It is also possible to change the position of the selected material sample by designating it on the layout drawing.

For each three-dimensional data model, how the texture of the sample data originally incorporated as material sample data or the sample data reflecting the actual texture obtained by a scanner or the like is used as the color data for each three-dimensional data of the three-dimensional data model. Or the method is determined in advance. This method is not particularly specified. It is also possible to use existing software. One method is to set a rule to develop the polygon on a plane for each three-dimensional data model, and to add the color of the product sample data to the polygon developed on the plane. There is a method of transferring data and three-dimensionalizing each polygon having color data in a procedure reverse to the above rule.

Next, a procedure for creating an arrangement plan, a plan view, or an elevation view from three-dimensional data will be described. If the three-dimensional data of the target space can be obtained, commercially available virtual reality production software using various hidden surface elimination methods other than those described below can be used. When creating a layout drawing or plan view, take out the CLR (Xu, Yu, Zu) of the three-dimensional data (Xu, Yu, Zu) in ascending order of the Z-axis data, and read the coordinates (Xu , Yu) pixel color data CLR (Xu, Yu)
When the value of another color data is given to the same coordinates later, if the color data is not transparent color, it will be used as the new value, and the layout drawing and plan view will be created .

In this case, since the wall surface is not given a thickness, an appropriate thickness is input in advance. When an elevation is created, three-dimensional data whose coordinate system is converted by the following equation and its color data are created in a separately prepared file. Xw = Xu * cosδ−Yu * sinδ, Yw = Xu * sinδ + Yu * cosδ, Zw =
Zu CLR (Xw, Yw, Zw) = CLR (Xu, Yu, Zu) Here, in the example shown in FIG. 4, δ1 (negative value) when wall AB is the background, wall B When -C is used as the background, it is δ2 (positive value).

The CLR (Xw, Yw, Zw) is extracted in order from the data having the largest Y-axis value of the three-dimensional data, and the color data CLR (Xw, Yw) of the pixel at the coordinates (Xw, Zw) of the separately prepared file is extracted. The value is entered, and when a value of another color data is given to the same coordinates later, if the color data is not a transparent color, the elevation is created by setting the new value to the new value.

Next, for the rewritten three-dimensional data,
A procedure for creating an image when viewed from the same viewpoint, or an image when viewed from different viewpoints while moving the viewpoint is described. Assuming that the coordinates of the new viewpoint S ′ are (Xs ′, Ys ′) and the viewing direction is changed by Δθ, the coordinates (Xu, Yu) of each original three-dimensional data are changed to new coordinates (Xv, Yv) as follows. Convert. Xv = Xu * cosΔθ−Yu * sinΔθ, Yv = Xu * sinΔθ +
Yu * cosΔθ, Zv = Zu. Also, the viewpoint S 'is S "(X
s ", Ys"). Xs "= Xs' * cosΔθ- Ys' * sinΔθ, Ys" = Xs' * sinΔθ +
Ys' * cosΔθ

From the newly obtained three-dimensional data (Xv, Yv, Zv), Xw = Xs + (Xv-Xs) * (Yt-Ys) / (Yv-Ys), Yw = Yq + (Zv-Zs) *
The position (Xw, Yw) on the image is obtained as (Yt-Ys) / (Yv-Ys), and an image is created by giving the same value as the color data CLR (Xv, Yv, Zv) of the three-dimensional data. In this case, (Yt-Ys) in the newly created coordinate system and (Yt-Ys) in the original coordinate system have the same value. By changing this value little by little, the same effect as changing the focal length of the camera can be obtained. When creating an image, color data is started to be copied from the back part, and if different color data is captured in the same pixel later, if the color is not transparent, it is changed to the new color data, so that overlapping is performed. The invisible part is hidden, and the same image as when actually seen is created.

When the viewpoint and the line of sight are changed, FIGS.
Taking 0 as an example, d1 of the part f of the original image viewed from the viewpoint 1
The color data of the -d2 plane is copied to the d3-d4 plane of the part f of the three-dimensional data. When an image viewed from the viewpoint 2 is created, the color data of the d5-d6 surface of the portion f of the image is extracted from the d3-d4 surface, so that the texture of the original image can be faithfully obtained without calculating the polygon three-dimensional data. You can create an image that reflects on FIG. 10 shows this in an elevation view. The data conversion procedure at this time uses the above-described method.

Also, an image created by a polygon three-dimensional data calculation from a three-dimensional data model can be used if the shape of the original data model can be approximated to a plurality of simple planes.
Based on the image, the three-dimensional data model can be replaced with a simple plane model, and an image from another viewpoint can be created in a similar manner. This reduces the amount of computation,
A textured image can be approximately created.

By creating images as viewed from successive different viewpoints and displaying them sequentially, it is possible to produce an image of walking around the room from one or more photos and perspectives.

From the photographs taken from two different viewpoints in the same room, a layout drawing is created by the above-described method, and the positions of the respective parts are determined more accurately. By complementing the data that is not used, a wider range of information can be obtained. As shown in FIG. 9, the basic composition made from a plurality of photographs is overlapped, and the coordinate system of all the photographs is unified into one of the coordinate systems, whereby the mutual relation of viewpoints in the same space is determined. .

The direction in which the same part can be seen from each viewpoint is slightly different, but there is one common point from all viewpoints, and it can be specified that this is the coordinate where the part is located. The size of the site can also be specified.
In addition, if a portion of the three-dimensional data of each part where color data was not obtained from one image is also reflected in another image, color data can be obtained by the same method as described above. it can.

In other words, by comparing the positions of the same part from two stereoscopic perspective images and setting the same more strictly, it is possible to accurately know the positional relationship between many parts without performing surveying. Will be possible. The above description has been given of an example in which the building is indoors, but the same applies to a case where a building is viewed from the outside and a perspective view. In this case, the wall may be replaced with the facade of the building, the floor may be replaced with the ground surface, and the other parts may be replaced with a planting wall or a fence.

It can be estimated that there is a light source such as the sun or illumination on a line connecting the coordinates of the shadow on the three-dimensional data and the corresponding part of the part where the shadow is cast. That is, the altitude and the direction of the sun are calculated, or a position which is several tens of centimeters below the ceiling along the extension of the line is set as the illumination position. Because the difference between the values of the color data CLR (X, Y, Z) where shadows are not and where it is not is known, the change in the shadow part when the position of the light source is changed is calculated and the coordinates are calculated The CLR value of the illuminated part is given as color data for the part that was illuminated in the shadowed part, and the CLR of the shadowed part is given as the shaded part.
By giving the value of, it is also possible to produce an image when the position and the direction of the light source are changed.

A new image such as a new plan view, an elevation view, a synthesized photograph, a perspective, and a continuous moving image obtained by the above processing by the image processing computer 20 is displayed on the monitor 30. Alternatively, the data is output to a printer or a plotter 31. Since the coordinates of the pixels obtained from the three-dimensional data are not in a uniform arrangement as in the original image, an appropriate method is selected to correct and output the uniform arrangement.

In a current digital camera, the number of pixels is at most one hundred and several hundred thousand. When extracting a boundary line or a specific point, a reading error due to the limited number of pixels cannot be avoided. However, when specifying a boundary line or finding an intersection, it is possible to extract the entire pixel representing the boundary line and reduce the error by the least squares method, and also derived by the least squares method The intersection of the lines can be determined to increase the measurement accuracy, and this technique is widely used.

When a boundary line is determined from a difference in color data, the boundary line is assumed to be a straight line or a curved line, and the color data of adjacent pixels located at the boundary is determined by the least squares method from a different pixel group. The boundary line can be corrected by determining the line and regrouping the pixels in such a unit that the pixels do not exceed the determined boundary line. Also in the case of extracting the basic composition as described above, the reading error can be reduced by finding the boundary line such as a wall or a floor by the least square method and arranging it on the line.

Further, by combining three-dimensional data having color data obtained from a plurality of images, more precise three-dimensional data can be obtained. In other words, one in which sufficient color data cannot be obtained from one camera image can be supplemented by the other camera image depending on the inclination angle of the surface.

When the basic composition of images taken from a plurality of viewpoints is superimposed, and an arbitrary point is collated in the plurality of images to determine the same object and its position is specified, changes in color data in the image are automatically performed. It is considered that the error rate is high at present. However, in the case of an interactive model, what is automatically determined by calculation from the change in color data in the image is displayed on the screen, and by confirming it by a person,
Work can be streamlined while eliminating errors. On the other screen B, the position of the point at which the color data has significantly changed on one screen A is on an extension line connecting the viewpoint of the screen A and the point on the image. Therefore, if a point having similar color data and a point at which similar color data changes are obtained on the line on the screen B, a three-dimensional position can be automatically determined with considerably high accuracy. If there are a plurality of candidates, all of them may be displayed, and the position may be determined by selection by a person. This makes it possible to quickly obtain three-dimensional data of the target space.

As shown in FIG. 9, when a space derived from photographs taken from two different viewpoints is placed in the same coordinate system, one point in the photographed space is defined by the two viewpoints and the points. By taking advantage of the fact that it is at the intersection of the lines connecting the captured images, it is possible to obtain the shape of the captured object as three-dimensional data by analyzing the two images even if the object has a complicated shape. it can. In this case, if two rectangular frames with known lengths of the sides joined at a known angle are photographed in advance, the basic composition derived from the two camera images is in the same coordinate system as described above. Can be set.

Also, an illumination is arranged between the two cameras,
The basic composition is obtained by photographing two rectangular frames that are in contact with each other at a certain angle in advance, the viewpoint of the two camera images and the target space are placed in the same coordinate system, and the wavelength or width of the spot or slit light source applied to the target object is If two or more pictures with different orientations and positions are taken at the same time, and the history of the light exposure is given as a spatial code to each pixel of the picture,
The surface of the target object can be divided into fine regions, and it is possible to recognize whether or not any of the regions is the same in pixel units or a small number of pixel groups on the screens of both cameras. That is, in a plurality of times of photographing, information that an arbitrary point on the surface of the object is illuminated or not illuminated is digitized and stored, and the arbitrary point can be distinguished from other points from the data. . By collating this on a pixel-by-pixel basis for each of the two cameras, it is possible to determine that any fine area on the surface of the object is the same on images captured by the two cameras.

As an example, when the slit pattern light is applied separately in the vertical direction and the horizontal direction, the following coding is performed. Pixel n = N (x, y, a, b, c, d, e, ..., p, q, r, s, t, u, v, ... , CLR) where N is the pixel number, x and y are the coordinates of the pixel, a and later are information such as light hitting or not hitting each time of shooting in the x-axis direction, and p and thereafter are information of each hitting time in the y-axis direction. CLR is the color data of the pixel, such as information on whether or not light is applied.

As described above, if a large number of shots are taken while changing the wavelength, width, direction, position, etc. of the light, the unit of pixels having the same data on one camera image becomes small.
The object surface can be divided into fine regions. Various methods have been developed and put into practical use for this method.
In particular, the method of giving a gray code pattern by projecting slit light has a characteristic that its numerical value increases in a certain direction, and is an excellent method for easily converting a sub-pixel into a space described later. Furthermore, by using this vertically and horizontally, the space can be divided into fine regions. By irradiating a rainbow-like light pattern and reading the wavelength for each pixel, the space can be completely coded by photographing twice in the vertical and horizontal directions.

The spatial code obtained by each pixel in the image is obtained by reading the image information of the fine surface of the target object whose shape is unknown. In active imaging in which a light irradiation location is read without corresponding one-to-one, a sampling error occurs at the position corresponding to the size of a pixel. Therefore, it is necessary to estimate and determine the position on the image to be collated by estimating from the state of peripheral pixels. Although this method has already been put into practical use, this method will be described as an example using a gray code pattern using a slit light source.

If there is no sharp edge around the pixel on the screen, the periphery is assumed to be a fine plane, and a fine three-dimensional space is assumed on the axis of the numerical value of the space code in the normal direction of the screen. The plane determined by the least-squares method from the spatial code values of the gray code pattern is set in the vertical and horizontal directions, respectively, and the coordinates where the plane projection line of the line corresponding to the spatial code value of the pixel intersects are defined as the coordinates of the two-dimensional spatial code. The coordinates of a point having a value. As a result, coordinates are obtained on the images of the two cameras. A set of points closest to each other on the extension of a line connecting these points and the respective viewpoints is obtained, and an intermediate point between the two points is set as the three-dimensional coordinates of the target object represented by the pixel.

When a virtual plane is obtained by the least squares method, if the correlation coefficient is less than a certain value due to a reason such as closeness to a sharp edge, it is considered that the selected peripheral pixel is not appropriate. The area for selecting pixels is narrowed so that pixels far away from are excluded, and the calculation is performed again. If it is conceivable to approximate a minute area on the surface of the measurement target object to a plane, such a method or simply proportionally dividing the values of adjacent pixels can make the space more subpixel.

In the method according to the present invention, the number of points to be sampled to determine the position of the viewpoint is 5 to 6 points.
This is the same as the conventional method, except that the number of steps of addition, subtraction, multiplication, and division in the calculation process is small, and the error of the parameter itself can be reduced. As described above, the procedure for finding a vanishing point is nothing but finding the intersection of a line derived by the least squares method from a number of points read linearly. Originally, the procedure of extracting one point by reducing the sampling error is to calculate the intersection of the lines obtained by such a least squares method, so there is no increase in the sampling error when calculating the vanishing point. Become.

Once the position of the viewpoint is determined, the distance to the image,
The so-called focal length is also calculated from the same vanishing point and the coordinates of the viewpoint. Therefore, the enlargement of the error at the time of parameter calculation in the present invention is only due to the procedure for obtaining the viewpoint from the vanishing point. Incidentally, the parameters in the present invention are the coordinates of the viewpoint and the image plane in the coordinate system of the actual three-dimensional space where the measurement target having the basic composition is located. In determining the center point of the camera image, an error can be removed by performing a test on each camera in advance. If the error in obtaining the viewpoint is small, it is considered that a reading error at an arbitrary point in the image is acceptable even in a system using two cameras independently.

Although the measuring method using the ITV camera and the pattern light projector as described above is excellent,
There is a need for high performance, such as no mechanical mechanism in the projector itself, little distortion, etc., and there have been problems such as high cost and inability to carry easily.

In the method according to the present invention, since the measurement is performed by using two camera images, there is a feature that the performance of the projector itself does not need to be so high. As a result, even if a mechanical mechanism is used for the projector, the reading error does not become large, and the slit light source is used vertically and horizontally, or a slit light source or a slit-shaped shield is run vertically and horizontally to create a space by time-series data for each pixel. By coding, the same effect as that of a fine spot light source can be obtained with a small number of images.

Further, the influence of errors due to the distortion of the slit light source can be greatly reduced. Furthermore, since two cameras shoot simultaneously, depending on the case, the color distribution of the shooting target,
There is an advantage that it is hardly affected by the reflectance and the like. As a result, the measurement system is inexpensive, easy to carry around, and when measuring immovable measurement objects from many angles, instead of the reference cube, a frame-shaped reference body joined by a rectangle at a fixed angle around the shooting object It can be substituted by arranging and photographing.

Further, when sub-pixels are formed in the space below the pixel unit, the sampling error between the two camera images can be reduced by the condition that the lines connecting both images and the line of sight intersect in the three-dimensional space. The two viewpoints placed in the same coordinate system are (X1, Y1, Z1) and (X2, Y2, Z2), and the line connecting each viewpoint and the corresponding point on the image is (α * a
1 + X1, α * b1 + Y1, α * c1 + Z1), (β * a2 + X2, β * b2 + Y2,
a1, a2, b1, b2, c1, and c2 are set so that β * c2 + Z2). Here, α and β are variables. Originally, the two lines should intersect, but in reality, it is expected that the two lines slightly pass from each other due to measurement errors and image distortion.

Accordingly, the points α and β at which the two points on the two lines are closest to each other are determined, and the intermediate value of the two points is substituted for the intersection. If the square of the distance between two points is regarded as a function of variables α and β, α and β are obtained so that the expressions obtained by differentiating the function with α and β are both 0.

[0101] In other ^{words, d1 = a1 2 + b1 2} + c1 2, d2 = a2 2 +
b2 ² + c2 ² , d3 = a1 * a2 + b1 * b2 + c1 * c2, d4 = a1 * (X1-X2) + b1
* (Y1-Y2) + c1 * (Z1-Z2), d5 = a2 * (X1-X2) + b2 * (Y1-Y2) + c2 *
(Z1-Z2), α = (-d2 * d4 + d3 * d5) / (d1 * d2-d3 ² ), β
= (D1 * d5-d3 * d4) / (d1 * d2-d3 2) with α, β is found,
Then, a point serving as a substitute for the intersection is set as the following coordinates. [(α * a1 + β * a2 + X1 + X2) / 2, (α * b1 + β * b2 + Y1 + Y2) / 2,
(α * c1 + β * c2 + Z1 + Z2) / 2]

In the completed three-dimensional data, a plurality of adjacent data are approximated by using appropriate curves and curved surfaces, and the data is further refined by using the curved lines and curved surfaces, so that a smoother surface is obtained. Three-dimensional data is obtained. By connecting these three adjacent points with a line and setting it as a polygon, three-dimensional data of the measured object can be approximately obtained. The size of the three-dimensional polygon is arbitrarily determined.

As described above, by using two cameras, it is possible to convert an object shape into three-dimensional data using an inexpensive system that is easy to carry while reducing sampling errors. Naturally, it is also possible to obtain more detailed three-dimensional data by enlarging and measuring a part of the target object and combining the three-dimensional data of the partial part into the entire three-dimensional data. This makes it possible to easily create the above-described three-dimensional data model for various furniture, trees, and other members having complicated shapes.

Also, in the case of creating a ground surface screen or a map from an aerial photograph, there is a method of matching the coordinate system between the image and the ground surface from the coordinates of arbitrary six points whose coordinates are known as described above. However, as long as there are sampling errors in reading the six points on the image, even if the parameters are determined by solving the 12-way equation, it is not possible to avoid the error enlarged by the complicated calculation formula. In the case of aerial photography, there is no way to reduce this error because it must rely on passive measurement, and from the change in color information on the ground surface, it has to search for a point whose coordinates are known to a known point. There is a problem that an error in data reading easily occurs. In particular, when the same position is specified from two aerial photographs, the large errors overlap, and the problem is exacerbated.

Therefore, the position of the viewpoint and the direction of the line of sight,
If the position of the image can be obtained, the influence caused by the sampling error can be reduced by several steps.

When a rectangle whose length of two sides on one plane is known can be set on the ground surface, its four corners A, B,
The coordinates of each point of C and D are (Va, Za), (Vb, Zb), (Vc,
Zc) and (Vd, Zd) are the points of intersection of the extension of the line connecting point A and point B and the extension of the line connecting point C and point D, as point R (Vr, Zr), A
Find the Q point (Vq, Zq) at the intersection of the extension of the line connecting the points D and D and the extension of the line connecting the points C and B, and rotate the entire image around the center point of the image so that Zq = Zr. Let it. When the coordinates of the center of the image are (V0, Z0), the V coordinate Vs of the viewpoint is Vs
= V0. The coordinates Zs immediately below the viewpoint shown in FIG. 6 are obtained by the following equation.

Zs = Zq + (Vq−V0) * (Vr−V0) / (Zq−Z0) The angle φ that looks down on the ground from the viewpoint is sin φ = (Zq−Z0) /
√ [-(Vr-Vs) * (Vq-Vs)]. If the quadrangle set on the ground is not a rectangle but a parallelogram, the angle at which the AB line intersects with the BC line is determined from the following equation as θ ゜, β = cotθ. sinφ = ｛β * (Vr-Vq) + √ ([β * (Vq-Vr)] ² -4 * (Vs-V
q) * (Vs-Vr))｝ / [2 * (Zr-Zs)]

Aerial photograph, any point on bird's-eye view perspective (Vw,
If the coordinates of the ground surface of (Zw) are (Xw, Yw), Xw = (Vw-Vs)
* Hs / [(Zr-Zw) * cosφ], Yw = (Zw-Zs) * Hs * tanφ / (Zr-Zw)
Becomes Here, if the distance between the points A and B is known, f1 = (Va-Vs) / (Zr-Za), f2 = (Vb-Vs) / (Zr-Zb), f3 = (Za
Assuming that -Zs) / (Zr-Za) and f4 = (Zb-Zs) / (Zr-Zb), Hs is a value of the following equation. Incidentally, the coordinates of the viewpoint are (0, 0, Hs). Hs = (distance between point A and point B) * cosφ / √ ｛(f1-f2) ² + [(f3-
f4) * sin φ] ² ｝

If the rectangle set on the ground surface has a known inclination, the created three-dimensional data is converted according to the inclination to convert it into three-dimensional data horizontal to the ground surface. A rectangle or parallelogram set on the ground surface is δy around the Y axis.
゜, δx ゜ around the X axis and δh ゜ around the H axis, the coordinates (Xw,
Yw, Zw) is converted to coordinates (Xu, Yu, Zu) in a coordinate system horizontal to the ground using the following formula.

X2 = Xw * cosδy−Hw * sinδy, Y2 = Yw, H2 =
Hw * cos δy + Xw * sinδy Y3 = Yw * cosδx + H2 * sinδx, Hu = H2 * cos δx−Yw * sin
δx, Xu = X2 * cosδh + Y3 * sinδh, Yu = Y3 * cosδh−X
2 * sinδh

The above formula is suitable for calculating from a photograph of a terrain where the ground surface is relatively horizontal. When photographing the ground from a high-rise building, many reinforced concrete buildings are projected, and in many cases it is not possible to find many suitable points on the ground surface. Yes, and this method can be applied to such cases.

Even when the height difference of the ground surface is large and a parallelogram point cannot be extracted as a plane, two or three points whose positions and altitudes on the ground surface are known are known.
If the location can be identified on the photograph, the image can be converted into a ground surface screen by the following method, and the terrain can be accurately determined by comparing the ground surface screen from a plurality of viewpoints. In this case, of course, the line of sight of the image is inclined downward,
It is desirable that the left and right sides of the image at that time be kept horizontal. That is, it is desirable that the camera has a function of taking pictures while keeping both ends of the camera horizontal or dropping a horizontal line in an image. In the latter case, the calculation is performed after rotating the image horizontally about the center of the image.

If a horizontal line is contained in the front of the image, the image can be rotated horizontally. Even if you are unsure whether you were able to shoot horizontally with the left and right sides of the image,
If a plurality of lines perpendicular to the ground plane can be extracted from a building or the like in the image, the image can be corrected horizontally from there. When a line perpendicular to the ground plane is extended, if there is no reading error, the plurality of vertical lines intersect each other at one point in the image. Since this is the point of (Vs, Zs), the image is rotated so that the line is vertical by connecting to the center point of the image.

The center point of the image is set to (V0, Z0). A line running up and down from the center of the image is set to V = 0, and a projection line on the ground surface is given as X = 0 with the Y axis of the ground surface being given.
Transform the Y coordinate. 2 or 3 points on the resulting image
The coordinates of the point are A (Va, Za), B (Vb, Zb), C (Vc, Zc)
And the coordinates of the ground surface corresponding to each are A ′ (Xa, Ya,
Ha), B '(Xb, Yb, Hb), C' (Xc, Yc, Hc), and the coordinates of the viewpoint are (Xs, Ys, Hs). When the position of the viewpoint or the like is obtained from two points whose coordinates are known, the following method is used.

In FIG. 6, θ is obtained from the following equation. (Ya-Yb) * sinθ + (Ha-Hb) * cosθ = Xa * (Za-Z0) / Va-Xb * (Zb-Z
0) / Vb Further, assuming that fa = Xa / Va and fb = Xb / Vb, Zs = [(fa * Za-fb * Zb)-(Ya-Yb) / sin θ] / (fa-fb), Zq = [(fa * Za-fb * Zb)-(Ha-Hb) / cos θ] / (fa-fb), Ys = Ya-fa * (Za-Zs) * sinθ, Hs = Ha + fa * (Zq -Za) * cosθ, Xs = 0

From the nature of the mathematical expression, two solutions can be obtained for sin θ and cos θ. However, from the condition that both cos θ and sin θ are positive values, the general shooting position, altitude, and angle at the time of shooting are known. Then you can choose the appropriate one. The value calculated from the above expression includes an error due to an error in reading coordinates from the image and an error when the image is slightly inclined from the horizontal, but the distance K between the image and the viewpoint is larger than the focal length of the camera.
Is known, then (Zq-Z0) / tanθ = (Z0-Zs) * tanθ =
By utilizing the fact that K is used, the image can be delicately rotated about the center of the image, and a position where the image is estimated to be more horizontal can be obtained to reduce the error.

When obtaining from three points, Xs, Zs,
θ, Zq, Hs, and Ys are obtained by the following equations. Xs = 0, fa = Xa / Va, fb = Xb / Vb, fc = Xc / Vc, F = Ya * (fb-f
c) + Yb * (fc-fa) + Yc * (fa-fb), Zs = (Ya * (fb * Zb-fc * Zc) + Yb * (fc * Zc-fa * Za) + Yc * (fa * Za-fb
* Zb)) / F sinθ = F / [fa * Za * (fb-fc) + fb * Zb * (fc-fa) + fc * Zc * (fa-f
b)]

G = Ha * (fb-fc) + Hb * (fc-fa) + Hc * (fa-fb), and Zq = (Ha * (fb * Zb-fc * Zc) + Hb * (fc * Zc-fa * Za) + Hc * (fa * Za-fb
* Zb)) / G cosθ = G / [fa * Za * (fb-fc) + fb * Zb * (fc-fa) + fc * Zc * (fa-f
b)] Hs = Xa * (Zq-Za) * cosθ / Va + Ha, Ys = Ya-Xa * (Za-Zs) * sinθ
/ Va

However, similarly to the above method, if the reading error from the image and the image are not perfectly horizontal, it is expected that the above sin θ and cos θ will not be perfectly matched due to the error. In this case as well, the coordinates of the image and the ground surface are delicately rotated, so that the following formula is best satisfied.
The inclination of the image is corrected so that cos θ can be matched, a state closer to horizontal is searched, and a solution with less error can be obtained. If the focal length of the camera is known as in the previous case, the condition is also corrected.

Hs = Xa * (Zq-Za) * cosθ / Va + Ha = Xb * (Zq-Zb) *
cosθ / Vb + Hb = Xc * (Zq-Zc) * cosθ / Vc + Hc This is a method for determining the position of the viewpoint from two known points. By using the angle of inclination when it does not match as a variable, the number of known coordinate points is increased and the inclination is obtained approximately. If a large number of coordinates cannot be obtained in the image due to the nature of the image or the point at which a known point cannot be obtained, it is considered to be effective as a method of reducing the calculation formula to a low order and minimizing the error.

The coordinates (V, Z) of an arbitrary point on the image can be calculated by the following equations using the coordinates (X, Z) in the same coordinate system as the ground surface.
Y, H) are converted. X = V, Y = (Z-Zs) * sinθ, H = Hs- (Zq-Z) * cosθ The coordinates of the viewpoint and the coordinates of any point on the image are given in the coordinate system of the ground surface, and the viewpoint and image By converting the point on the ground surface on the extension of the line connecting the above arbitrary points as the position on the ground surface of the arbitrarily selected point, and copying the color data of the point to that point, a composite image of the ground surface (Hereinafter referred to as ground image)
Is obtained. It should be noted that this method is difficult to obtain an accurate numerical value when the ground surface is almost horizontal and an error is likely to occur. Therefore, two methods will be selected according to the condition of the ground surface.

Normally, each point on the ground surface has a difference in elevation, and unless the whole area is horizontal or the inclination is not constant, the image created in this way indicates the positional relationship of the ground surface. Not accurately reflected. As mentioned earlier, this ground image remains distorted due to the difference in elevation of the ground surface, so by comparing the ground surface images obtained from images taken from multiple viewpoints, the exact position and elevation of that point can be determined. It is necessary to obtain accurate three-dimensional data of the ground surface.

In this case, even if the basic compositions obtained from a plurality of images are superimposed, there is generally a difference in elevation on the ground surface as shown in FIG. Occurs. By utilizing this, it is possible to know the height difference between the position where the basic composition is set and the position of the object.
That is, a ground surface image is created from a plurality of aerial photographs, they are superimposed in the same coordinate system, and a line connecting the position and the viewpoint of the target object is drawn for each ground surface image. It is a coordinate that exists, and the difference between the intersection and the position of the object on the ground map can be regarded as a deviation due to a height difference.

It is to be noted that this intersection is expected to pass slightly in a three-dimensional space from the sampling error, and thus it is desirable to obtain the intersection as described above. Aerial image coordinates (Xu, Z
Assuming that the difference of the position of the object in u) on the ground surface drawing is ΔY in the Y-axis direction, the position of the reference rectangle and the height difference of the object are −ΔY * (Zt-Zu) * cotφ / (Zu-Zs). The collation work of the same point on both screens can be semi-automated by the method described above. In this case, not just the point,
Orthogonal lines on building walls can be used.

If a combination of a plurality of three points is selected from the coordinates of four or more points on the screen and the average value of the calculated viewpoint position and the like is obtained, the sampling error can be reduced. If collation is performed not only with two images but also with more aerial photographs of the same area, the reading error on each screen is reduced, more accurate three-dimensional data is obtained, and a ground surface screen can be created.

Incidentally, an image photographed horizontally on the ground surface can be similarly used as this one image. In this case, since θ = 90 °, Zq becomes infinite, this formula cannot be used, and a separate formula needs to be used. That is, the viewpoint is just above the center point of the image, and the altitude is obtained by the following equation. Hs = (Zb * Ya * Hb-Za * Yb * Ha) / (Ya * Zb-Yb * Za) However, the actual position of each point is the ground surface of the line connecting the center point of the image and the point on the image. Since it is on the projection line or an extension thereof, if this image is superimposed on another image, accurate coordinates of each point can be obtained in the same manner.

According to this method, even if the viewpoints of adjacent photographs are obtained from three different points, the ground surface screen obtained from the pair of photographs is in the same three-dimensional coordinate system.
With respect to the overlapping portion, an accurate position and altitude of an arbitrary point can be obtained.

Further, by determining the positions and elevations of many points on the ground surface, polygons are formed by connecting three adjacent points, and three-dimensional data that approximately expresses the ground surface with these polygons can be obtained. . That is, three points of the original image, A, B,
C coordinates (Xa, Ya, 0), (Xb, Yb, 0), (Xc, Yc,
0) is the corrected three points, A '(Xa', Ya ', Ha'), B '
(Xb ', Yb', Hb ') and C' (Xc ', Yc', Hc '). Further, each point in triangle ABC is transformed into a modified triangle A'B'C ', and the color data is transferred to create a correct map of the ground surface. At this time, each converted point has the altitude of the position as data together with the color data.

As shown in FIG. 13, each point (pixel) of the original image is represented by a line connecting the viewpoint and the point (pixel) and a triangle A ′.
Corresponding to the intersection of B'C ', the color data of the point is given to the coordinates of the three-dimensional data. The ground surface screen (map) is created by extracting only the X coordinate and the Y coordinate from the three-dimensional data.

[0130] The points of the corrected stereoscopic data are not equally distributed as in the original image. In creating an image, color data is assigned to each pixel evenly arranged based on the information, and a ground surface screen is created. If each polygon is sufficiently fine with respect to the height difference of the ground surface, a fairly accurate image with little distortion is created.

In the preceding paragraph, each polygon is converted so as to trace the undulation of the ground surface. Therefore, at a point with a slope, even an image of a building, a building, a tree, or the like (hereinafter, referred to as a building, etc.) is corrected according to the slope of the ground surface. Naturally, each building etc. should not be corrected along the slope of the ground surface, so cut out the original image and save it separately before the conversion procedure in the previous section, and save it on the finally created ground surface screen You will be inset.

An image of a building or the like is corrected in size in proportion to the height difference from the viewpoint in consideration of the elevation difference between the F1 point and the F2 point, and is corrected into a three-dimensional shape as shown in FIG. To obtain 3D data. Since a normal building or the like has a vertical wall surface, a roof is located immediately above a lower position of the wall surface. Using this, the shape and position of the building can be specified. When buildings and the like are arranged on the ground surface screen, only the image of the roof portion is drawn. At the same time, color data is not given to a place where the building is not seen from the viewpoint because it is not visible. However, color data of the nearest surrounding ground surface is approximately given, and this is substituted. It should be noted that this point can be supplemented by image information from a plurality of viewpoints.

As described above, various calculation methods have been described in each section, but these calculation methods can be applied to calculation methods of other terms. That is, in the procedure of arranging illumination between the two cameras described above, photographing two rectangular frames that are in contact with each other at a predetermined angle in advance, finding the basic composition, and finding the three-dimensional data of the object, When it is possible to set a rectangle whose surface has a known length of two sides on one plane, or when there are two or three points whose ground surface position and altitude are known
If you use the calculation method when you know the point and you can identify the location on the photograph, you can use 3 to 4 without using the reference frame.
The position of a viewpoint or the like can be determined based on position data obtained by measuring a point in a pinpoint manner.

In the case of reading from a photograph taken of the indoor space, three points not present on one plane whose relative positional relationship is known, for example, two points present on the boundary line between the ceiling and the wall surface whose distance between them is known are known. If you can see something like a point on the floor, you can determine the position of the viewpoint. When measuring monuments attached to buildings, etc., instead of the reference frame,
A line on the wall of the building may be used, and the length may be one side.

In this embodiment, three-dimensional data can be created from still images such as photographs and perspectives with a simple configuration and simple processing with high accuracy. You can't feel the usability just by looking at it, you can't grasp what kind of atmosphere it will be if you put it in a room, and if you go to a furniture store and see the real thing, you can feel the usability It is difficult to imagine the situation when placed in your own home, wallpaper, curtains, combination with other furniture, etc., but there is a three-dimensional data creation device of this embodiment in a furniture store. If you bring a picture of the place you are thinking of putting, you can see the image of placing it in the room at the place where you selected the furniture, you can understand and select the furniture It made.

Also, when a building is renovated, it is possible to see in advance a composite photograph of how the house will look if it is remodeled from the current exterior photo. It is also possible to create an image of walking around.

If three-dimensional data of a cityscape can be obtained, a virtual travel experience of walking around a strange overseas cityscape,
Such a game set in an actual city can be realized with high reality.

When a moving image in which the direction of the line of sight is gradually changed from a moving viewpoint is created based on a huge number of polygon three-dimensional data, an image created by calculating the huge number of polygon three-dimensional data is used. By making only one of the several images and creating the remaining images based on the method of the present invention, it is possible to reduce the load on the computer and approximately create a highly realistic moving image in real time Become.

Further, if the three-dimensional data obtained from a plurality of images is collated, it is possible not only to measure the positional relationship between a large number of transplants and buildings, but also indoors, as well as to view the layout without measuring the positional relationship. Can be done, even for objects with complex shapes,
By arranging a rectangular parallelepiped frame around the frame and simultaneously storing the frame and the object in an image, it is also possible to convert a complicated shape of the object into three-dimensional data.

An excellent feature of this embodiment is that it does not require detailed information on the angle and focal length of the camera at the time of shooting. If it is known that images taken from a plurality of viewpoints have taken the same object in the same state, stereoscopic data can be derived. Therefore, even when taking an aerial photograph or taking an object having a complicated shape outdoors, the purpose can be achieved with a simple system that is easy to carry. What is required is that the center of the image is desirably placed on the vector of the line of sight, so that there is no error in the arrangement of the film or the element for storing the image with respect to the camera lens.

In addition, three-dimensional measurement of an object having a complicated shape is also performed.
The measurement error can be reduced, and the measurement can be performed with a simple portable device.The three-dimensional data model for each part described above can be easily created. It is also possible to obtain three-dimensional data of the ground surface. If three-dimensional data can be obtained from the aerial photograph, it becomes possible to create a survey map including elevation information of the area without omitting local work.

[0142]

As described above, according to the present invention, a still image in a space whose spatial structure is unknown is fetched, and a viewpoint for the still image is determined based on the positional relationship between straight lines in the fetched still image. 3D data of the spatial structure in the still image is created based on the information of the still image and the viewpoint, so the 3D data can be created accurately from the still images such as photographs and perspectives with a simple configuration and simple processing. It has the effect that it can be done.

Further, the three-dimensional data obtained from the image information of the still images having different viewpoints in the same object space are collated, and the relative positional relationship between the parts arranged in the same object space is obtained.
Because we obtained three-dimensional data of all parts projected in the target space, we see it as a layout drawing without measuring the positional relationship of many transplants and buildings indoors as well as outdoors Even for an object having a complicated shape, a rectangular parallelepiped frame is arranged around the object, and the frame and the object are simultaneously included in the image, so that the complicated shape of the object can be converted into three-dimensional data. It has the effect of.

Further, the still image is classified into a plurality of parts for each part in the still image, and the three-dimensional data of each part is approximated to the closest one of a plurality of three-dimensional data models prepared in advance. Then, as the three-dimensional data, it was added to the three-dimensional data created from the still image,
It is possible to easily create three-dimensional data of each part in the still image.

Further, since the color information from the still image is transferred to the three-dimensional data to create three-dimensional data including the color information of the still image, the color information is added to the three-dimensional data and the three-dimensional data is displayed. This has the effect of making it easier to see. In addition, the three-dimensional data of an arbitrary part is replaced with three-dimensional data of another part selected from a plurality of separately prepared samples, and an image is created when the target space in which the part is replaced is viewed from that viewpoint. Therefore, there is an effect that three-dimensional data having various spatial structures can be easily created.

Further, since a plan view, a layout view, an elevation view, and a still image when the viewpoint and the line of sight direction are arbitrarily changed are synthesized from the three-dimensional data created from the still image, A plan view, a layout view, an elevation view, and the like can be easily created, and a stereoscopic image whose viewpoint and viewing direction are arbitrarily changed can be created with a small amount of processing. In addition, according to the continuously moving viewpoint, the viewpoint direction, the combined viewpoint and the still image when the viewpoint direction is arbitrarily changed are sequentially displayed,
Since a moving image as viewed while moving in the target space is created, there is an effect that stereoscopic data created from a still image can be viewed as a moving image.

Further, a plurality of points on the surface of the object to be measured are simultaneously measured from cameras located at a plurality of viewpoints having different positions, and a vanishing point is measured by placing a reference frame in the same space. Obtain the viewpoint of the camera, superimpose and match the basic composition derived from the images taken by the plurality of cameras,
Since the three-dimensional data of the object to be photographed is created by specifying the positions of these many points, accurate three-dimensional data can be created with a simple configuration without having to tighten the camera installation conditions. It has the effect that it can be created.

In addition, a plurality of points on the surface of the object to be measured are simultaneously measured from cameras located at a plurality of viewpoints having different positions, and the camera coordinates are calculated from two, three or four points whose coordinates in the same space are known. Since the viewpoint is obtained, the basic composition derived from the images taken by the plurality of cameras is overlapped and collated, the positions of the many points are specified, and the three-dimensional data of the object to be photographed is created. , Two or three in the same space
Alternatively, if the coordinates of four points are known, there is an effect that three-dimensional data of an object to be photographed can be created with a simple configuration.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a three-dimensional data creation device according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating an outline of a stereoscopic data creating process according to the embodiment;

FIG. 3 is an explanatory diagram for describing details of each process of a three-dimensional data creation process according to the embodiment;

FIG. 4 is an explanatory diagram for describing details of each process of a three-dimensional data creation process according to the embodiment;

FIG. 5 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 6 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 7 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 8 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 9 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 10 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 11 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 12 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 13 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

FIG. 14 is an explanatory diagram for describing details of each processing of the three-dimensional data creation processing according to the embodiment;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Digital input device 11 Scanner 12 Camera 13 Pattern projector 20 Image processing computer 21 Three-dimensional data model for each part 22 Product sample data 23 Mouse / keyboard 30 Monitor 31 Printer / plotter

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2F065 AA04 BB05 FF04 FF05 JJ03 JJ05 JJ19 JJ26 QQ31 UU05 5B050 AA09 BA06 BA09 BA11 BA13 BA15 DA02 DA04 EA05 EA24 EA27 EA29 FA06 5B057 AA01 AA13 CD02 CA01 CA08 CD08 CE08 DA07 DA17 DB02

Claims (10)

[Claims] 1. A still image in a space whose spatial structure is unknown is captured, a viewpoint for the still image is determined based on a positional relationship of a straight line in the captured still image, and based on information of the still image and the viewpoint. A method of creating three-dimensional data having a spatial structure in the still image. 2. A means for capturing a still image of a space whose spatial structure is unknown, wherein the captured still image is captured as a projected image of the space, and based on a positional relationship of straight lines in the still image, Means for calculating a viewpoint for a still image, means for creating three-dimensional data of a spatial structure in the still image based on information of the still image and the viewpoint, and means for outputting the created three-dimensional data. Characteristic three-dimensional data creation device. 3. Matching three-dimensional data obtained from image information of still images having different viewpoints in the same target space to obtain a relative positional relationship between respective parts arranged in the same target space,
3. The three-dimensional data creating apparatus according to claim 2, further comprising means for obtaining three-dimensional data of all the parts projected in the target space. 4. The still image is classified into a plurality of parts for each part in the still image, and the three-dimensional data of each part is set to a closest one from a plurality of part-specific three-dimensional data models prepared in advance. 4. The three-dimensional data creating apparatus according to claim 2, further comprising means for approximating and adding the three-dimensional data to the three-dimensional data created from the still image. 5. The apparatus according to claim 2, further comprising means for transferring color information from said still image to said three-dimensional data to create three-dimensional data including color information of said still image. 3D data creation device as described. 6. A three-dimensional data of an arbitrary part is replaced with a three-dimensional data of another part selected from a plurality of separately prepared samples, and an image obtained when the target space in which the part is replaced is viewed from the viewpoint is obtained. The three-dimensional data creation device according to claim 4 or 5, further comprising a creation unit. 7. A method for combining a three-dimensional data created from the still image with a plan view, a layout view, an elevation view, and a still image when a viewpoint and a viewing direction are arbitrarily changed. The three-dimensional data creation device according to claim 2, 3, 4, 5, or 6. 8. A still image in which the synthesized viewpoint and the line of sight are arbitrarily changed according to the viewpoint and the line of sight moving continuously, and the moving image is viewed while moving in the target space. 8. The three-dimensional data creating apparatus according to claim 7, further comprising means for creating such a moving image. 9. Simultaneously measure a plurality of cameras and a plurality of points on the surface of an object to be measured from cameras located at a plurality of viewpoints having different positions, and erase a reference frame in the same space. Measure points to obtain camera viewpoints, superimpose and compare basic compositions derived from images captured by the multiple cameras, identify the positions of many points, and obtain three-dimensional data of the object that was captured. And a means for creating a three-dimensional data. 10. Simultaneously measure a plurality of cameras and a plurality of points on a surface of an object to be measured from cameras located at a plurality of viewpoints having different positions, and obtain two, three or three coordinates whose coordinates in the same space are known. Obtain the viewpoint of the camera from four points, superimpose and compare the basic composition derived from the images taken by the plurality of cameras, specify the positions of the many points, and create the three-dimensional data of the object to be photographed A three-dimensional data creating apparatus, comprising: