JPH09119819A - Three-dimensional information reconstruction apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus by which the restriction on an object is small with a simple apparatus configuration and by which precise three- dimensional information can be reconstructed. SOLUTION: An apparatus is provided with an image input means 1 by which the image of an object 14 is input, with an image storage means 2 by which image information to be input from the image input means is stored, with an image processing means 4 by which three-dimensional information on a subject is computed by using the stored information and with a means 3 by which an 'approximate direction' to the object of the input image is obtained. The apparatus is constituted in such a way that information on the input direction of the image which is supplied and input by the means 3 to obtain the direction is supplied to the image processing means 4 so as to correspond to the information stored in the image storage means 2 corresponding to the input direction and that prescribed three-dimensional information is reconstructed by making use of the 'approximate direction', i.e., the information on the approximate direction in which a camera is directed in the photographing operation of the image when the three-dimensional information is reconstructed by using a plurality of images.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional information reconstructing apparatus for reconstructing three-dimensional information of a subject by calculating the depth dimension of the subject using an image taken by a camera or the like.

[0002]

2. Description of the Related Art In recent years, non-contact observation or measurement of a three-dimensional shape of an object has become an important technique in designing and manufacturing industrial products and computer graphics. 2. Description of the Related Art Conventionally, for example, an apparatus has been used that scans a laser beam while placing an object on a turntable while rotating it, acquires the three-dimensional information of the object using the reflected information, and calculates a three-dimensional shape.

Further, in order to simplify the apparatus structure itself and reduce the cost, the subjects are photographed from various angles, and the parts of the subjects in these images are associated with each other, whereby the depth information is calculated by the principle of triangulation. To get (ie
Stereo vision) is known. In addition,
Since it is difficult to completely automatically perform the process of accurately obtaining the corresponding points on the image, a method in which the operator inputs the corresponding relation is disclosed in Japanese Patent Laid-Open No. 6-215097. .

[0004]

However, the above-mentioned conventional techniques have problems that must be solved in terms of the structure and cost of the device, the restriction of the object and the accuracy of the obtained information. Therefore, the present invention has been made in view of such circumstances, and provides a three-dimensional information reconstructing apparatus capable of reconstructing accurate three-dimensional information with a simple apparatus configuration and less restrictions on an object. It is intended to be provided.

[0005]

Means and Actions for Solving the Problems The above-mentioned problems are solved by the means listed below, and each has the following actions. (Means 1) Image input means for inputting an image of an object,
An image storage unit that stores an image input from the image input unit, a processing unit that calculates three-dimensional information of a subject using the information stored in the image storage unit, Means for obtaining the "direction of", and the input direction information of the image obtained by the means for obtaining the direction is
There is provided a three-dimensional information reconstructing device characterized by inputting to the processing means so as to correspond to the information stored in the image storage means corresponding to the input direction. (Claim 1: The embodiments of FIGS. 1, 2, 6, 8, and 9 correspond).

(Operation 1) When reconstructing three-dimensional information using a plurality of images, the "general direction" of the camera when the images are taken is obtained from the means for obtaining the general direction. (Means 2) The means for obtaining the approximate direction are model generating means for generating a "schematic shape (ie, model)" of a predetermined object, direction input means for inputting a photographing direction, and the model generating means. The three-dimensional information reconstructing apparatus according to [1], further comprising: a unit for displaying the model of the object generated in 1. and a shooting direction input by the direction input unit. (Claim 2: The embodiment of FIGS. 1 and 2 corresponds).

(Operation 2) When the three-dimensional information is reconstructed by using a plurality of images, the object model generation means, the direction input means, and the display means based on the object model and the photographing direction are displayed. Display output.

(Means 3) The means for obtaining the approximate direction is the angle formed by the means for rearranging the images stored in the image storage means and the input direction of the plurality of images input from the image input means. Means for inputting, means for calculating a rough direction of each input image with respect to the object using the information of the rearranged image and the angle formed by the image,
The three-dimensional information reconstructing device according to [1] is provided (claim 3: corresponds to the embodiment of FIG. 8).

(Operation 3) When three-dimensional information is reconstructed using a plurality of images, a means for inputting an arrangement relationship between overlapping images and an angle formed by the plurality of images is provided and can be input in advance.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION The contents of the present invention will be described below with reference to a plurality of embodiments. [First Embodiment] FIG. 1 is a functional block diagram showing a schematic structure of a three-dimensional information reconstructing apparatus according to the present invention.

(Structure 1) As shown in the figure, the present apparatus includes an image input section 1 for inputting an image of a certain object, an image storage section 2 for storing the input image, and an operation section for instructing a desired operation. 3, an image processing unit 4 that performs a predetermined process on the image, an image display unit 5 that displays an image image,
It is mainly composed of a data storage unit 6 for storing and storing the image-processed image data.

The image input unit 1, the image storage unit 2, the image processing unit 4, and the data storage unit 6 are sequentially connected in series, while the operation unit 3 is connected to the image processing unit 4 and the image display unit 5. It is connected to the image display unit 5 so as to connect and instruct to display desired processing, and to sequentially supply image information to be displayed from the image storage unit 2 and the image processing unit 4.

More specifically, the image input section 1 has a lens 10, and further an image sensor 11, a process circuit 12, and a memory 13 in order to optically image and photoelectrically convert an object 14.
And are configured in series to supply the obtained image signal to the subsequent image storage unit 2. Further, it is understood that the image processing unit 4 is configured by the corresponding area detection unit 20, the projection relation calculation unit 21, the model storage unit 22, and the three-dimensional information calculation unit 23 as illustrated.

The operation unit 3 is the model storage unit 2 described above.
2, and the image display unit 5 is connected so that stereoscopic information and the like is supplied from the three-dimensional information calculation unit 23. The image input unit 1 is composed of, for example, an electronic camera and captures an image of the object 14. The captured image information is output to the image storage unit 2. The image information stored in the image storage unit 2 is displayed on the image display unit 5 such as a CRT and is output to the image processing unit 4. In the image processing unit 4, desired processing of the image information is performed in accordance with the input given by the operator via the operation unit 3 to reconstruct the three-dimensional size. The obtained three-dimensional information is displayed on the image display unit 5, stored in the data storage unit 6, and output to the outside of the device for use in a system such as CAD or CG not shown.

(Operation 1) Next, the operation of this apparatus will be described with reference to FIG. FIG. 2 illustrates a detailed configuration of the three-dimensional information reconstructing device according to the first embodiment.

The electronic camera constituting the image input unit 1 includes a lens 10, an image pickup element 11, a process circuit 12 and a memory 13.
Is provided. An object 14, which is a subject, is photographed from a plurality of viewpoints, and its image is formed by the lens 10, converted into an electric signal by the image pickup device 11, processed by the process circuit 12 as a video signal, and recorded in the memory 13. This image information is input to the image storage unit 2. The image information accumulated in the image storage unit is displayed on the display unit 5 and is used for inputting an approximate viewpoint position / direction. This operation is specifically performed as follows.

FIG. 3 shows a display example in the device of the embodiment according to the present invention, (a) is an example of a selection menu screen displayed on the display part of the device, and (b) is a viewpoint on the display part of the device. 6 illustrates a display screen for inputting information.

That is, the image display section 5 is shown in FIG.
As in (b), a menu screen for selecting the general shape of the object 14 is displayed together with the image of the subject accumulated in the image storage unit 2. The schematic shape is, for example, a basic three-dimensional shape such as a rectangular parallelepiped, a cone or a cylinder, and if necessary, a variation in shape such as an aspect ratio may be provided. The operator views the displayed outline shape of the target object and selectively selects a shape close to the shape from the selection menu screen (see FIG. 3A). By the way, in the case of this example, "cuboid 1" is selected and instructed.

The selection menu may display coordinate axes instead of the schematic shape. For example, X in the length direction
A coordinate axis may be set and displayed such that the axis and the width direction are the Y axis and the height direction is the Z axis.

Next, images taken by the electronic camera 1 are sequentially displayed on the screen, and "viewpoint information" indicating from which direction the object 14 is viewed is input on the display screen (see FIG. 3).
(B)). Therefore, by this operation, the approximate “line-of-sight direction information” of the image is obtained and input to the image processing unit 4 together with the image information.

The image processing unit 4 uses a plurality of images to reconstruct three-dimensional information. The general principle of stereo vision based on "the principle of triangulation" is, for example, the first in "Robot Vision" (published by Asakura Shoten) by Horn.
It is explained in Chapter 3. Therefore, based on this principle, if it is found where the same point is located in each image in images observed from different viewpoints, the spatial position of the point can be obtained.

What is practically difficult in this process is to find the correspondence between the respective images, especially when it is not given how the object 14 is observed on the image. Since an attempt is made to interpret the elements of the object in the entire range of, the amount of processing becomes enormous and realization becomes difficult.

Therefore, in the three-dimensional information reconstructing apparatus according to the present invention, the "schematic shape" (hereinafter referred to as "model") and the "image direction" of the object 14 are operated by the operation described above. Is given, it becomes possible to process using this model. That is, the corresponding area detection unit 20 of the image processing unit 4 extracts features such as contours and surfaces from the image,
Correlation is performed based on the characteristics of the model output from the model storage unit 22.

Since the model is selected and the rough observation direction is given, it is possible to predict what kind of feature will be observed in which range on the image. By searching the selected features from the real image within this prediction range,
The position where the features of the model are observed in the real image is obtained. This result is given to the projection relation calculation unit 21, and the parameters of the projection relation between the object and the image are obtained.

That is, the point M on the object is the point m on the image.
When the image is projected on, the optical axis of the camera is the z-axis, the lens position is the origin, the spatial coordinates of M are (x, y, z), and the plane coordinates of m are (u, v). Holds.

-U / f = x / z (Equation 1) -v / f = y / z (Equation 2) where f represents the focal length of the camera.

If the parameters such as the position of the camera in the space and the focal length are determined from the above equations (1) and (2), it is possible to calculate in what range of the space the part of interest on the image is located. . Further, by performing the above-mentioned processing on another image, the elements on the image and the range in space when observed from different viewpoints are obtained.

These results are processed by the three-dimensional information calculation unit 23, and the spatial position of the element is obtained by defining the area where the existence ranges from a plurality of viewpoints intersect as shown in FIG. FIG. 4 is a diagram illustrating elements and spatial positions on an image observed from a plurality of different viewpoints.

Therefore, the three-dimensional shape of the subject can be obtained by similarly performing the above-mentioned processing for a plurality of elements. The obtained three-dimensional information is stored in the data storage unit 6
The image display section 5 simultaneously displays and outputs the image data.

(Modification 1) Note that the three-dimensional information may be modified here if necessary. For example, when the user wants to transform the obtained 3D information and then uses it, the corresponding portion of the displayed 3D information is designated by a "pointing device" or the like and modified to give the desired transformation. May be given. This data storage unit 6
The three-dimensional information temporarily stored in can be output and used for other systems.

According to the above example, in this apparatus, the operator inputs the "general photographing direction" of the image of the object, but other methods can be used to input the general photographing direction. Conceivable. For example, the model of the object may be displayed while being rotated, and the button may be pressed at a position approximately matching the image captured by the camera. Alternatively, a development view may be used as shown in FIG. FIG. 5 is an example of a display screen that displays a photographed image and its corresponding exploded view in the apparatus according to the embodiment of the present invention.

More specifically, the operator selects a development view that matches the general shape of the object from the menu on the screen. Then, the image stored in the image storage unit is displayed on the screen, and the observation direction is designated by superimposing it on the corresponding position on the development view. By repeating this operation for each captured image from different positions in sequence, the approximate capturing direction is input. After that, a predetermined feature extraction process is performed on the image, and the projection relationship between the object and the image defined in the development view regarding the extracted feature is obtained.

Similarly, by performing the same processing as described above on another image, the elements on the image and the range in space when observed from different viewpoints are obtained. Therefore, these results are processed by the three-dimensional information calculation unit 23, and the three-dimensional shape of the subject is obtained.

(Effect 1) The apparatus of the first embodiment described above is configured so that the operator inputs and designates the "approximate shooting direction" of the image of the object by any method as described above. This has the advantage of facilitating the three-dimensional information reconstruction process.

[Second Embodiment] Next, a second embodiment of the three-dimensional information reconstructing apparatus according to the present invention will be described with reference to FIG.

FIG. 6 shows the configuration of the three-dimensional information reconstructing device according to the second embodiment. (Structure 2) As shown in the figure, the present apparatus includes an image input unit 1 for inputting an image of a certain object 14, an image storage unit 2 for storing the input image, and an operation unit 3 for instructing a desired operation.
And an image processing unit 4 that performs a predetermined process on the image.
And an image display unit 5 for displaying an image image, and a data storage unit 6 for storing and storing image-processed image data.

The image input unit 1, the image storage unit 2, the image processing unit 4, and the data storage unit 6 are sequentially connected in series, while the operation unit 3 is connected to the image processing unit 4 and the image display unit 5. It is connected to the image display unit 5 so as to connect and instruct to display desired processing, and to sequentially supply image information to be displayed from the image storage unit 2 and the image processing unit 4.

Here, the same components as those of the apparatus of the first embodiment described above are designated by the same reference numerals, and detailed description thereof will be omitted. For details, see Object 14
In order to optically image and photoelectrically convert
Has a lens 10, an image sensor 11 and a process circuit 1
2 and the memory 13 are connected in series, and the subsequent image storage unit 2
The obtained image signal is supplied to.

Further, the image processing section 4 includes a corresponding area detecting section 2
It is understood that 0 and the projection relation calculation unit 21, the image arrangement calculation unit 24 for obtaining the positional relation between the adjacent images, and the three-dimensional information calculation unit 23 are connected and configured as illustrated. The operation unit 3 is connected to the above-described image arrangement calculation unit 24, and the image display unit 5 is connected so that stereoscopic image information and the like are supplied from the three-dimensional information calculation unit 23.

(Operation 2) For example, when the image of the object 14 is taken by the electronic camera as the image input unit 1, the images are taken so that part of the screens overlap each other. Specifically, it is preferable that about half of the object on the screen is photographed so as to be included in adjacent images. The image thus captured is output to the image storage unit 2 and stored therein.

The stored image is displayed on the display unit 5.
Here, the operator arranges the “overlaps” between the screens on the display screen so as to match each other. At this time, not only images taken by moving the camera in parallel but also images taken from different angle directions may be arranged. In this case, since the appearance of the target object differs depending on the direction, the overlapping portions of the target object do not completely match each other, but other images may be joined by making a rough overlap.

As a result, as shown in FIG. 7, an image in which the spatial connection of images is developed on a plane is obtained. Here, since the images of the object have been captured from four directions, for example, when the right end image and the left end image overlap each other, that information is also input together. For that purpose, an overlapping area may be designated by a pointing device such as a mouse. Alternatively, the angle formed by the images may be input as a numerical value. For example, in the case of taking a picture by making one complete rotation, the angle is 360 degrees.

The image data arranged in this way is first input to the corresponding area detecting section 20 of the image processing section 4. The corresponding area detection unit 20 extracts features such as contours and planes from the images, and based on this, the adjacent images are associated with each other. Alternatively, an appropriate window region may be provided for each image, the correlation value between the two window regions may be obtained while moving one window region, and the position at which the correlation value becomes maximum may be obtained and associated. Good. In this processing as well, since the rough corresponding arrangement is given in advance, the “correspondence” is obtained by searching the limited range, and the processing is easy.

The obtained correspondence result is given to the image layout calculation section 24, and the positional relationship between them is obtained from the correspondence between adjacent images. Here, the spatial angle formed by the image is also obtained from the relationship of the arrangement of the elements of the object on the image. That is, as shown in FIG. 7, when the same object is observed from different angles, the element positions are observed differently,
The angle can be calculated. An example in which a spatial connection of images is developed two-dimensionally is shown in a developed view in (a), and a display example by shooting from two different directions is shown in (b).

The method for this calculation is, for example,
13th of "Robot Vision" by Horn, published by Asakura Shoten
The techniques described in the chapter can be used. Alternatively, when both ends of the surface on which the object is present have parallel ridge lines, the lines are displayed on the screen as described in, for example, Kenichi Kanaya "Image Understanding" (published by Morikita Publishing), Chapter 1. The point of intersection (ie
The direction of the image plane with respect to the target plane can be obtained from the position of "vanishing point").

By repeating the above-described processing, the positional relationship of images obtained by observing the object 14 from various angles can be obtained. This result is processed by the projection relation calculation unit 25, and the projection relation between the image and the object 14 in space is obtained. By using the result, it is calculated in which range of space the element on the image of the three-dimensional information calculation unit 22 is, and by combining the element on the image and the range in space when observed from different viewpoints, The spatial position of the element is determined. Therefore, similarly, such a process is performed for a plurality of elements to obtain the three-dimensional shape of the subject.
Then, the three-dimensional information obtained in this way is stored in the subsequent data storage unit 6.

(Modification 2) In the above description, the angle formed by the images is calculated by using the correspondence between the adjacent images, but as another method, for example, the angle formed by the images is given as a numerical value. You may do it. The configuration at this time is illustrated in FIG. That is, FIG. 8 shows another configuration relating to the three-dimensional information reconstructing apparatus of the second exemplary embodiment.

As shown in the figure, the image processing section 4 is provided with an image angle storage section 28. The information about the angle formed by the image input from the operation unit 3 is stored in the image angle storage unit 28, and this information is processed by the projection relation calculation unit 25 to obtain the projection relation between the image and the object in space. To be The result is used to calculate in which range of space the element on the image of the three-dimensional information calculation unit 23 is located, and by combining the element on the image and the range in space when observed from different viewpoints, The spatial position of the element is determined. By performing similar processing on multiple elements in this way,
The three-dimensional shape of the subject can be obtained.

However, in this case, it is not always necessary to input the angle data for all the images, and for example, it may be input only for several reference images. Therefore, the angle data for each image can be obtained by interpolating the data.

Further, the method of inputting the angle data for the reference image and the method of obtaining the angle formed by the image from the positional relationship of the object elements on the image may be used together.

(Effect 2) Therefore, in the apparatus of the second embodiment described above, the operator inputs and designates the "schematic arrangement" of the image of the object by any method as described above. An approximate shooting direction is required, and as a result, there is an advantage that the three-dimensional information reconstruction processing becomes easy.

[Third Embodiment] A third embodiment of the three-dimensional information reconstructing apparatus according to the present invention will be described with reference to FIG.

FIG. 9 shows the configuration of the three-dimensional information reconstructing device according to the third embodiment. (Structure 3) As shown in the figure, the present device includes an image input unit 1 for inputting an image of a certain object 14, an image storage unit 2 for storing the input image, and an operation unit 3 for instructing a desired operation. An image processing unit 4 that performs a predetermined process on the image
And an image display unit 5 for displaying an image image, and a data storage unit 6 for storing and storing image-processed image data.

The image input unit 1, the image storage unit 2, the image processing unit 4, and the data storage unit 6 are sequentially connected in series, while the operation unit 3 is connected to the image processing unit 4 and the image display unit 5. It is connected to the image display unit 5 so as to connect and instruct to display desired processing, and to sequentially supply image information to be displayed from the image storage unit 2 and the image processing unit 4.

The same components as those of the apparatus according to the first or second embodiment described above are designated by the same reference numerals. More specifically, the image input unit 1 includes a lens 10, an image sensor 11, a process circuit 12, and a memory 13 that are serially configured to optically capture an object 14 and perform photoelectric conversion. The obtained image signal is supplied to the unit 2.

The image processing unit 4 also includes a pattern detection unit 27 for extracting a calibration pattern, a projection relation calculation unit 21, a feature extraction unit 26 for extracting characteristic elements of an object, and a tertiary It is configured to be connected to the raw information calculation unit 23 as shown in the figure. In this example, it is understood that there is no operation unit.

The image of the object 14 is photographed by the image input unit 1. At this time, the general camera position information at the time of photographing is given in advance and recorded together with the image.
As an example of means for this purpose, a predetermined reference pattern, which will be described later, is photographed on the screen when the object is photographed, and the camera position is calculated based on this (that is, the reference pattern).

FIG. 10 is a perspective view showing a polygonal stage (stand) on which a "calibration pattern 30" is drawn as an example of a reference pattern. As an example of the above-mentioned reference pattern, a polygonal stage in which a "calibration pattern 30" such as a lattice is drawn on the side surface as shown in FIG. 10 is used. Object 14 on this stage
The image of the object 14 is captured by the electronic camera (not shown) of the image input unit 1 in a state where the object is placed. At this time, the grid-like calibration pattern 30 is also photographed on a part of the screen. The object 14 is photographed from a plurality of viewpoints, and at this time, the calibration patterns 30 on different surfaces of the stage can also be photographed at the same time. The image thus photographed is output to the image storage unit 2 and temporarily stored therein.

The stored image data is first input to the pattern detection unit 27 while the feature extraction unit 26 of the image processing unit 4 extracts the elements of the object. The pattern detection unit 27 extracts the calibration pattern 30 from the image by an operation such as “contour extraction” or “binarization”. The result is processed by the projection relation calculation unit 25,
The projection relationship between the image and the stage in space is required. That is, as described above, the point on the image and its spatial position are given by the relational expressions as shown in (1) and (2),
If this correspondence is obtained for a certain number of points or more, the projection-related parameters can be obtained.

Therefore, the calibration pattern 3
0 has a predetermined grid-like pattern so that such an operation can be easily performed. Then, using the obtained projection relationship, the three-dimensional information calculation unit 23 performs three-dimensional information calculation processing on the elements of the object on the image extracted by the feature extraction unit 26. That is, the existence range in space of the object element photographed on the image is calculated using the projection relationship from the space to the image. By repeating this process for a plurality of images of the same object, the elements on the image and the range in space when observed from different viewpoints can be combined, and the spatial position of the element can be obtained. Then, similarly, the above-described processing is performed on a plurality of elements to obtain the three-dimensional shape of the subject. Then, the obtained three-dimensional information is stored in the data storage unit 6.

(Modification 3) Since the arrangement and dimensions of the grid used as the calibration pattern 30 are known in advance, it is easy to obtain the projection relationship by image processing of this.

Further, by using the stage as shown in FIG. 10, the spatial positional relationship of different surfaces is known, and it is easy to obtain the spatial positional relationship of images observed from greatly different viewpoints. In order to automatically determine which side of the stage is photographed in the image, it is determined that the colors of the surfaces are different and determination is performed using this color difference, or there is a special method such as specifying the surface. A pattern may be used.

(Operation and Effect 3) As described above, the third
In the apparatus of the embodiment, since the “approximate shooting direction” of the image of the target object is calculated from the calibration pattern captured on the same screen as the image of the target object, the operator does not need to input the designation. The device can automatically judge,
The advantage is that the three-dimensional information reconstruction process is easy and quick.

(Other Modifications) As described above, still another method can be used as the means for calculating the general information at the time of image input shown in the present invention. For example, a position information sensor may be built in or externally attached to the camera, and position information may be recorded for each shooting, and may be recorded in a memory together with image data and output. Examples of such sensors include:
A magnetic sensor-based position sensor or an optical position sensor, which is also used in VR systems and the like, can be used.

Further, for example, a piezo acceleration sensor used for detecting a shake of a video camera or the like may be used to calculate the rough information by integrating the information. Furthermore, a camera may be attached to a mechanical arm and the displacement of this arm may be detected by an encoder.

Besides, various modifications can be made without departing from the scope of the present invention. Although the description has been given above based on a plurality of embodiments, the present invention includes the following inventions.

[1] An image input means for inputting an image of an object, an image storage means for storing the image input from the image input means, and a tertiary image of an object using information stored in the image storage means. Processing means for calculating the original information, means for obtaining a rough direction of the input image with respect to the object,
The input direction information of the image obtained by the means for obtaining the direction is input to the processing means so as to correspond to the information stored in the image storage means corresponding to the input direction. Original information reconstruction device.

(Function and Effect 1): The three-dimensional information reconstruction process becomes easy. [2] The means for obtaining the approximate direction is a predetermined object model generation means, a direction input means for inputting a photographing direction, an object model generated by the model generation means and the direction input means. The three-dimensional information reconstructing device according to [1], further comprising: a unit that displays the image capturing direction.

(Advantageous effect 2): The search range at the time of performing the image reconstruction calculation becomes small, and the three-dimensional information reconstruction processing becomes easy. [3] The means for obtaining the general direction includes means for rearranging the images stored in the image storage means, and means for inputting an angle formed by the input directions of the plurality of images input from the image input means. And means for calculating an approximate direction of each input image with respect to the object using the information of the rearranged image and the angle formed by the image. [1] The described three-dimensional information reconstruction device.

(Operation and Effect 3): The search range of the operation for obtaining the correspondence between a plurality of images becomes small, and the three-dimensional information reconstruction processing becomes easy. (4) The means for obtaining the approximate direction includes means for rearranging the images stored in the image storage means, and means for calculating an angle formed by the input directions of the plurality of images input from the image input means. And means for calculating an approximate direction of each input image with respect to the object using the information of the rearranged image and the angle formed by the image, [1]. 3D information reconstruction device.
(The embodiment of FIG. 6 corresponds).

(Function and Effect 4): When three-dimensional information is reconstructed using a plurality of images, a means for inputting the arrangement relationship of a plurality of overlapping images and calculating the angle formed by the images from this arrangement relationship is provided. Be prepared and be asked for the angle. The search range of the calculation for obtaining the correspondence between a plurality of images becomes small, and the three-dimensional information reconstruction processing becomes easy.

(5) Means for obtaining the above general direction are
A means for forming a predetermined calibration on the image,
The three-dimensional information reconstructing apparatus according to [1], including a unit that calculates a shooting direction of the image from the calibration pattern. (Fig. 9, Fig. 1
0 embodiment corresponds).

(Effect 5): When the three-dimensional information is reconstructed using a plurality of images, means for forming a calibration in the image and a target of the input image based on the calibration pattern photographed in the image It is possible to know the approximate position by providing a means for calculating the “approximate position” with respect to the object. The calculation of the projection relationship from the space to the image becomes easy, and the three-dimensional information reconstruction processing becomes easy.

[0074]

As described above, according to the three-dimensional information reconstructing apparatus of the present invention, the search range when performing the image reconstruction operation becomes small, and the search range of the operation for obtaining the correspondence between a plurality of images becomes small. Similarly, the calculation for obtaining the projection relationship from the space to the image is facilitated, and thus the three-dimensional information reconstruction process is facilitated.

More specifically, in the apparatus of the first embodiment, the operator inputs and specifies the "general photographing direction" of the image of the object by some method, so that the three-dimensional information reconstruction processing becomes easy. . In the apparatus of the second embodiment, the “approximate imaging direction” is obtained on the apparatus side by inputting and designating the “approximate arrangement” of the image of the object. Further, in the apparatus of the third embodiment, the “approximate shooting direction” of the image of the object is configured to be calculated from the calibration pattern taken on the same screen as the image of the object. The user does not particularly need to input the designation, and therefore, the device can be automatically judged without having an operation unit. As a result, the three-dimensional information reconstructing process is easy and quick, and the configuration is simple.

Therefore, it is an object of the present invention to provide a three-dimensional information reconstructing device capable of reconstructing accurate three-dimensional information with a simple device structure and less restrictions on an object.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a three-dimensional information reconstructing apparatus according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram illustrating an operation of the three-dimensional information reconstructing device according to the first embodiment.

FIG. 3 shows a display example in the device of the embodiment according to the present invention, (a) is an example of a selection menu screen displayed on the display part of the device, and (b) is a viewpoint on the display part of the device. A display screen for entering information.

FIG. 4 is an explanatory diagram showing elements and spatial positions on an image observed from a plurality of different viewpoints.

FIG. 5 is an example of a display screen displaying a captured image and its corresponding exploded view in the apparatus according to the embodiment of the present invention.

FIG. 6 is a configuration diagram illustrating an operation of a three-dimensional information reconstructing device according to a second embodiment of the invention.

7A and 7B show display examples of element positions in the apparatus according to the embodiment of the present invention, FIG. 7A is a development view showing an example in which the spatial connection of images is developed in a plane, and FIG. 7B is different. Explanatory drawing which shows the example of a display by the photography from 2 directions.

FIG. 8 is a configuration diagram illustrating another configuration of the three-dimensional information reconstructing device according to the second embodiment of the invention.

FIG. 9 is a configuration diagram illustrating an operation of a three-dimensional information reconstructing device according to a third embodiment of the invention.

FIG. 10 is a perspective view showing a polygonal stage in which a “calibration pattern” is drawn as an example of a reference pattern.

[Explanation of symbols]

1 ... Image input unit, 2 ... Image storage unit, 3 ... Operation unit, 4 ... Image processing unit, 5 ... Image display unit,
6 ... Data storage unit, 10 ... Lens,
11 ... Image sensor, 12 ... Process circuit, 13
... memory, 14 ... object, 20 ... corresponding area detection unit,
21 ... Projection-related calculation unit, 22 ... Model storage unit, 2
3 ... Three-dimensional information calculation unit, 24 ... Image placement calculation unit, 2
5 ... Projection relation calculation unit, 26 ... Feature extraction unit, 27
... pattern detection section, 28 ... image angle storage section, 30 ... calibration pattern.

Claims (3)

[Claims] 1. An image input unit for inputting an image of an object, an image storage unit for storing an image input from the image input unit, and a three-dimensional image of an object using information stored in the image storage unit. The image input direction information obtained by the means for obtaining the direction is the image corresponding to the input direction. A three-dimensional information reconstructing device, wherein the information is input to the processing means so as to correspond to the information stored in the storage means. 2. The means for obtaining the general direction includes a model generating means for generating a general shape (ie, model) of a predetermined object, a direction inputting means for inputting a photographing direction, and the model generating means. 3. The three-dimensional information reconstructing apparatus according to claim 1, further comprising: a unit that displays the model of the object that has occurred and the imaging direction that has been input by the direction input unit. 3. The means for obtaining the general direction inputs the angle formed by the means for rearranging the images stored in the image storage means and the input direction of the plurality of images input from the image input means. And a means for calculating an approximate direction of each input image with respect to the object using the information of the rearranged image and the angle formed by the image. Item 3. The three-dimensional information reconstruction device according to item 1.