JPWO2006075528A1 - 3D object measuring device - Google Patents

Abstract

A pedestal on which a three-dimensional object to be photographed is placed, a pedestal surrounded by an inner wall surface of a cylinder, and a mirror surface cylindrical body having the inner wall surface of the cylinder as a mirror surface, and a lens optical axis facing the pedestal and a lens center axis of the mirror surface cylindrical body. A three-dimensional object on the pedestal was directly viewed by a camera equipped with a fish-eye lens arranged so as to match with, and a photographing and recording means for recording an image obtained through the fish-eye lens. The direct image and the reflection image of the three-dimensional object on the pedestal, which is reflected on the mirror surface of the inner wall of the mirror-cylindrical body, are photographed together. As a result, it is possible to obtain a plurality of images that cover the entire upper, lower, left, and right sides of the three-dimensional object used for stereo measurement from one captured image, and it is possible to easily take correspondence between feature points between images. A multi-viewpoint image of the object from all directions can be obtained by one shot, and the three-dimensional shape of the object can be measured from one image by catadioptric stereoscopic vision.

Description

  According to the present invention, a three-dimensional object is photographed from a plurality of angles, and a plurality of images are captured based on the correspondence between feature points in one image and feature points (corresponding feature points) in other images corresponding to the feature points. The present invention relates to a three-dimensional object measuring device that performs stereo measurement processing of a used three-dimensional object and generates three-dimensional image data of a three-dimensional object.

In recent years, an informatization promotion plan for school education is being promoted in elementary, junior high, and high school education, and measures are being promoted so that computers can be used in all classes of elementary, junior high, and high schools. In addition, broadband networks have become widespread in homes, and various data on the Internet can be easily searched and browsed.
As described above, while hardware resources and network environments are being prepared in the education field, it is pointed out that it is important to enhance attractive educational contents that can be easily used. One of the representative educational contents is an electronic picture book of insects and animals. Even today, there is an electronic picture book that digitizes insect and animal information, and it can be browsed via the Internet. However, the current electronic picture book is a two-dimensional plane that is published in the existing picture book. There are many electronic picture books that can be searched and browsed by importing photographs as digital data into a database, and in particular, in the future, three-dimensional electronic picture books that allow you to see the three-dimensional structure and movement of insects and animals. Development of various electronic picture books is desired.

  Now, it takes a lot of labor and cost by an expert to take in three-dimensional shapes of insects and animals at present, but in the field of education, the dynamic development of various electronic picture books that meet the needs of teachers. For production, it is desirable that teachers at the school can easily edit the electronic picture book without going through experts. In other words, a device that allows ordinary people such as school teachers to easily measure the shape and movement of a three-dimensional object and import it as three-dimensional data, and create an electronic picture book or add content based on the three-dimensional data. Development is needed.

The technology of measuring the shape of a three-dimensional object and capturing it as three-dimensional data is mainly studied in the field of computer vision. There are two methods for measuring the shape of a three-dimensional object: an active measurement method and a passive measurement method. It is known (for example, Patent Document 1).
The former active measurement method includes a direct measurement method using laser and ultrasonic waves, and a pattern projection method in which a pattern is projected by a projector and the surface shape is measured by distortion of the pattern. The latter passive measurement method includes a stereo measurement method in which stereo measurement is performed using two or more cameras. A method of stereo measurement using one camera using a mirror has also been proposed.

JP 2001-241928 A

  First, the conventional active measurement method has a first problem that the measurement range is narrow. In both the direct measurement method using laser and ultrasonic waves, and the pattern projection method in which a pattern is projected by a projector, etc., only the range that is directly irradiated by the irradiation light and is observed by a measuring device such as a camera is measured. There was a problem that I could not.

  The second problem of the conventional active measurement method is that pattern or color image data cannot be obtained. Since the conventional active measurement method is for measuring the outer shape of a three-dimensional object and cannot directly obtain the pattern and color data, it is necessary to prepare another pattern and color data. In other words, this active measurement method is a method mainly for generating a three-dimensional wire frame or a three-dimensional polygon from a three-dimensional object, and it is necessary to separately prepare a developed image as a texture image for the pattern and color data. A large number of steps and a great deal of labor are required to generate the three-dimensional image data.

  Next, the conventional passive measurement method has a problem that the scale of the apparatus becomes large as a first problem. In order to obtain three-dimensional image data based on a photographed image of a three-dimensional object, which is originally a two-dimensional image, a plurality of two-dimensional images obtained by photographing the three-dimensional object from a plurality of viewpoints are required. In order to obtain 3D image data that can withstand the above conditions, it is necessary to prepare a large number of cameras around the 3D object and photograph the entire top, bottom, left, and right sides of the 3D object. As described above, in order to obtain three-dimensional image data that can withstand viewing as an electronic picture book, there is a problem that the scale of the device becomes large.

A second problem with the conventional passive measurement method is that it becomes difficult to match feature points with each other when a large number of images are prepared from a large number of viewpoints. In the stereo measurement method, it is necessary to perform stereo measurement between many images taken from many viewpoints, and in the process, feature points such as singular points and edge parts on the image are selected and matching is performed using the feature points as clues. Do. However, a large number of feature points exist in a three-dimensional object having a complicated shape such as insects and animals, and as the number of feature points increases, the amount of calculation for performing matching increases rapidly, and matching is performed. Becomes difficult.
As a device for reducing the number of cameras in order to reduce the device scale, there is a device that uses a plane mirror. This device alleviates the first problem of the passive measurement method to some extent, but In order to obtain an image that covers the entire top, bottom, left, and right, multiple cameras are still required. Further, the second problem of the passive measurement method remains as an irreversible problem with this device.

Further, for the purpose of simplifying the system configuration, there is known a catadioptric stereo system in which a single camera measures a three-dimensional shape of an object by using a reflected image and a refracted image by a mirror or a prism. A catadioptric stereo system is a system in which multiple lights that travel from different trajectories starting from the same point on the surface of an object are incident on a single camera using optical equipment, and are equivalent to the images observed by the camera from multiple viewpoints. It is a system that realizes stereoscopic vision with a single camera by capturing an image.
This catadioptric stereo vision system has an advantage that an image with parallax can be obtained with one camera, so that correction processing of camera parameters is not necessary, and images of respective viewpoints are taken in synchronization. Further, by arranging the camera and the mirror so that the epipolar line is the same as that on the scanning line, it is possible to limit the range where the corresponding points exist, as in a stereoscopic vision system using a plurality of cameras.
However, the omnidirectional shape measuring method based on the catadioptric stereo system has a problem that it is difficult to obtain a correct omnidirectional shape due to the occurrence of irregular reflection by a mirror and the expansion of the corresponding point search range. .

  In view of the above problems, according to the present invention, the device scale is small, and an image covering the entire upper, lower, left, and right sides of a three-dimensional object is obtained from a small number of captured images captured by a small number of cameras (one or two). In addition, the feature points between the images can be easily matched, and a multi-viewpoint image from all directions of the object can be obtained in one shot. It is an object of the present invention to provide a three-dimensional object measuring device capable of measuring the three-dimensional all-round shape of an object.

In order to achieve the above object, a three-dimensional object measuring device of the present invention is a pedestal on which a three-dimensional object to be imaged is placed, a pedestal surrounded by an inner wall surface of a cylinder, and a mirror surface cylinder having the inner wall surface of the cylinder as a mirror surface. A body, a fish-eye lens facing the pedestal and arranged so that a lens optical axis coincides with a cylinder center axis of the mirror-cylindrical body, and a photographing recording means for recording an image obtained through the fish-eye lens. A direct image of a three-dimensional object directly on the pedestal and a reflection of the three-dimensional object on the pedestal reflected on the mirror surface of the inner wall of the mirror-finished cylindrical body. The image is taken together with the image.
According to the above configuration, the fisheye lens can shoot an image of all viewing angles radially around the lens optical axis, the direct image of the three-dimensional object directly mounted on the pedestal facing the fisheye lens, and the side surface of the mirror cylinder. It is possible to shoot together with the reflection image of the three-dimensional object reflected on the mirror surface of the inner wall. Due to the nature of the cylindrical mirror surface, this reflection image is a reflection of the front image seen from all directions surrounding the three-dimensional object, and the reflection image is taken from all directions. The front image can be obtained at one time.
In addition to placing the three-dimensional object to be photographed on the pedestal, the three-dimensional object may be fixed and supported by a means for fixing and supporting the three-dimensional object, for example, a wire. Also, the table may be used as a pedestal.

Here, there may be a plurality of reflection images of the three-dimensional object reflected on the mirror surface of the side wall of the mirror cylinder. That is, a so-called single reflection image reflected once by a mirror surface and captured by a fisheye lens, a so-called double reflection image reflected twice by a mirror surface and captured by a fisheye lens, etc. is reflected n times (n is a natural number) on the mirror surface. There may be a so-called n-times reflection image captured by a fisheye lens.
It should be noted that although the above is the description of still images, if still images that are continuous in time series are obtained, they can be treated as moving images, and three-dimensional moving image data relating to the movement of the target object can also be obtained. .

Next, in the three-dimensional object measuring apparatus of the present invention, the feature point extracting means for extracting and determining the feature points on the photographed image photographed by the photographing recording means of the camera, and the feature point extracting means for extracting the feature points. Corresponding feature points on the reflection image corresponding to the feature points on the direct image are searched and determined on the reflection image, or correspond to the feature points on the reflection image extracted by the feature point extraction means. Corresponding feature point searching means for searching and determining corresponding feature points on the direct image on the direct image, and the direct image and the reflection image based on the correspondence between the feature points and the corresponding feature points. A stereoscopic measurement process is performed, and a three-dimensional image data generation unit that generates three-dimensional image data of the three-dimensional object from the direct image and the reflection image is provided, and the corresponding feature point searching unit includes the center of the image in the captured image. It is preferable that the corresponding feature point is searched by searching on an extension line connecting the feature point and the feature point.
With the above configuration, it is possible to easily match the feature point and the corresponding feature point corresponding to the feature point between the direct image and the reflected image in the stereo measurement process. That is, in the three-dimensional object measuring device of the present invention, the reflection image is an image developed radially from the center of the pedestal (center of the cylinder), and the feature point and the corresponding feature point are always on the same straight line. Therefore, there is an advantage that the corresponding point search becomes easy.

Next, in the three-dimensional object measuring device of the present invention, the pedestal is made of a transparent material, and the object is made visible from below the pedestal. Two cameras, a first camera and a second camera, are used as the cameras. A camera is preferably provided, and the fisheye lens of the first camera and the fisheye lens of the second camera are preferably arranged so as to face each other with the pedestal interposed therebetween.
With the above configuration, it is possible to capture the upper image and the lower image of the three-dimensional object at the same time by capturing the images with the fisheye lens from the two directions of the so-called upper surface and lower surface of the mirror surface cylindrical body.

Next, in the three-dimensional object measuring device of the present invention, the camera is movable with respect to the mirror-finished cylindrical body along the cylindrical center axis, and the distance between the pedestal and the fisheye lens is variable. Is preferred.
The position where the reflection image of the three-dimensional object is reflected on the mirror surface of the side wall of the mirror-cylindrical body varies depending on the size (height, thickness) and shape of the three-dimensional object to be measured. It is preferable that the position of the fish-eye lens (the distance between the base and the fish-eye lens) can be adjusted in order to properly capture the direct image and the reflected image. Therefore, the distance between the pedestal and the fisheye lens is variable.

Next, in the three-dimensional object measuring device of the present invention, the pedestal, the mirror-cylindrical body, and the camera are allowed to move relative to each other, and with the pedestal fixed, the center of the pedestal is set as the center of rotational movement. It is preferable that the mirror-cylindrical body and the camera can be freely rotated in a dimensional space.
With the above configuration, even when the three-dimensional object has a raised side surface shape or when the three-dimensional object has a concave shape, it is possible to obtain an image required for stereo measurement of the portion. For example, when a three-dimensional object has a steep side surface shape, an image cannot be directly obtained with a fish-eye lens with the mirror surface cylindrical body and the base fixed. Also, if the three-dimensional object has a slightly deep concave shape on the upper surface, it is possible to directly obtain an image with the mirror cylinder and the camera fixed, but as a reflection image, the concave portion has a peripheral wall surface. You can't get behind the shadows of. Therefore, in the state where the pedestal is fixed, it is possible to freely rotate and move the mirror cylinder and the camera in a three-dimensional space with the center of the pedestal as the center of rotational movement. In the case of or in the case of the latter concave shape, it is possible to adjust the arrangement of the mirror surface cylindrical body and the camera so that the direct image and the reflection image of those parts can be taken at different angles. Is.

  According to the three-dimensional object measuring device of the present invention, a fisheye lens can photograph a video of all viewing angles radially around the lens optical axis, and a photographed image of a three-dimensional object placed on a pedestal can be obtained by combining a mirror surface cylindrical body. By doing so, a direct image of the 3D object placed on the pedestal facing the fisheye lens and a reflected image of the 3D object reflected on the mirror surface of the inner wall of the side surface of the mirror cylinder are taken together. can do. Further, according to the three-dimensional object measuring device of the present invention, it is possible to easily match the feature point and the corresponding feature point corresponding to the feature point between the direct image and the reflection image in the stereo measurement process.

An embodiment of a three-dimensional object measuring device according to the present invention will be described below with reference to the drawings. However, the technical scope of the present invention is not limited to the specific applications, shapes and dimensions shown in the following embodiments.
Further, when measuring the three-dimensional object, the means for supporting the three-dimensional object may be any means capable of supporting the object, and examples thereof include a pedestal, a hanging thread, a wire, and a needle. Here, the pedestal is not limited to a small plate-like one that is built into the device housing, and includes a large one such as a desk or a stand. When a desk is used as the supporting means, a mirror-finished cylindrical body, which will be described later, is erected on a desk as a pedestal which is the supporting means.
In the embodiments described below, the supporting means for the three-dimensional object will be described as a pedestal which is a small disk-shaped plate.

1 shows an example of a three-dimensional object measuring device of the present invention according to a first embodiment.
FIG. 1 is a diagram schematically showing the basic configuration of a three-dimensional object measuring device 100 according to the first embodiment.

  Reference numeral 10 is a pedestal on which a three-dimensional object to be photographed is placed. The shape of the pedestal 10 is not limited, but is, for example, a disk shape. The size of the pedestal 10 is such that a three-dimensional object to be photographed can be placed.

20 is a mirror-finished cylindrical body. The mirror-cylindrical body 20 has a cylindrical shape whose cross section is a perfect circle, and its inner wall surface is a mirror surface. That is, the mirror-finished cylindrical body in the present invention means that the inner wall surface of the cylindrical body is cylindrical and has a mirror surface. The outer wall surface of the cylindrical body does not necessarily have to be cylindrical and need not be a mirror surface. For example, the external shape is a quadrangular prism and is made of a plastic material, but there is also included one having a cylindrical hollow inside and a mirror-finished inner wall surface.
The inner diameter of the mirror-cylindrical body 20 has such a size that the pedestal 10 is housed therein, and in this example, the pedestal 10 is housed in the bottom surface portion of the mirror-cylindrical body 20.
Assuming that the height of the three-dimensional object to be imaged is L, the height of the mirror-cylindrical body 20 may be L or more from the base 10 that is the base, but for example, a height of about 2L to 10L from the base 10. Can be

30 is a camera. The camera 30 includes a fisheye lens 31 and a photographing/recording means 32.
The fish-eye lens 31 faces the pedestal 10 and is arranged such that the optical axis of the lens coincides with the central axis of the mirror cylinder 20. In this example, the fish-eye lens 31 is arranged on the upper surface of the mirror-cylindrical body 20 and is installed downward so as to face the pedestal 10. The central axis (lens optical axis) of the fish-eye lens 31 is arranged so as to coincide with the central axis (cylindrical central axis) of the cylindrical mirror body 20.
The photographing/recording means 32 is not particularly limited as long as it can record an image obtained through the fisheye lens 31, and may be a so-called analog image recording means for exposing and recording on an analog film, and a light receiving element such as CCD (Charge Coupled Device). It may be a so-called digital recording means that receives light and records it as digital data. As an example, here, the photographing and recording means 32 is a digital image recording means.

  The image data processing device 40 receives a two-dimensional image taken by the camera 30 and generates three-dimensional image data by stereo measurement processing. The image data processing device 40 includes a feature point extraction unit 41, a corresponding feature point search unit 42, and a three-dimensional image data generation unit 43 (not shown in FIG. 1), which will be described later.

Next, in the configuration example of the three-dimensional object measuring apparatus 100 shown in FIG. 1, the principle and procedure of photographing an object placed on the pedestal 10 and obtaining three-dimensional image data by the stereo measurement method will be described. Here, as an example, an example of reading a three-dimensional object (object 1) having a cone shape (cone shape) will be described.
In order to obtain three-dimensional image data from two-dimensional image data based on the stereo measurement method, as a procedure 1, a procedure for acquiring a two-dimensional image obtained by photographing a target three-dimensional object from a plurality of viewpoints (each different angle). The procedure 2 is a procedure of performing matching processing between the characteristic points between the acquired two-dimensional images and associating each element of the 3D object with each other. The procedure 3 is the difference calculation and the inference calculation based on the stereo measurement processing. Therefore, it is necessary to take a procedure to generate it as three-dimensional image data.

  First, as a procedure 1, the three-dimensional object measuring apparatus 100 of the present invention obtains at least two images, a direct image and a reflection image, by performing a single shooting. Here, the direct image refers to an image obtained by receiving light rays that are directly incident on the fisheye lens 31 from the three-dimensional object on the pedestal 10, that is, an image of the three-dimensional object that is directly visible from the viewpoint of the fisheye lens 31. The reflection image is an image obtained by receiving a ray of light reflected from the three-dimensional object on the pedestal 10 on the mirror surface of the inner wall of the specular cylindrical body 20 and incident on the fisheye lens 31, that is, the specular cylindrical body 20 from the viewpoint of the fisheye lens 31. It refers to a reflection image of a three-dimensional object that is reflected on the mirror surface of the inner wall. In the reflection image, a once-reflection image R1 that is once reflected on the inner wall mirror surface of the mirror surface cylindrical body 20 and enters the fisheye lens 31, and a twice-reflection image that is twice reflected on the inner wall mirror surface of the mirror surface cylindrical body 20 and enters the fisheye lens 31. It includes images obtained by multiple reflections such as R2. That is, what is reflected n times (n is a natural number) on the mirror surface of the inner wall of the mirror-cylindrical body 20 and enters the fisheye lens 31 is an n-times reflected image Rn.

FIG. 2 is a diagram schematically showing how the direct image and the reflected image are received by the camera 30, in a vertical section of the three-dimensional object measuring apparatus 100. In FIG. 2, only a certain point A of the three-dimensional object 1 is focused. Light emitted from a point A of the three-dimensional object 1 placed on the pedestal 10 and directly incident on the fisheye lens 31 is an optical path A0 that directly forms an image. Light emitted from the point A, reflected once on the mirror surface of the inner wall of the mirror-cylindrical body 20 and incident on the fisheye lens 31 is an optical path A1 that forms a once-reflected image. The light emitted from the point A, reflected twice on the mirror surface of the inner wall of the mirror-cylindrical body 20 and incident on the fish-eye lens 31 is an optical path A2 that forms a twice-reflected image.
Although there is an optical path that reflects three times or more, in FIG. 2, only the optical path A2 that reflects twice is shown, and more than that is not shown. Further, although FIG. 2 shows the optical path focusing only on the point A, the light emitted from all the points on the surface of the object 1 enters the fisheye lens 31 according to the same principle as in FIG.

FIG. 3 is a diagram schematically showing an image obtained by photographing the cone-shaped object 1 placed on the pedestal 10 with the camera 30. For reference, a certain point A on the object 1 is shown as a small dot in a white circle. The image seen in the center of FIG. 3 is the direct image D0 of the object 1, which is the cone shape seen from directly above. In FIG. 3, the circle directly surrounding the image D0 is the pedestal 10 and the edge E of the mirror-cylindrical body 20. Next, the outer ring-shaped object surrounding the edge E is the single reflection image R1 of the object 1 reflected on the mirror surface of the inner wall of the mirror-cylindrical body 20. The side surface of the cone shape is reflected around the mirror surface of the inner wall of the surface cylindrical body 20 in a circular manner. Next, the outer ring-shaped object surrounding the once-reflection image R1 is the twice-reflection image R2 of the object 1 reflected on the mirror surface of the inner wall of the mirror-cylindrical body 20. Similar to the one-time reflection image R1, the side surface of the cone shape is reflected around the inner wall mirror surface of the surface cylindrical body 20 around the mirror surface, but the double-reflection image R2 has a point-symmetrical image. That is, the image of a certain point A on the object 1 is on the right side in the direct image D0 (image point G0) and on the right side in the single reflection image R1 (image point G1), but on the left side in the double reflection image R2. It is reflected (image point G2).
It should be noted that in FIG. 3, reflection images of three or more reflections (reflection image R3 and the like) are not shown.
In the three-dimensional object measuring device 100 of the present invention, a plurality of images including the direct image and the reflection image are obtained by once photographing as described above.

  Here, the reason why the direct image and the reflection image obtained by the three-dimensional object measuring apparatus 100 of the present invention can be used for three-dimensional stereo measurement as images taken from a plurality of viewpoints at different angles will be described.

  FIG. 4 is an equivalent development of the shooting viewpoints of the optical path A0 forming a direct image, the optical path A1 forming a single reflection image, and the optical path A2 forming a double reflection image for the point A of the object 1 described in FIG. It is the figure which showed the mode that it did. Although the actual fish-eye lens 31 is at the shooting viewpoint F0, the optical path A1 corresponding to the once-reflection image R1 is equivalent to the optical path A1' incident on the shooting viewpoint F1. Similarly, the optical path A2 corresponding to the twice-reflected image R2 is equivalent to the optical path A2' incident on the photographing viewpoint F2. That is, the point A in the single-reflection image R1 is equivalent to the point A in the image that would be obtained by shooting the object 1 with the virtual camera 30′ at the shooting viewpoint F1, and the point A in the double-reflection image R2. Can be said to be equivalent to the point A in the image that would be obtained by shooting the object 1 with the virtual camera 30″ at the shooting viewpoint F2. When the inner diameter of the mirror-cylindrical body 20 is r, the shooting viewpoint F1 is a position 2r away from the fisheye lens 31, and the shooting viewpoint F2 is a position 4r away from the fisheye lens 31.

  Although the above description using FIG. 4 has focused on only the point A on the right side of the object 1, in reality, all the points on the cone-shaped surface are between the facing mirror surface of the inner surface of the mirror surface cylindrical body. Therefore, the one-time reflection image R1 shown in FIG. 3, which is reflected on the mirror surface of the inner wall of the mirror-cylindrical body 20, has a virtual camera around 360 degrees on the circumference of the radius 2r with the cylinder center axis as the center of rotation. The object 1 is photographed while moving 30′, and it is an image equivalent to an image in which only the central image of each photographed image (a front image of the cone shape that is directly facing directly to the photographing viewpoint) is collected. . Similarly, in the double reflection image R2, the object 1 is photographed while moving the virtual camera 30″ around the circumference of the radius 4r 360 degrees around the cylinder center axis as the center of rotation, and the center image of each photographed image is taken. The image is equivalent to the one that collects only the images. That is, the direct image D0, the reflection image R1, and the reflection image R2 obtained by the three-dimensional object measuring apparatus 100 of the present invention are not only the direct image from the actual viewpoint F0 where the fisheye lens 31 is arranged but also the radius from the cylinder center axis. It includes images photographed from innumerable (number of resolutions) photographing viewpoints arranged so as to surround the circumference of 2r and the circumference of radius 4r. However, although the shooting viewpoints are innumerable, the reflection image is equivalent to the front-to-back image of each shooting viewpoint, that is, a collection of only the central image facing directly in front of the images shot from each shooting viewpoint, Focusing on one point on the object 1 (for example, point A), the number of image data that can be used for stereo measurement is not infinite, and the image of the point directly displayed on the image D0 (for example, the image Point G0), an image of the point (for example, image point G1) in the single reflection image R1 captured from the viewpoint of radius 2r, and a double reflection image R2 captured from the viewpoint of radius 4r. There are only three image data of the image of the relevant point (for example, image point G2). That is, in this example, as the image data used for the stereo measurement processing, the same effect as that obtained when the two-dimensional images from three viewpoints were obtained at one time was obtained.

Here, a method of determining a parameter of a light ray angle which is incident on a camera from a point through a fisheye lens in stereoscopic vision measurement of the three-dimensional object measuring apparatus of the present invention will be described.
With the fisheye lens, the relationship between the angle φ formed by the light ray incident on the fisheye lens and the optical axis and the distance d from the image center on the image plane onto which the light ray is projected can be expressed by the following mathematical formula 1.

FIG. 9 shows a relationship diagram between the angle φ and the distance d from the center of the image. Here, f is the focal length. Further, the function g(φ) is the projection characteristic of the fisheye lens, and the characteristic differs for each lens.
By storing the projection characteristics of the fisheye lens as a database, the angle of the light ray incident on the camera can be determined by the distance from the image center of the point of interest in the captured image.

Next, the procedure 2 of the stereo measurement in the three-dimensional object measuring device 100 of the present invention will be described.
In the procedure 2, stereo measurement processing is performed between the plurality of acquired two-dimensional images and correspondence is performed for each element of the three-dimensional object. However, performing the stereo measurement processing for all pixels entails a huge calculation cost. Normally, image data is transformed into a frequency domain by DCT transformation, Fourier transformation, or the like, singular points or edges in the image are extracted, representative ones of them are selected as characteristic points, and characteristic points in the image are combined with each other. Search and match. Here, a feature point in another image corresponding to a feature point in an image is called a corresponding feature point.

In the case of the simple matching algorithm in the conventional technology, all the feature points are searched and matched by the trial and error method between images, but when the shape of the object becomes complicated, many singular points and edges are generated in the image. Since there are existing feature points, a large number of feature points are selected, and the calculation cost of the matching process becomes enormous.
However, when the three-dimensional object measuring apparatus 100 of the present invention is used, the matching processing between the feature points can be executed using the geometrical relationship between the direct image and the reflection image, so that the calculation cost is significantly reduced. can do. The principle can be explained as follows. As described with reference to FIGS. 3 and 4 above, the images captured by the camera 30 include the direct image D0 and the reflection image (here, the single reflection image R1 and the double reflection image R2). Although shown, there is an important geometrical relationship between the direct image D0 and the reflection images (single reflection image R1 and double reflection image R2). The geometrical relationship mentioned here is a relationship in which corresponding feature points are on the same straight line drawn so as to pass through the center of the image.

An image of a certain point on the object is reflected on the mirror surface of the inner wall of the mirror-faced cylindrical body that faces it. In the case of a double reflection image, it is point-symmetrical, but since it is a position folded back with the center point in between, the feature point on the reflection image corresponding to a certain feature point on the direct image is the feature on the direct image. It is on the extension of the straight line connecting the point and the center point of the image. With reference to FIG. 3, when searching for a corresponding feature point with respect to an image point G0 that is a certain feature point on the image D0, if a search is made on an extension of a straight line connecting the image center C and the image point G0, An image point G1 which is a corresponding feature point in the double reflection image R1 and an image point G2 which is a corresponding feature point in the double reflection image R2 are found. If this geometrical relationship is incorporated into the matching algorithm for the search of the feature points, the calculation cost can be significantly reduced.
Through the above procedure, the corresponding feature points corresponding to the feature points are efficiently searched for by using the geometrical relationship between the feature points.

For reference, an example of an image obtained by photographing a pyramid-shaped object in the prototyped three-dimensional object measuring apparatus 100 of the present invention will be shown.
FIG. 5 shows an image obtained by placing a pyramid-shaped object in the center of the pedestal and capturing a direct image of the object and a reflection image reflected on the mirror surface of the inner wall of the mirror cylinder. The radiation drawn in the image is a line added to the image by post-processing so that the correspondence between the direct image and the reflection image can be easily seen, and such radiation is not reflected from the beginning. Absent.

  The geometrical relationship between the above-mentioned feature points does not change even if the object is placed at a position deviated from the center of the pedestal. FIG. 6 is a diagram showing that the geometrical relationship between the feature points is maintained even when the object is placed at a position displaced from the center of the pedestal. It is placed at a position biased to the upper right, and the reflection image reflected on the mirror surface of the inner wall of the mirror-cylindrical body is much distorted compared to Fig. 5, but the feature points must be on a straight line passing through the center of the image. I understand. In FIG. 6, for the sake of clarity, attention is paid to two feature points directly in the image, and two straight lines connecting the feature points and the center of the image are additionally drawn by post-processing. It can be seen that corresponding feature points in the once-reflection image and the twice-reflection image exist on the straight line.

  The reason for this will be described with reference to FIG. FIG. 7 shows an example of the loci of rays that are reflected twice from one point of the object by the mirror and then enter the camera. 7A is observed from the side, and FIG. 7B is observed from directly above. When the optical axis of the camera and the central axis of the cylindrical mirror coincide with each other, in order for a ray to enter the camera, the ray must pass through the central axis of the cylindrical mirror. When the three-dimensional object measuring apparatus of the present invention is viewed from above as shown in FIG. 7(b), the tangents of the two points of the cylindrical mirror facing each other with the central axis of the cylinder sandwiched are always shown in FIG. 7(b). Become parallel. Therefore, the light ray passing through the center of the cylinder always enters the mirror surface having parallel tangent lines perpendicularly when observed from directly above. The rays starting from the same point always exist on the same straight line, so the epipolar line of the virtual camera is a straight line passing through the center of the cylinder.

FIG. 8 shows a simulation image of a virtual object.
An image is obtained in which the image directly seen from the camera is located at the center of the captured image and the image reflected by the cylindrical mirror is present around it. Further, the reflected images are arranged concentrically from the center of the image in ascending order of the number of reflections.
Here, when the object shown in FIG. 8A has a conical shape, the light ray incident on the camera from the point A on the object surface moves only on the broken line connecting the image center and the point A. Therefore, the search range of the set of corresponding points existing in the captured image can be limited to the straight line passing through the center of the image, and the amount of calculation and the false detection of corresponding points can be reduced.
Similarly, when the object of FIG. 8B has a shape of a quadrangular pyramid, the light ray incident on the camera from the point B on the object surface moves only on the broken line connecting the image center and the point B. Become.
Even when the object in FIG. 8C has a conical shape and the image is placed with the position deviated from the center, the light rays incident on the camera from the point C on the surface of the object are at the center of the image. It moves only on the broken line connecting the point C and the point C.

Next, the procedure 3 for generating three-dimensional image data based on the stereo measurement processing in the three-dimensional object measuring device 100 of the present invention will be described.
In step 3, stereo measurement processing of the direct image and the reflection image is performed based on the correspondence between the feature points and the corresponding feature points obtained in step 2, and the 3D image data of the 3D object is obtained from the direct image and the reflection image. To generate. For example, the difference between the direct image data and the reflected image data or the inference calculation is performed as the three-dimensional image data. In the three-dimensional object measuring device 100 of the present invention, this procedure 3 is not particularly limited, and can be widely applied as long as it is an algorithm for performing stereo measurement processing.

  FIG. 12 is a block diagram showing components of the three-dimensional image data processing device 40 in the three-dimensional object measuring device 100 of the present invention. Note that the hardware may be one that specifically realizes the three-dimensional image data generation processing based on the stereo measurement method by using an information processing organization in which general-purpose personal computer resources and software modules are combined. It may be realized using coded dedicated hardware.

  Reference numeral 41 is a feature point extraction unit that extracts and determines the feature points on the captured image captured by the capturing and recording unit 32 of the camera 30. The present invention is not particularly limited as long as it is an effective feature point extraction algorithm and can be widely applied. For example, the image data is transformed into the frequency domain by the DCT transform, and the singular points or edges in the image are extracted to extract the feature points. Use the selected algorithm.

  Reference numeral 42 is a corresponding feature point searching means, which is a means for searching and determining a corresponding feature point on the reflection image corresponding to the feature point on the direct image extracted by the feature point extracting means 41, or a feature. This is a means for searching and determining a corresponding feature point on the direct image corresponding to the feature point on the reflection image extracted by the point extracting means 41 on the direct image. The corresponding feature point searching means 42 incorporates an algorithm for executing the search for the corresponding feature points by searching using the above-mentioned geometrical relationship, that is, on an extension line connecting the center of the image and the feature points. deep.

  Reference numeral 43 denotes a three-dimensional image data generating means, which performs stereo measurement processing between the direct image and the reflection image based on the correspondence between the feature points obtained by the corresponding feature point searching means 42 and the corresponding feature points. This is a part for generating three-dimensional image data of an object from a direct image which is a two-dimensional image and a reflection image.

Here, as an example of the algorithm of the corresponding point searching means 42, the details of the method using SSD (Sum of Squared Difference) will be described. As described above, the corresponding point searching means 42 can apply an existing stereo measurement processing algorithm, and can apply a wide range of algorithms such as DP matching, which is a non-linear matching method, in addition to SSD.
First, in the algorithm of the corresponding feature point searching means 42, a captured image is expanded in polar coordinates for simplification of processing, and then a small area centering on a certain point is cut out as a search source area from the expanded captured image in polar coordinates. Then, a process for searching an area that is considered to be most similar to the search source area is performed.
In this embodiment, SSD (Sum of Squared Difference) is calculated between two areas, and it is determined that the smaller the SSD value, the more similar the areas are. However, since the image reflected by the cylindrical mirror surface and incident on the camera includes distortion in the radial direction, the distortion of the reflected image should be corrected when obtaining the similarity between regions. However, since the magnitude of the distortion of the reflected image depends on the normal direction of the object plane, which is an unknown parameter, it is impossible to analytically correct the distortion. Therefore, in the present embodiment, in addition to the movement of the search area, the center value of the most similar area is calculated by dynamically enlarging/reducing the reflection image for checking the similarity in the radial direction. It is decided that the set of is the set of corresponding points.
In the following, regarding the algorithm of the corresponding feature point search means 42, (1) polar coordinate expansion processing of the captured image, (2) normalization processing of the size of the search target area, (3) limited processing of the search target area, (4) SSD The calculation processing of the value will be described separately.

(1) Polar coordinate expansion process of captured image As a pre-process for simplifying the calculation of SSD, a polar coordinate expansion process of the captured image is performed. FIG. 10A shows an image example of the result of polar coordinate expansion. As described above, in the image captured by the three-dimensional object measuring device of the present invention, the set of corresponding points always exists on the same straight line passing through the center of the image. Therefore, the range in which a point corresponding to a certain point in the image is searched by expanding the captured image into a polar coordinate image in which the vertical axis represents the distance from the image center and the horizontal axis represents the angle is shown in FIG. It will be limited to the area of the broken line shown in (b).
Assuming that the coordinate value in the image before the polar coordinate expansion is (u, v) and the coordinate value after the polar coordinate expansion is (t, w), the correspondence relationship between the pixel values in the conversion is represented by the following mathematical formulas 2 and 3. Here, t and w are integer values. Further, U represents the number of pixels in the u and v axis directions in the image before polar coordinate expansion, and T and W represent the number of pixels in the t axis and w axis directions in the image after polar coordinate expansion. The image before the polar coordinate expansion is photographed so that the central axis of the cylinder exists at the center of the image.
In the image before polar coordinate expansion, coordinate values (u, v) that do not satisfy the condition of the following Expression 4 are not subjected to conversion.

(2) Normalization Processing of Size of Search Target Area Next, normalization processing of size of the search target area will be described. As described above, the size of the reflected image in the captured image in the w-axis direction changes depending on the incident angle of the light ray and the normal direction of the object plane. For example, in FIGS. 10A and 10B, the size of the image of the object image reflected twice by the mirror and incident on the camera in the w-axis direction is smaller than that of the image reflected once and incident on the camera. .. However, since the normal direction of the object plane is unknown at the measurement stage, it is impossible to analytically correct the change in the image size in the w-axis direction. Therefore, in the three-dimensional object measuring device of the present invention, the SSD value is calculated while dynamically enlarging or reducing the local region in which the similarity is to be examined in the w-axis direction. Here, in order to calculate the SSD value, the size of the reference area (Source) and the evaluation target area (Target) in the search must be the same.
In general, changes in the image size depending on the incident angle of light rays and the direction normal to the object plane are non-linear, but the small image distortion in the local region is small even if it is approximated by linear scaling. Therefore, in the present three-dimensional object measuring device, in order to simplify the processing, the local area is enlarged or reduced by linear interpolation.
In addition, the color of each pixel is represented by the RGB color system. The luminance values l _i (t,w) (i=r,g,b) of R, G, and B at the point (t, w) in the normalized region are the luminance values o of the point before normalization. _i (i = r, g, b) and the target scale s when Source is 1.0 can be calculated from the following Equation 5. Here, trunc(x) is a function that rounds down the decimal point of x.

(3) Search Target Area Limiting Process Next, the search target area limiting process will be described. In the n-th order reflection image of the image captured by the three-dimensional object measuring device of the present invention, the region Target corresponding to the region Source centered on the point (t _S , w _S ) is the spatial region of the image when occlusion does not occur. It does assume a continuity be maintained even reflected in the mirror, a point obtained by using the following formula 6 (t, w n _^g) and present in the vicinity of the. Here, w ⁿ _min and w ⁿ _max respectively indicate the upper limit value and the lower limit value at which the n-th reflection image exists, and w ^S _min and w ^S _max indicate the upper limit value and the lower limit at which Source exists. Indicates the value.

Here, w ⁿ _g a point determined by the equation 6 are those to estimate the position of the Target corresponding to Source based on linear ratio. As described above, the distortion of the reflected image in the w-axis direction is generally non-linear, but the error when the non-linear distortion is approximated by a linear ratio is locally small.
Therefore, when searching for an area corresponding to the point (t _S, w _S) region around the Source, the w-axis direction of the range of the search target, is limited within a certain range around the w ⁿ _g Therefore, the search area is reduced.
When searching the Target corresponding to the region Source whose center is the point (t _S , w _S ) from the n-th reflection image, the upper end w ⁿ _gmin and the lower end w ⁿ _gmax of the range of w which is the center point of the search are as follows. This is expressed by Equation 7. Further, when an image reflected in the mirror and Source areas represent the upper ^w _{n gmin} and lower ^w _{n gmax} by the following equation (8). Here, a is a constant indicating the width of the search range, and the larger the value, the wider the search range.

(4) SSD Value Calculation Processing Next, the SSD value calculation processing will be described. In order to evaluate the similarity between the Source and the Target normalized by linear interpolation, the SSD value between the Source and Target areas is used as the evaluation amount in this three-dimensional object measuring apparatus. The SSD value d _ST between the source area S and the target area T is calculated by the following mathematical expression 9. In the following mathematical formulas, l ^S _i and l ^T _i (t,w) (i=r,g,b) are the brightness of each pixel in Source and Target, respectively. Further, the range of possible values for l ^S _i and l ^T _i (t,w) (i=r,g,b) is both 0 and 255 inclusive.

Here, a flow for searching a Target corresponding to a certain Source will be described with reference to FIG. FIG. 11 shows an example in which the image directly observed from the camera is the entire set S _{all of} Source. The search candidate area corresponding to the Source area Sa (Sa belongs to the set of S _all ) cut out from S _{all with an} arbitrary window size can be limited to the area indicated by the broken line in FIG. 11. Further, the Target corresponding to the Source is considered to exist in the above-described local area, and the area is cut out while changing the window size in the search candidate area, and the SSD value with Sa is calculated. Then, the area Ta 1 having the smallest SSD value is selected as an area corresponding to Sa, and the respective center points are set as a set of corresponding points used in stereoscopic viewing. Similarly, the corresponding points are searched for all the pixels included in S _all .

  As described above, according to the three-dimensional object measuring apparatus 100 of the present invention described in the first embodiment, the three-dimensional object directly placed on the pedestal facing the fisheye lens is directly photographed, and the mirror surface of the inner wall of the side surface of the mirror cylinder is displayed. A reflection image of a three-dimensional object that is reflected can be captured at one time, and a feature point and a corresponding feature point corresponding to the feature point can be easily matched between the direct image and the reflection image, and a stereo image can be obtained. It is possible to generate three-dimensional image data from the two-dimensional image data based on the measurement process.

An example of a three-dimensional object measuring apparatus 100a of the present invention according to a second embodiment is shown.
The three-dimensional object measuring apparatus 100a according to the second embodiment is different from the three-dimensional object measuring apparatus 100 according to the first embodiment in that the pedestal 10 is made of a transparent material, and the first camera 30a is used as the camera 30 as shown in FIG. The second camera 30b is provided with two cameras, and the fisheye lens 31a of the first camera 30a and the fisheye lens 31b of the second camera 30b are arranged so as to face each other with the pedestal 10 interposed therebetween.
This aims at capturing a plurality of images from different viewpoints at the same time not only for the upper image of the object 1 but also for the lower image.

  In the configuration in which one camera 30 is provided above the mirror-cylindrical body 20 shown in the first embodiment, the captured images are the direct image D0 of the upper surface of the object 1 and the reflected image Rn of the upper surface of the object 1. , The direct image D0 of the lower surface of the object 1 and the reflected image Rn of the lower surface of the object 1 cannot be captured together.

Therefore, the three-dimensional object measuring apparatus 100a according to the second embodiment has a configuration in which not only the camera 30a above the specular cylindrical body 20 but also the camera 30b below the specular cylindrical body 20 is provided, and only the upper image of the object 1 is obtained. Of course, as for the lower image, the image D0 and the reflected image Rn are simultaneously captured at the same time. The pedestal 10 needs to be made of a transparent material such as a glass plate so that the object 1 can be photographed from below, and the mirror-cylindrical body 20 needs to extend below the pedestal 10.
Since the principle of capturing the direct image D0 and the reflection image Rn of the lower surface of the object 1 using the lower camera 30b and the principle of the stereo measurement process are the same as those in the first embodiment, the description thereof is omitted here.

An example of a three-dimensional object measuring device 100b of the present invention according to a third embodiment is shown.
The three-dimensional object measuring apparatus 100b according to the third embodiment is different from the three-dimensional object measuring apparatus 100 according to the first embodiment in that the camera 30 is movable with respect to the mirror-cylindrical body 20. As shown in FIG. The camera 30 can move up and down along the central axis of the mirror surface cylindrical body 20, and the distance between the pedestal 10 and the fisheye lens 31 is variable.
The camera 30 may be moved by the user's hand, but by combining the camera case with a stepping motor mechanism, the camera case can be moved according to the user's operation input. With the controllable configuration, the distance between the fisheye lens 31 and the object on the pedestal 10 can be accurately adjusted.

Considering the distance between the pedestal 10 and the fisheye lens 31, the following can be said.
There are advantages and disadvantages when the distance between the pedestal 10 and the fisheye lens 31 is small (when the distance between the object 1 and the camera 30 is short), and conversely, when the distance between the pedestal 10 and the fisheye lens 31 is large (the object 1 and the camera 30). (When the distance of 30 is long) also has advantages and disadvantages, and it is necessary to adjust the distance between the pedestal 10 and the fish-eye lens 31 by combining both.

FIG. 15 is a diagram schematically illustrating advantages and disadvantages when the distance between the pedestal 10 and the fisheye lens 31 is small.
When the distance between the pedestal 10 and the fish-eye lens 31 is reduced, the object can be enlarged by the camera 30, and a detailed direct image D0 of the object surface can be obtained. That is, the resolution of the image directly increases. This is a merit.

  On the other hand, the angle of incidence on the fish-eye lens 31 of the optical path A1 that forms the single-reflection image R1 of the object reflected on the mirror surface of the inner wall of the mirror-cylindrical body 20 becomes large. That is, the height of the photographing viewpoint F1 becomes low, and the once-reflection image R1 corresponds to an image of an object photographed from a viewpoint at a low position in the oblique direction. The angle of incidence on the fisheye lens 31 of the optical path A2 forming the double reflection image R2 of the object reflected on the mirror surface of the inner wall of the mirror-cylindrical body 20 is further increased, and the double reflection image R2 shows the object in a diagonally lower position. However, since the single reflection image R1 originally corresponds to the image captured from a lower position in the diagonal direction, the difference between the double reflection image R2 and the single reflection image R1 is small, which is suitable for stereo measurement. The two-dimensional image used has a small amount of information. This can be said to be a disadvantage.

Next, FIG. 16 is a diagram schematically explaining the advantages and disadvantages when the distance between the pedestal 10 and the fisheye lens 31 is large.
When the distance between the pedestal 10 and the fish-eye lens 31 increases, the object cannot be taken close-up by the camera 30 and the resolution of the direct image D0 on the object surface becomes lower than in the case of FIG. This can be said to be a disadvantage.

  On the other hand, the angle of incidence on the fish-eye lens 31 of the optical path A1 forming the once reflected image R1 of the object reflected on the mirror surface of the inner wall of the mirror-cylindrical body 20 is smaller than that in the case of FIG. That is, the height of the photographing viewpoint F1 becomes high, and the once-reflection image R1 corresponds to an image of the object photographed from a viewpoint at a high position in the oblique direction. The angle of incidence on the fisheye lens 31 of the optical path A2 forming the double reflection image R2 of the object reflected on the mirror surface of the inner surface of the specular cylindrical body 20 becomes large, and the double reflection image R2 is obtained by photographing the object from a diagonally low position viewpoint. The change in the incident angle of the single reflection image R1 and the change of the incident angle of the double reflection image is larger than that in the case of FIG. That is, the difference between the twice-reflection image R2 and the once-reflection image R1 is larger than that in the case of FIG. 15, and the two-dimensional image used for stereo measurement has a larger amount of information than that in the case of FIG. This is a merit.

In this way, there are trade-offs between the merits and demerits that occur when the distance between the pedestal 10 and the fisheye lens 31 is large, and the merits and demerits that occur when the distance between the pedestal 10 and the fisheye lens 31 is small. Furthermore, since the size (height) of the object to be three-dimensionally measured actually affects the variable amount, the distance between the fisheye lens 31 and the object 1 is considered instead of the distance between the pedestal 10 and the fisheye lens 31. There is a need.
Therefore, in the three-dimensional object measuring apparatus 100b of the third embodiment, the camera 30 is movable up and down with respect to the mirror surface cylindrical body 20, and the camera 30 moves up and down along the central axis of the mirror surface cylindrical body 20. The distance between the pedestal 10 and the fisheye lens 31 is variable.

Considering a configuration in which the position of the camera 30 is fixed and the zoom mechanism is provided in the camera 30 instead of moving the camera 30, the size of the direct image D0, particularly, if the optical zoom mechanism is mounted, the direct image is obtained. Although the resolution of D0 can be manipulated, the height of the photographing viewpoint F1 of the reflected image R1 and the photographing viewpoint F2 of the reflected image R2 cannot be manipulated. Further, when the zoom mechanism is used, the peripheral image cannot be captured while the central image looks large, and the higher-order reflection image Rn, and in some cases, the double-reflection image R2 or the single-reflection image R1 is captured in the captured image. It may disappear. Therefore, the three-dimensional object measuring apparatus 100b according to the third embodiment places importance on operating the heights of the photographing viewpoint F1 of the reflection image R1 and the photographing viewpoint F2 of the reflection image R2 while considering the height of the object 1. The camera 30 can move up and down along the central axis of the mirror surface cylindrical body 20, and the distance between the pedestal 10 and the fisheye lens 31 is variable.
However, the above description is not intended to exclude the camera 30 equipped with the optical zoom mechanism, but the camera 30 of the three-dimensional object measuring apparatus 100 according to the present invention is not provided with the optical zoom mechanism or the digital zoom mechanism. It is, of course, possible to employ zoom photography within a range in which the reflection image, which is a peripheral image, can be captured in the image.

An example of a three-dimensional object measuring apparatus 100c of the present invention according to a fourth embodiment is shown.
The three-dimensional object measuring apparatus 100c of the fourth embodiment enables relative movement of the mirror surface cylindrical body 20 and the camera 30 with respect to the pedestal 10 with respect to the three-dimensional object measuring apparatus 100 of the first embodiment. As shown in the figure, with the pedestal 10 fixed, with the center of the pedestal 10 as the center of rotational movement, the mirror-cylindrical cylindrical body 20 and the camera 30 can be freely rotated in a three-dimensional space.
The mirror-cylindrical body 20 may be rotationally moved by the user, but the mirror-cylindrical body 20 and the stepping motor mechanism may be combined to rotate the mirror-cylindrical body 20 according to the user's operation input. If the movement can be controlled, the direction of the photographing viewpoint of the camera 30 can be accurately adjusted so that the angle allows photographing of a region that could not be photographed in the basic posture.

  The three-dimensional object measuring apparatus 100c according to the fourth embodiment can provide a mirror-cylindrical body 20 even when there is a blind spot depending on the shape or direction of the object 1 to be three-dimensionally measured in the basic posture and the direct image D0 or the reflected image Rn cannot be obtained. The camera 30 is integrated with the pedestal 10 to form another angle, and a direct image D0 or a reflection image Rn is obtained even for a portion where a photographed image cannot be obtained due to a blind spot.

  When there is a blind spot depending on the shape and direction of the object 1 and a portion where the direct image D0 or the reflection image Rn cannot be obtained, the following three cases are generally assumed.

  The first is the case where the object 1 has a slightly deep concave shape. For example, when the concave shape is on the upper surface, the image D0 can be directly obtained for the part, but the reflection image R1 or the reflection image R2 obtained from the oblique photographing viewpoint F1 or the photographing viewpoint F2 has a concave bottom portion. May become a blind spot in the peripheral area and may not be captured. In addition, for example, when the concave shape is on the side surface, the reflection image R1 and the reflection image R2 obtained from the oblique photographing viewpoint F1 and the photographing viewpoint F2 facing the part can be obtained, but the direct image D0 has the concave shape. The bottom part may be a blind spot in the peripheral part and may not be captured.

  The second is when the side surface of the object 1 is nearly vertical. In this case, although the reflection image R1 and the reflection image R2 obtained from the oblique photographing viewpoint F1 and the oblique photographing viewpoint F2 facing the part can be obtained, the direct image D0 has too large an angle to obtain an effective image. There are cases. Even if a plurality of reflection images Rn can be obtained, the direct image D0 is important information in the three-dimensional stereo measurement, so it can be said that it is recommended to devise the shooting direction to obtain the direct image D0.

Thirdly, the object 1 has a complicated shape, and a part of the object 1 is shaded by another part. It is typically the root-like part of the ankle joint in insects. For example, although the image D0 can be directly obtained for the part, the reflection image R1 or the reflection image R2 obtained from the oblique photographing viewpoint F1 or the photographing viewpoint F2 may not be captured because it is behind other parts.
Even when there is a blind spot depending on the shape and direction of the object 1 and a portion where the direct image D0 or the reflected image Rn cannot be obtained, the three-dimensional object measuring apparatus 100c according to the fourth embodiment integrates the mirror-finished cylindrical body 20 and the camera 30. The angle is changed with respect to the pedestal 10, and an image at another angle is obtained for obtaining the direct image D0 or the reflection image Rn necessary for the relevant part.

Here, as an example, a case where the side surface of the object 1 is nearly vertical will be described.
FIG. 18 is a diagram schematically showing the light reception of the direct image and the reflection image in the camera 30 in the vertical section of the three-dimensional object measuring apparatus 100c, as in FIG. As shown in FIG. 18, regarding the point B on the side surface portion of the object 1, the optical path B0 corresponding to the direct image D0 cannot be directly incident on the fisheye lens 31. That is, in the direct image D0, the image of the point B on the side surface is not included, or even if it is included, effective information cannot be obtained because the shooting angle is extremely shallow. On the other hand, the optical path B1 of the single reflection corresponding to the reflection image R1 is incident on the fisheye lens 31, and the image of the point B on the side surface portion is included in the single reflection image R1. Similarly, the double-reflection optical path B2 corresponding to the reflection image R2 is incident on the fisheye lens 31, and the double-reflection image R2 also includes the image of the point B on the side surface portion. Thus, although the plurality of reflection images R1 and R2 include the image information about the point B, the information about the point B is not obtained in the direct image D0.

Next, FIG. 19 shows that the mirror-cylindrical cylindrical body 20 and the camera 30 are integrally rotated clockwise in a vertical plane with the center of the pedestal 10 being the center of the rotational movement with the pedestal 10 fixed from the basic posture of FIG. It shows the state of being done. As shown in FIG. 19, the point B on the side surface of the object 1 is in a position that can be easily seen from the shooting viewpoint of the camera 30, and the optical path B0 corresponding to the direct image D0 is directly incident on the fisheye lens 31. That is, the image of the point B on the side surface portion is effectively included in the direct image D0. The single-reflection optical path B1 corresponding to the reflection image R1 also directly enters the fisheye lens 31, and the image of the point B on the side surface portion is included in the single-reflection image R1. Similarly, the double reflection image R2 also includes the image of the point B on the side surface portion.
As described above, with respect to the point B on the side surface portion where sufficient image information is not obtained in the direct image D0 in the basic posture of FIG. 19, sufficient image information is obtained in the direct image D0 in the posture of FIG.

The posture of FIG. 19 is an example of rotation for obtaining the direct image D0 of the point A on the raised right side surface of the object 1, but the left side surface, front surface (front side surface), back surface (back side) of the object 1 are illustrated. In order to obtain the direct image D0 of another side such as the side), the mirror-cylindrical body 20 and the camera 30 may be separately rotated so that the target side faces the fisheye lens 31. Here, although the rotation control means is not limited, for example, a stepping motor or the like may be used.
The purpose of the three-dimensional object measuring device of the present invention is to obtain a plurality of images used for three-dimensional stereo measurement with a small number of times of photographing. In addition, although the angle is changed and another shot is taken, multiple images of a part that is originally difficult to shoot due to the shape and posture of the object can be obtained in two shots. It can be said that the object of the present invention is to obtain a plurality of images used for three-dimensional stereo measurement.
Although the preferred embodiments of the present invention have been illustrated and described above, it will be understood that various modifications can be made without departing from the technical scope of the present invention.

In the fifth embodiment, a prototype of the three-dimensional object measuring device of the present invention is manufactured and the shape of the real object is measured. Below, the result of the shape measurement data using the measured image is shown.
The object measurement environment of the fifth embodiment will be described. As the camera used for the measurement, a Depict D1E manufactured by Opteon was used, and an image of 1024×1024 (pixels) was cut out from an image taken at 1392×1040 (pixels) and used. A ring light was used as a light source, and light was projected from around the camera to the object. The inner diameter of the cylindrical mirror is 90 (mm) and the height is 100 (mm). However, since the camera used in this embodiment can obtain only gray scale images, l ^S _i (t,w) and l ^T _i (t,w) (i=r,g,b) in the above-mentioned mathematical expression 9 are obtained. Is expressed by Equation 10 below. Here, l ^S _gray (t,w) and l ^T _gray (t,w) are the lightness values of the captured images obtained from the camera at Source and Target, respectively. The range of possible values is 0 or more and 255 or less.

In this embodiment, a conical object shown in FIG. 20 is used as the measurement target. The diameter of the bottom surface of this conical object is 56 (mm) and the height is 34 (mm), and the conical surface has a grayscale scene image as a texture.
When the conical object shown in FIG. 20 was photographed by the three-dimensional object measuring device of the present invention, the photographed image shown in FIG. 21 was obtained. In this embodiment, the shape of a conical object is measured from this image.
Here, the window size of the SSD in this embodiment is set to 5×5 (pixels). In addition, the scale s in Equation 5 was changed by 0.1 from 0.5 to 2.0, and the shape was measured by stereoscopic vision using the image directly observed by the camera and the image reflected once by the cylindrical mirror. FIG. 22 shows the result of cone shape measurement by the three-dimensional object measuring device of the present invention.
FIG. 23 shows a scatter diagram in which the horizontal axis represents the distance from the image center and the vertical axis represents the height, regarding the shape of the cone measured by the three-dimensional object measuring apparatus of the present invention.

  As described above, as a result of attempting to measure the shape of an actual object, it has been shown that the three-dimensional object measuring apparatus of the present invention can be used to measure the entire circumference shape of an actual object with a single image. According to the fifth embodiment, the present three-dimensional object measuring device can measure the entire circumference shape of an object with a simple device configuration including only a camera and a cylindrical mirror and a simple shooting process of shooting only one image. It was proved to be useful for the measurement of all-round shape.

The three-dimensional object measuring apparatus of the present invention puts an object in a cylindrical mirror and shoots the object from above with a camera, so that a multi-viewpoint image from all directions of the object can be obtained by one shooting, and a catadioptric stereoscopic image As a result, it is possible to measure the three-dimensional all-round shape of an object from a single image, so it can be used in content creation applications in the educational field, such as electronic picture books of insects and small animals, and by using those contents. , It is possible to enhance the description of moving images. This three-dimensional object measuring device can be used not only in the education field but also in the medical field, academic research field, and other purposes.
The three-dimensional object measuring apparatus of the present invention allows a user to easily measure the shape of a personal possession or efficiently measure the entire circumference shape of many objects because of the simplicity of the apparatus configuration and the photographing process. Useful for purposes. In addition, since the entire circumference shape of an object can be measured from a single image, it is possible to record the entire circumference shape along with the movement of an animal by continuously capturing images. Can be enhanced.

FIG. 2 is a diagram schematically showing the basic configuration of the three-dimensional object measuring device according to the first embodiment. The figure which showed typically the light reception in the camera of a direct image and a reflection image in the longitudinal section of a three-dimensional object measuring device. The figure which shows typically the image obtained by photographing the cone-shaped object placed on the pedestal with a camera. A diagram schematically showing a state in which the shooting viewpoints of the direct image D0, the single reflection image R1, and the double reflection image R2 for the point A of the object are equivalently developed. Figure showing an image taken together with a direct image of a pyramid-shaped object placed in the center of the pedestal and a reflection image reflected on the mirror surface of the inner wall of the mirror cylinder Diagram showing that the geometric relationship between feature points is maintained even when an object is placed at a position displaced from the center of the pedestal The figure which shows an example of the locus|trajectory of the light ray which reflects twice from the one point of an object to the mirror, and injects into a camera. Figure showing a simulation image of a virtual object Relationship diagram between angle φ and distance d from image center Image taken as a result of polar coordinate expansion Diagram showing the search target area FIG. 3 is a block diagram showing components that execute a stereo measurement process in the three-dimensional object measuring apparatus of the present invention. The figure which showed typically the basic composition of the three-dimensional object measuring device which concerns on Example 2. Diagram schematically showing the basic configuration of the three-dimensional object measuring device according to the third embodiment. Diagram that schematically illustrates the advantages and disadvantages when the distance between the pedestal and the fisheye lens is small Diagram that schematically explains the advantages and disadvantages when the distance between the pedestal and the fisheye lens is large The figure which showed typically the basic composition of the three-dimensional object measuring device which concerns on Example 4. The figure which represented typically the optical path of the direct image and the reflection image in the basic posture in the longitudinal section of the three-dimensional object measuring device. The figure which shows the state which rotated the mirror surface cylindrical body and the camera as one body in the vertical direction by 45 degrees in the clockwise direction in the vertical plane with the center of the pedestal as the center of the rotational movement. A diagram showing a real object having a conical shape used as a measurement target Captured image of a real object with a conical shape used as a measurement target Figure showing the result of cone shape measurement using measured images Diagram showing the relationship between the distance from the center of the cone and the height

Explanation of symbols

1 Object 10 Pedestal 20 Mirror Surface Cylindrical Body 30, 30a, 30b Camera 31 Fisheye Lens 32 Photographing Recording Means 40 Image Data Processing Device 41 Feature Point Extracting Means 42 Corresponding Feature Point Searching Means 43 3D Image Data Generating Means 100, 100a, 100b, 100c 3D object measuring device

[0005]
May be fixed and supported by being pierced with a means for fixing and supporting, for example, a wire. Also, the table may be used as a pedestal.
[0013]
Here, there may be a plurality of reflection images of the three-dimensional object reflected on the mirror surface of the side wall of the mirror cylinder. That is, a so-called single reflection image reflected once by the mirror surface and captured by the fisheye lens, a so-called double reflection image reflected twice by the mirror surface and captured by the fisheye lens, etc. is reflected n times (n is a natural number) on the mirror surface. There may be a so-called n-times reflection image captured by a fisheye lens.
It should be noted that, although the above is a description of still images, if still images that are continuous in time series are obtained, they can be treated as moving images, and three-dimensional moving image data relating to the movement of the target object can also be obtained. Become.
[0014]
In order to achieve the above-mentioned object, a three-dimensional object measuring device of the present invention includes a pedestal on which a three-dimensional object to be imaged is placed, the pedestal surrounded by an inner wall surface of a cylinder, and the inner wall surface of the cylinder is a mirror surface. And a fish-eye lens arranged to face the pedestal so that the optical axis of the lens coincides with the central axis of the cylinder of the mirror-cylindrical body, and an image recording unit for recording an image obtained through the fish-eye lens. And a direct image of a three-dimensional object directly on the pedestal, and a three-dimensional image on the pedestal reflected on the inner wall of the mirror-finished cylindrical body, by means of the photographing and recording means of the camera. A three-dimensional object measuring apparatus for photographing together with a reflection image of an object, the characteristic point extracting means for extracting and determining characteristic points on the photographed image photographed by the photographing and recording means of the camera, and the characteristic point extraction. On the reflection image, the corresponding feature point on the reflection image corresponding to the feature point on the direct image extracted by means is searched and determined, or on the reflection image extracted by the feature point extracting means. Means for searching and determining a corresponding feature point on the direct image corresponding to the feature point on the direct image, and searching on an extension line connecting the center of the image and the feature point in the captured image. Corresponding feature point searching means for searching for the corresponding feature point by means of performing a stereo measurement process of the direct image and the reflection image based on the correspondence between the feature point and the corresponding feature point, It is characterized by comprising a three-dimensional image data generating means for generating three-dimensional image data of the three-dimensional object from the reflection image.
With the above configuration, it is possible to easily match the feature point and the corresponding feature point corresponding to the feature point between the direct image and the reflected image in the stereo measurement process. That is, in the three-dimensional object measuring device of the present invention, the reflection image is an image developed radially from the center of the pedestal (center of the cylinder), and the feature point and the corresponding feature point are always on the same straight line. Therefore, there is an advantage that the corresponding point search becomes easy.
In other words, the image of a certain point on the object to be photographed is reflected on the inner wall mirror surface of the mirror-faced cylindrical body facing directly, and in the case of the double reflection image, it becomes point symmetric, but since it is the position folded back with the center point in between. After all, the characteristic point on the reflection image corresponding to a certain characteristic point on the direct image is on the extension of the straight line connecting the characteristic point on the direct image and the center point of the image. By incorporating this geometrical relationship into the matching algorithm for searching the feature points, the calculation cost can be significantly reduced.
The present invention, by combining a mirror surface cylindrical body and a fisheye lens arranged so that the lens optical axis coincides with the cylinder center axis of the mirror surface cylindrical body, without using a complicated feature point detection algorithm, It is possible to efficiently search for the corresponding feature point corresponding to the feature point by using the geometrical relation of.
The geometrical relationship between the above-mentioned feature points does not change even if the object is placed at a position deviated from the center of the pedestal. Even when the object is placed at a position deviated from the center of the pedestal, the geometric relationship between the feature points is maintained. When the object is placed at a position displaced from the center of the pedestal, the reflection image reflected on the mirror surface of the inner wall of the mirror-cylindrical body is distorted, but the feature points are present on a straight line passing through the image center. This is because when the optical axis of the camera and the central axis of the cylindrical mirror coincide, the ray must pass through the central axis of the cylindrical mirror in order for a ray to enter the camera. The tangents of the two points of the cylindrical mirror facing each other are always parallel, and the light ray passing through the center of the cylinder is always incident perpendicularly to the mirror surface with parallel tangents when observed from directly above. Is.
[0015]
Next, in the three-dimensional object measuring device of the present invention, the pedestal is made of a transparent material.

[0006]
And making the object visible from below the pedestal, and having two cameras, a first camera and a second camera, as the cameras, the fisheye lens of the first camera and the fisheye lens of the second camera Are preferably arranged so as to face each other across the pedestal.
With the above configuration, it is possible to capture the upper image and the lower image of the three-dimensional object at the same time by capturing the images with the fisheye lens from the two directions of the so-called upper surface and lower surface of the mirror surface cylindrical body.
[0016]
Next, in the three-dimensional object measuring device of the present invention, the camera is movable with respect to the mirror-finished cylindrical body along the central axis of the cylinder, and the distance between the pedestal and the fisheye lens is set as an object to be photographed. It is preferable that the height of the three-dimensional object is L, and the height is variable from L to 10L from the base that is the base.
The position where the reflection image of the three-dimensional object is reflected on the mirror surface of the side wall of the mirror-cylindrical body varies depending on the size (height, thickness) and shape of the three-dimensional object to be measured. It is preferable that the position of the fish-eye lens (the distance between the base and the fish-eye lens) can be adjusted in order to properly capture the direct image and the reflected image. Therefore, the distance between the pedestal and the fisheye lens is variable.
[0017]
Next, in the three-dimensional object measuring device of the present invention, the pedestal, the mirror-cylindrical body, and the camera are allowed to move relative to each other, and with the pedestal fixed, the center of the pedestal is set as the center of rotational movement. It is preferable that the mirror-cylindrical body and the camera can be freely rotated in a dimensional space.
With the above configuration, even when the three-dimensional object has a raised side surface shape or when the three-dimensional object has a concave shape, it is possible to obtain an image required for stereo measurement of the portion. For example, when a three-dimensional object has a steep side surface shape, an image cannot be directly obtained with a fish-eye lens with the mirror surface cylindrical body and the base fixed. Also, if the three-dimensional object has a slightly deep concave shape on the upper surface, it is possible to directly obtain an image with the mirror cylinder and the camera fixed, but as a reflection image, the concave portion has a peripheral wall surface. You can't get behind the shadows of. Therefore, in the state where the pedestal is fixed, it is possible to freely rotate and move the mirror cylinder and the camera in a three-dimensional space with the center of the pedestal as the center of rotational movement. Even in the case of or the latter concave shape above, it

Claims (5)

A pedestal on which a three-dimensional object to be photographed is placed,
A mirror surface cylindrical body that surrounds the pedestal with an inner wall surface of a cylinder, and uses the inner wall surface of the cylinder as a mirror surface,
A camera provided with a fisheye lens facing the pedestal and arranged so that a lens optical axis thereof coincides with a cylinder center axis of the mirror-finished cylindrical body, and a photographing recording means for recording an image obtained through the fisheye lens. Prepare,
By the photographing and recording means of the camera, a direct image of the three-dimensional object directly on the pedestal and a reflection image of the three-dimensional object on the pedestal reflected on the mirror surface of the inner wall of the mirror cylinder are combined. A three-dimensional object measuring device for shooting. Feature point extracting means for extracting and determining feature points on a photographed image photographed by the photographing recording means of the camera;
The corresponding feature point on the reflection image corresponding to the feature point on the direct image extracted by the feature point extracting unit is searched and determined on the reflection image, or extracted by the feature point extracting unit. Corresponding feature point searching means for searching and determining corresponding feature points on the direct image corresponding to the feature points on the reflection image on the direct image,
Stereo measurement processing of the direct image and the reflection image is performed based on the correspondence relationship between the feature point and the corresponding feature point, and three-dimensional image data of the three-dimensional object is generated from the direct image and the reflection image. A three-dimensional image data generating means,
3. The three-dimensional structure according to claim 1, wherein the corresponding feature point searching means searches the corresponding image by searching an extension line connecting the center of the image and the feature point in the captured image. Object measuring device. The pedestal is made of a transparent material, and the object is visible from below the pedestal,
As the camera, two cameras, a first camera and a second camera, are provided,
The three-dimensional object measuring device according to claim 1, wherein the fish-eye lens of the first camera and the fish-eye lens of the second camera are arranged so as to face each other with the pedestal interposed therebetween.   The three-dimensional object according to any one of claims 1 to 3, wherein the camera is movable along the central axis of the cylinder with respect to the mirror-cylindrical body, and a distance between the pedestal and the fish-eye lens is variable. Measuring device. Enables relative movement between the pedestal, the mirror-cylindrical body, and the camera,
5. Any one of claims 1 to 4 which, while the base is fixed, enables free rotational movement of the mirror-cylindrical body and the camera in a three-dimensional space with the center of the base as the center of rotational movement. A three-dimensional object measuring device according to claim 2.