JP4742190B2 - 3D object measuring device - Google Patents

Description

【Technical field】
[0001]
  In the present invention, a three-dimensional object is photographed from a plurality of angles, and a plurality of images are obtained based on the correspondence between feature points in one image and feature points (corresponding feature points) in another image corresponding to the feature points. The present invention relates to a three-dimensional object measurement apparatus that performs stereo measurement processing of a used three-dimensional object and generates three-dimensional image data of the three-dimensional object.
[Background]
[0002]
  In recent years, informatization promotion plans for school education have been promoted in elementary, middle and high school education, and measures are being promoted so that computers can be used in the classes of all classes at all elementary, middle and high schools. In addition, broadband networks have become widespread at home, and it has become possible to easily search and browse various data on the Internet.
  As described above, while hardware resources and network environments are being developed in the education field, the importance of enhancing attractive educational content that can be easily used has been pointed out. One of the typical educational contents is an electronic picture book of insects and animals. Even now, there is an electronic picture book that digitizes information on insects and animals, and it can be browsed via the Internet, but the current electronic picture book includes a two-dimensional plane published in the existing picture book. There are many electronic pictorial books that only take photographs etc. as digital data, make them databaseable, and can be searched and browsed. Especially in the future, 3D electronic pictorial books that can see the three-dimensional structure and movement of insects and animals, etc. Development of various electronic pictorial books is desired.
[0003]
  Now, taking in the three-dimensional shapes of insects and animals requires a great deal of labor and cost by specialists, but the dynamic development and development of various electronic pictorial books that meet the needs of teachers in the field of education For production, it is desirable that school teachers can easily edit electronic picture books without going through specialists. In other words, an ordinary person such as a school teacher can easily measure the shape and movement of a 3D object, import it as 3D data, and create an electronic picture book or add content based on the 3D data. Development is needed.
[0004]
  The technology for measuring the shape of a 3D object and importing it as 3D data has been researched mainly in the field of computer vision, and there are two methods for measuring the shape of a 3D object: an active measurement method and a passive measurement method. Known (for example, Patent Document 1).
  As the former active measurement method, there are a direct measurement method using a laser or an ultrasonic wave, a pattern projection method in which a pattern is projected by a projector or the like, and a surface shape is measured by distortion of the pattern. The latter passive measurement method includes a stereo measurement method in which two or more cameras are used for stereo measurement. A method of performing stereo measurement using a single camera using a mirror has also been proposed.
[0005]
[Patent Document 1]
JP 2001-241928 A
DISCLOSURE OF THE INVENTION
[Problems to be solved by the invention]
[0006]
  First, the conventional active measurement method has a problem that a measurement range is narrow as a first problem. Both direct measurement methods using lasers and ultrasonic waves, and pattern projection methods that project patterns with projectors, etc., measure only the area that is directly exposed to the irradiation light and that is observed by a measuring instrument such as a camera. There was a problem that it was not possible.
[0007]
  Further, the conventional active measurement method has a second problem that image data of a pattern or color cannot be obtained. The conventional active measurement method is for measuring the outer shape of a three-dimensional object, and cannot obtain pattern and color data directly. Therefore, it is necessary to prepare other patterns and color data. In other words, this active measurement method is mainly a method for generating a three-dimensional wire frame or a three-dimensional polygon from a three-dimensional object, and it is necessary to prepare a development image separately for the pattern and color data as a texture image. In order to generate the dimensional image data, a large number of steps and a great deal of labor are required.
[0008]
  Next, the conventional passive measurement method has a problem that the scale of the apparatus becomes large as a first problem. In order to obtain three-dimensional image data based on a photographed image of a three-dimensional object that is originally a two-dimensional image, a plurality of two-dimensional images obtained by photographing the three-dimensional object from a plurality of viewpoints are required. In order to obtain three-dimensional image data that can withstand, it is necessary to prepare a large number of cameras around the three-dimensional object and to photograph the entire circumference of the three-dimensional object in all directions. Thus, in order to obtain 3D image data that can withstand viewing as an electronic pictorial book, there is a problem that the scale of the apparatus becomes large.
[0009]
  Further, the conventional passive measurement method has a problem that when a large number of images are prepared from a large number of viewpoints, it is difficult to match feature points. In the stereo measurement method, it is necessary to perform stereo measurement between a large number of images taken from a large number of viewpoints. In the process, feature points such as singular points and edge portions are selected, and matching is performed using the feature points as clues. To do. However, 3D objects with complex shapes such as insects and animals have many feature points, and as the number of feature points increases, the amount of calculation for matching them increases rapidly and takes matching. It becomes difficult.
  As a device for reducing the number of cameras in order to reduce the size of the device, there is a device using a plane mirror, but this device alleviates the first problem of the passive measurement method to some extent, but the three-dimensional object In order to obtain an image that covers the entire upper, lower, left, and right circumferences, a plurality of cameras are still required. Furthermore, with this apparatus, the second problem of the passive measurement method remains as a problem that cannot be alleviated.
[0010]
  Also, for the purpose of simplifying the system configuration, a catadioptric stereo vision system that measures the three-dimensional shape of an object with a single camera using a reflected image or a refracted image by a mirror or a prism is known. A catadioptric stereo vision system is equivalent to an image observed with a camera from multiple viewpoints by using multiple optical beams to enter multiple light beams traveling on different orbits starting from the same point on the object surface. It is a system that realizes stereo viewing with a single camera by taking an image.
  This catadioptric stereo vision system has the advantage that a camera with parallax can be obtained with a single camera, so that camera parameter correction processing is not required and images of each viewpoint are photographed in synchronization. In addition, by arranging cameras and mirrors so that the epipolar line is the same as that on the scanning line, it is possible to limit the range where the corresponding points exist, as in the stereo vision system using a plurality of cameras.
  However, the object shape measurement method based on the catadioptric stereo vision system has a problem that it is difficult to obtain a correct object shape due to the occurrence of irregular reflection by a mirror and the expansion of the corresponding point search range. .
[0011]
  In view of the above problems, the present invention has a small apparatus scale, and an image that covers the entire upper, lower, left, and right circumferences of a three-dimensional object is obtained by a small number of captured images captured by a small number of cameras (or two). It is possible to easily correspond between feature points between images, and a multi-viewpoint image from all directions of an object can be obtained by one shooting, and one image is obtained by catadioptric stereovision. An object of the present invention is to provide a three-dimensional object measuring apparatus capable of measuring the three-dimensional shape of an object from the above.
[Means for Solving the Problems]
[0012]
  In order to achieve the above object, a three-dimensional object measuring apparatus of the present invention includes a pedestal on which a three-dimensional object to be photographed is placed, a specular cylinder that surrounds the pedestal with an inner wall surface of a cylinder, and the inner wall surface of the cylinder is a mirror surface. A fisheye lens facing the pedestal and having a lens optical axis coinciding with the cylindrical central axis of the specular cylindrical body, and photographing recording means for recording an image obtained through the fisheye lens A direct image obtained by directly viewing the three-dimensional object on the pedestal and the reflection of the three-dimensional object on the pedestal reflected on the mirror surface of the inner wall of the specular cylindrical body. The image is taken together.
  According to the above configuration, the fish-eye lens can shoot an image of the entire viewing angle radially around the lens optical axis, the direct image obtained by directly photographing the three-dimensional object placed on the pedestal facing the fish-eye lens, and the side surface of the mirror cylinder The reflection image of the three-dimensional object reflected on the mirror surface of the inner wall can be taken at the same time. Due to the nature of the cylindrical mirror surface, this reflection image reflects the front image seen from the omnidirectional viewpoint that surrounds the periphery of the three-dimensional object, and the reflection image is taken from the omnidirectional viewpoint. A front image can be obtained at a time.
  In addition to placing the three-dimensional object to be photographed on the pedestal, the three-dimensional object may be fixedly supported by being stabbed with, for example, a wire. Moreover, you may utilize a tabletop as a base.
[0013]
  Here, there may be a plurality of reflected images of the three-dimensional object reflected on the mirror surface of the inner wall of the side surface of the mirror cylinder. In other words, a so-called once-reflected image reflected once by a mirror surface and captured by a fish-eye lens, or a so-called twice-reflected image reflected twice by a mirror surface and captured by a fish-eye lens is reflected n times (n is a natural number) on the mirror surface. Thus, there may be a so-called n-th reflection image captured by the fisheye lens.
  Although the above is an explanation of still images, if still images that are continuous in time series are obtained, they can be handled as moving images, and three-dimensional moving image data relating to the movement of the target object can also be obtained. Become.
[0014]
  In order to achieve the above-described object, the three-dimensional object measurement apparatus of the present invention includes a pedestal on which a three-dimensional object to be imaged is placed, the pedestal is surrounded by an inner wall surface of the cylinder, and the inner wall surface of the cylinder is a mirror surface. A mirror-shaped cylindrical body, a fish-eye lens facing the pedestal and arranged so that a lens optical axis coincides with a cylindrical central axis of the mirror-shaped cylindrical body, and a photographing recording means for recording an image obtained through the fish-eye lens A direct image obtained by directly viewing the three-dimensional object on the pedestal, and the three-dimensional image on the pedestal that is reflected on the mirror surface of the inner wall of the mirror cylindrical body. A three-dimensional object measuring apparatus for photographing together with a reflection image of an object, wherein a feature point on a photographed image photographed by a photographing recording means of the camera is extracted and determined A corresponding feature point on the reflected image corresponding to the feature point on the direct image extracted by the output unit and the feature point extracting unit is searched and determined on the reflected image, or the feature point extracting unit Means for searching for and determining a corresponding feature point on the direct image corresponding to the feature point on the reflected image extracted by the method on the direct image, the center of the image and the feature point in the captured image Corresponding feature point search means for searching for the corresponding feature point by searching on an extension line connecting the two, and stereo measurement of the direct image and the reflected image based on the correspondence between the feature point and the corresponding feature point 3D image data generation means for performing processing and generating 3D image data of the 3D object from the direct image and the reflection image is provided. That.
  With the above configuration, it is possible to easily match a feature point and a corresponding feature point corresponding to the feature point between the direct image and the reflected image in the stereo measurement process. That is, in the three-dimensional object measuring apparatus of the present invention, the reflection image is an image developed radially from the center of the pedestal (cylindrical center), and the feature points and the corresponding feature points corresponding to each other are always on the same straight line. Therefore, there is an advantage that the corresponding point search is simplified.
  That is, an image of a certain point on the object to be imaged is reflected on the inner wall mirror surface of the mirror cylinder facing the opposite side, and in the case of a twice-reflected image, it is point-symmetrical, but it is a position folded around the center point. Eventually, the feature point on the reflected image corresponding to a certain feature point on the direct image is on an extension of a straight line connecting the feature point on the direct image and the center point of the image. By incorporating this geometrical relationship into the feature point search matching algorithm, the calculation cost can be greatly reduced.
  The present invention combines a specular cylinder and a fisheye lens arranged so that the lens optical axis coincides with the cylindrical central axis of the specular cylinder without using a complicated feature point detection algorithm. The corresponding feature point corresponding to the feature point can be efficiently searched using the geometrical relationship.
  The geometric relationship between the feature points described above does not change even if the object is placed at a position shifted from the center of the pedestal. Even when the object is placed at a position shifted from the center of the pedestal, the geometric relationship between the feature points is maintained. When the object is placed at a position deviated from the center of the pedestal, the reflected image reflected on the inner wall mirror surface of the mirror cylindrical body is distorted, but the feature points exist on a straight line passing through the image center. This is because, when the optical axis of the camera and the central axis of the cylindrical mirror coincide with each other, in order for a light beam to enter the camera, the light beam must pass through the central axis of the cylindrical mirror. The two tangents of the cylindrical mirrors facing each other are always parallel, and the light rays that pass through the center of the cylinder always enter the mirror surface with parallel tangents perpendicularly when viewed from directly above. It is.
[0015]
  Next, in the three-dimensional object measurement apparatus according to the present invention, the pedestal is formed of a transparent material, the object is visible from the lower side of the pedestal, and the two cameras of the first camera and the second camera are used as the cameras. Preferably, a camera is provided, and the fisheye lens of the first camera and the fisheye lens of the second camera are arranged so as to face each other with the pedestal in between.
  With the above-described configuration, it is possible to simultaneously capture an upper image and a lower image of a three-dimensional object by photographing with a fisheye lens from two directions of the upper and lower surfaces of the mirror surface cylindrical body.
[0016]
  Next, in the three-dimensional object measurement apparatus of the present invention, the camera can move along the cylindrical central axis with respect to the mirror cylindrical body, and the distance between the pedestal and the fisheye lens is set as a subject to be photographed. It is preferable that the height of the three-dimensional object is L and is variable within a range of L to 10 L from the base that is the base.
  The position at which the reflection image of the three-dimensional object is reflected on the mirror surface of the inner wall of the side surface of the mirror cylinder varies depending on the size (height, thickness) and shape of the three-dimensional object to be measured. It is preferable that the position of the fisheye lens (the distance between the pedestal and the fisheye lens) can be adjusted in order to appropriately capture the direct image and the reflected image. Therefore, the distance between the pedestal and the fisheye lens is variable.
[0017]
  Next, in the three-dimensional object measuring apparatus of the present invention, the pedestal, the specular cylindrical body, and the camera can be moved relative to each other, and the pedestal is fixed and the center of the pedestal is set as the center of rotational movement. It is preferable that the mirror cylinder and the camera can be freely rotated and moved in a dimensional space.
  With the above configuration, even when the three-dimensional object has a sharp side shape or when the three-dimensional object has a concave shape, an image necessary for stereo measurement can be obtained for the portion. For example, when a three-dimensional object has a sharp side shape, an image cannot be obtained directly with a fisheye lens while the mirror cylinder and the pedestal are fixed. Also, if the 3D object has a slightly deep concave shape on the top surface, it is possible to obtain an image directly with the mirror cylinder and the camera fixed, but the concave part is the peripheral wall surface as a reflection image You can't get behind it. Thus, with the pedestal fixed, the center of the pedestal is used as the center of rotational movement, and the mirror cylindrical body and the camera can be freely rotated and integrated in a three-dimensional space. In the case of the above-mentioned or the latter concave shape, it is possible to adjust the arrangement of the mirror cylindrical body and the camera so that the direct image and the reflected image of those portions can be taken at different angles. It is.
【The invention's effect】
[0018]
  According to the three-dimensional object measuring apparatus of the present invention, an image of the entire viewing angle can be photographed radially around the lens optical axis by a fish-eye lens, and a photographed image of a three-dimensional object placed on a pedestal is obtained by combining a mirror cylindrical body. As a result, a direct image obtained by directly photographing the three-dimensional object placed on the pedestal facing the fisheye lens and a reflected image of the three-dimensional object reflected on the mirror surface of the side wall of the mirror cylindrical body are photographed at once. can do. Further, according to the three-dimensional object measurement apparatus of the present invention, it is possible to easily match a feature point and a corresponding feature point corresponding to the feature point between the direct image and the reflected image in the stereo measurement process.
BEST MODE FOR CARRYING OUT THE INVENTION
[0019]
  Hereinafter, embodiments of the three-dimensional object measurement apparatus of the present invention will be described with reference to the drawings. However, the technical scope of the present invention is not limited to the specific applications, shapes and dimensions shown in the following examples.
  In measuring a three-dimensional object, any means for supporting the three-dimensional object may be used as long as it can support the object, and examples thereof include a pedestal, a hanging thread, a wire, and a needle. Here, the pedestal is not limited to a small plate-like one incorporated in the apparatus housing, but includes a large one such as a desk or a table. When the desk is used as the support means, a mirror cylinder described later is placed on a desk as a pedestal as the support means.
In the embodiment described below, the support means for the three-dimensional object will be described as a pedestal that is a small disk-like plate.
[Example 1]
[0020]
  1 illustrates an example of a three-dimensional object measurement apparatus according to a first embodiment of the present invention.
  FIG. 1 is a diagram schematically illustrating a basic configuration of a three-dimensional object measurement apparatus 100 according to the first embodiment.
[0021]
  Reference numeral 10 denotes a pedestal on which a three-dimensional object to be photographed is placed. Although the shape of the base 10 is not limited, For example, it is set as a disk shape. The size of the pedestal 10 is set so that a three-dimensional object to be photographed can be placed.
[0022]
  Reference numeral 20 denotes a mirror cylindrical body. The mirror cylindrical body 20 has a cylindrical shape with a perfect circle in cross section, and its inner wall surface is a mirror surface. That is, the mirror surface cylindrical body referred to in the present invention refers to a cylindrical body whose inner wall surface is cylindrical and has a mirror surface. The outer wall surface of the cylindrical body does not necessarily have to be a cylindrical shape or a mirror surface. For example, the outer shape is a quadrangular prism and is made of a plastic material, but includes a cylindrical hollow on the inside and a mirror surface on the inner wall.
  The inner diameter of the mirror cylindrical body 20 is large enough to accommodate the pedestal 10 therein. In this example, the pedestal 10 is accommodated in the bottom surface portion of the mirror cylindrical body 20.
  The height of the mirror cylindrical body 20 may be at least L from the base 10 serving as a base, where L is the height of the three-dimensional object to be imaged. It can be.
[0023]
  Reference numeral 30 denotes a camera. The camera 30 includes a fisheye lens 31 and a photographing recording means 32.
  The fisheye lens 31 faces the pedestal 10 and is arranged so that the lens optical axis coincides with the cylindrical central axis of the mirror surface cylindrical body 20. In this example, the fisheye lens 31 is disposed on the upper surface of the mirror cylindrical body 20 and is installed downward so as to face the pedestal 10. The central axis (lens optical axis) of the fisheye lens 31 is arranged so as to coincide with the central axis (cylindrical central axis) of the cylindrical mirror body 20.
  The photographing recording means 32 is not particularly limited as long as it can record an image obtained through the fisheye lens 31, and may be a so-called analog image recording means for exposing and recording on an analog film, and a light receiving element such as a CCD (Charge Coupled Device). It is also possible to use so-called digital recording means for receiving light and recording as digital data. As an example, the photographing recording means 32 is here a digital image recording means.
[0024]
  The image data processing device 40 receives a two-dimensional image photographed by the camera 30 and generates three-dimensional image data by stereo measurement processing. The image data processing device 40 includes a feature point extraction unit 41, a corresponding feature point search unit 42, and a three-dimensional image data generation unit 43 (not shown in FIG. 1), which will be described later.
[0025]
  Next, in the configuration example of the three-dimensional object measuring apparatus 100 shown in FIG. 1, the principle and procedure for obtaining the three-dimensional image data by photographing the object placed on the pedestal 10 and using the stereo measurement method will be described. Here, as an example, an example of reading a three-dimensional object (object 1) having a cone shape (cone shape) will be described.
  In order to obtain three-dimensional image data from two-dimensional image data based on the stereo measurement method, as a procedure 1, a procedure for obtaining a two-dimensional image obtained by photographing a target three-dimensional object from a plurality of viewpoints (different angles). Procedure 2 is a procedure for performing matching processing between feature points between a plurality of acquired two-dimensional images and associating each element of a three-dimensional object. Procedure 3 is difference calculation or inference calculation based on stereo measurement processing. Therefore, it is necessary to follow the procedure to be performed as three-dimensional image data.
[0026]
  First, as the procedure 1, the three-dimensional object measuring apparatus 100 according to the present invention obtains at least two images, that is, a direct image and a reflected image by one shooting. Here, the direct image refers to an image obtained by receiving a light beam directly incident on the fisheye lens 31 from the three-dimensional object on the pedestal 10, that is, an image of a three-dimensional object that can be directly seen from the viewpoint of the fisheye lens 31. The reflected image is an image obtained by receiving a light beam reflected from the three-dimensional object on the pedestal 10 and reflected on the inner wall mirror surface of the specular cylindrical body 20 and incident on the fisheye lens 31, that is, from the viewpoint of the fisheye lens 31. A reflection image of a three-dimensional object reflected on the inner wall mirror surface. The reflected image includes a one-time reflected image R1 that is reflected once on the inner wall mirror surface of the mirror cylindrical body 20 and enters the fisheye lens 31, and a two-time reflected image that is reflected twice on the inner wall mirror surface of the mirror cylinder 20 and incident on the fisheye lens 31. Includes images obtained by multiple reflections such as R2. That is, what is reflected n times (n is a natural number) on the inner wall mirror surface of the mirror cylindrical body 20 and enters the fisheye lens 31 is an n-time reflected image Rn.
[0027]
  FIG. 2 is a diagram schematically showing how the direct image and the reflected image are received by the camera 30 in the longitudinal section of the three-dimensional object measuring apparatus 100. In FIG. 2, attention is paid only to a certain point A of the three-dimensional object 1. Light emitted from a point A of the three-dimensional object 1 placed on the base 10 and directly incident on the fisheye lens 31 is an optical path A0 that directly forms an image. Light that is emitted from the point A, reflected once on the inner wall mirror surface of the mirror cylindrical body 20, and incident on the fisheye lens 31 is an optical path A1 that forms a once reflected image. Light emitted from the point A, reflected twice on the inner wall mirror surface of the mirror cylindrical body 20 and incident on the fisheye lens 31 is an optical path A2 that forms a twice reflected image.
  Although there are optical paths that reflect three or more times, FIG. 2 shows only the optical path A2 that reflects twice, and illustration of the optical paths beyond that is omitted. Further, FIG. 2 shows an optical path focusing only on the point A, but light emitted from all points on the surface of the object 1 enters the fisheye lens 31 according to the same principle as in FIG.
[0028]
  FIG. 3 is a diagram schematically showing an image obtained by photographing the object 1 having a cone shape placed on the pedestal 10 with the camera 30. A point A of the object 1 is shown as a small white dot for reference. In FIG. 3, the image seen at the center is the direct image D0 of the object 1, and the cone shape is seen from directly above. In FIG. 3, the circle directly surrounding the image D <b> 0 is the pedestal 10 and the edge E of the specular cylindrical body 20. Next, the outer ring-shaped object surrounding the edge E is a one-time reflected image R1 of the object 1 reflected on the inner wall mirror surface of the mirror cylinder 20. The cone-shaped side surface is reflected so as to go around the inner wall mirror surface of the surface cylindrical body 20. Next, the outer ring-shaped object surrounding the once reflected image R1 is a twice reflected image R2 of the object 1 reflected on the inner wall mirror surface of the mirror cylindrical body 20. Similar to the single reflection image R1, the cone-shaped side surface is reflected around the inner wall mirror surface of the surface cylindrical body 20, but in the double reflection image R2, the image is a point-symmetric image. That is, the image of a certain point A on the object 1 is on the right side in the direct image D0 (image point G0) and on the right side in the single reflection image R1 (image point G1), but on the left side in the double reflection image R2. It is reflected (image point G2).
  In FIG. 3, the reflection image (reflection image R3 etc.) of three or more reflections is not shown.
  In the three-dimensional object measuring apparatus 100 of the present invention, a plurality of images, that is, a direct image and a reflected image are obtained by one shooting as described above.
[0029]
  Here, the reason why the direct image and the reflected image obtained by the three-dimensional object measuring apparatus 100 of the present invention can be used for three-dimensional stereo measurement as images taken from a plurality of viewpoints at different angles will be described.
[0030]
  4 equivalently develops the photographing viewpoints of the optical path A0 that forms a direct image with respect to the point A of the object 1 described in FIG. 2, the optical path A1 that forms a once-reflected image, and the optical path A2 that forms a twice-reflected image. It is the figure which showed the mode which performed. Although the actual fisheye lens 31 is the photographing viewpoint F0, the optical path A1 corresponding to the once reflected image R1 is equivalent to the optical path A1 'incident on the photographing viewpoint F1. Similarly, the optical path A2 corresponding to the twice reflected image R2 is equivalent to the optical path A2 'incident on the photographing viewpoint F2. That is, the point A in the once reflected image R1 is equivalent to the point A in the image that would be obtained if the object 1 was photographed by the virtual camera 30 ′ at the photographing viewpoint F1, and the point A in the twice reflected image R2 Can be said to be equivalent to a point A in the image that would be obtained if the object 1 was photographed by the virtual camera 30 ″ at the photographing viewpoint F2. If the inner diameter of the mirror cylinder 20 is r, the photographing viewpoint F1 is a position 2r away from the fisheye lens 31, and the photographing viewpoint F2 is a position 4r away from the fisheye lens 31.
[0031]
  The above description using FIG. 4 has been described focusing only on the point A on the right side of the object 1, but in practice, all the points on the cone-shaped surface are between the inner wall mirror surfaces of the facing mirror cylindrical body. Therefore, the one-time reflected image R1 reflected on the inner wall mirror surface of the mirror cylindrical body 20 shown in FIG. 3 is 360 degrees around the circumference of the radius 2r with the center axis of the cylinder as the center of rotation. The object 1 is photographed while moving 30 ′, and is an image equivalent to an image obtained by collecting only the central images of the respective photographed images (front images that are directly opposite to the photographing viewpoint in the cone shape). . Similarly, the twice-reflection image R2 is obtained by shooting the object 1 while moving the virtual camera 30 ″ around the circumference of the radius 4r around the circumference of the cylinder with the center axis of the cylinder as the center of rotation, and the center image of each shot image. It is an image equivalent to an image obtained by collecting only the images. That is, the direct image D0, the reflected image R1, and the reflected image R2 obtained by the three-dimensional object measuring apparatus 100 of the present invention are added to the radius from the center axis of the cylinder in addition to the direct image from the actual viewpoint F0 where the fisheye lens 31 is disposed. It includes images taken from innumerable (number of resolutions) photographing viewpoints arranged so as to surround a circumference 2r apart and a circumference 4r away from the circumference. However, although the number of shooting viewpoints is innumerable, the reflection image is equivalent to a confrontation image of each shooting viewpoint, that is, a collection of only the central images that face the front of the images taken from each shooting viewpoint. If one point on the object 1 is noticed (for example, point A), the number of image data that can be used for stereo measurement is not innumerable, but an image of the point (for example, an image) directly copied to the image D0. The point G0), the image of the point (for example, the image point G1) captured in the single reflection image R1 captured from the viewpoint having the radius 2r, and the two-reflection image R2 captured from the viewpoint of the radius 4r. There are only three image data of the image of the corresponding point (for example, the image point G2). That is, in this example, as the image data used for the stereo measurement process, the same effect as that obtained when two-dimensional images from three viewpoints are obtained at one time is obtained.
[0032]
  Here, a method for determining the parameter of the ray angle incident on the camera from the point through the fisheye lens in the stereo vision measurement of the three-dimensional object measurement apparatus of the present invention will be described.
  In the fisheye lens, the angle φ formed by the light beam incident on the fisheye lens and the optical axis and the light beam are projected.
The relationship of the distance d from the image center on the image plane can be expressed by the following formula 1.
[0033]
[Expression 1]
[0034]
  FIG. 9 shows a relationship diagram between the angle φ and the distance d from the image center. Here, f is a focal length. The function g (φ) is a projection characteristic of the fisheye lens, and the characteristic differs for each lens.
  By having the projection characteristics of the fisheye lens as a database, the angle of light rays incident on the camera can be determined by the distance from the image center of the point of interest in the captured image.
[0035]
  Next, procedure 2 for stereo measurement in the three-dimensional object measurement apparatus 100 of the present invention will be described.
  In the procedure 2, stereo measurement processing is performed between the acquired two-dimensional images and association is performed for each element of the three-dimensional object. However, performing stereo measurement processing for all the pixels increases the calculation cost. Usually, the image data is converted to the frequency domain by DCT transform or Fourier transform, singular points and edges in the image are extracted, their representative ones are selected as feature points, and feature points in the image are Search and match. Here, feature points in other images corresponding to feature points in an image are referred to as corresponding feature points.
[0036]
  In the case of a simple matching algorithm in the prior art, all feature points between images are searched and matched by trial and error, but when the shape of the object becomes complicated, many singular points and edges are included in the image. The number of feature points is selected, and the calculation cost of the matching process becomes enormous.
  However, when the three-dimensional object measuring apparatus 100 of the present invention is used, the matching process between the feature points can be executed using the geometric relationship between the direct image and the reflected image, thereby greatly reducing the calculation cost. can do. The principle can be explained as follows. As described with reference to FIGS. 3 and 4 above, the image captured by the camera 30 includes a direct image D0 and a reflected image (here, two images of a once-reflected image R1 and a twice-reflected image R2). Although shown, there is an important geometric relationship between the direct image D0 and the reflected image (the once reflected image R1 and the twice reflected image R2). The geometric relationship referred to here is a relationship in which corresponding feature points exist on the same straight line drawn so as to pass through the center of the image.
[0037]
  An image of a certain point on the object is reflected on the inner wall mirror surface of the mirror cylinder facing the object. In the case of a twice-reflection image, it is point-symmetrical, but since it is a position folded around the center point, the feature point on the reflection image corresponding to a certain feature point on the direct image is eventually the feature on the direct image. It is on the extension of a straight line connecting the point and the center point of the image. Referring to FIG. 3, when searching for a feature point corresponding to an image point G0 which is a certain feature point directly on the image D0, if a search is performed on an extension line of a straight line connecting the image center C and the image point G0, An image point G1 that is a corresponding feature point in the twice-reflected image R1 and an image point G2 that is a corresponding feature point in the twice-reflected image R2 are found. If this geometric relationship is incorporated in a matching algorithm for searching for feature points, the calculation cost can be greatly reduced.
  By the above procedure, the corresponding feature points corresponding to the feature points are efficiently searched using the geometric relationship between the feature points.
[0038]
  As a reference, an example of an image obtained by photographing a pyramid-shaped object in the prototype three-dimensional object measuring apparatus 100 of the present invention is shown.
  FIG. 5 shows an image obtained by placing a pyramid-shaped object at the center of the pedestal and taking a direct image thereof and a reflection image reflected on the inner wall mirror surface of the mirror cylindrical body. The radiation drawn in the image is a line added to the image by post-processing so that the correspondence between the direct image and the reflected image can be easily seen, and such radiation is not reflected from the beginning. Absent.
[0039]
  The geometric relationship between the feature points described above does not change even if the object is placed at a position shifted from the center of the pedestal. FIG. 6 is a diagram showing that the geometric relationship between the feature points is maintained even when the object is placed at a position shifted from the center of the pedestal. Compared to FIG. 5, the object is moved from the center of the pedestal. The reflection image reflected on the inner wall mirror surface of the mirror cylinder is greatly distorted compared to FIG. 5, but the feature points exist on a straight line passing through the center of the image. I understand. In FIG. 6, for easy understanding, attention is paid directly to two feature points in the image, and two straight lines connecting these feature points and the center of the image are additionally drawn by post-processing. It can be seen that corresponding feature points in the once reflected image and the twice reflected image exist on the straight line.
[0040]
  The reason for this will be described with reference to FIG. FIG. 7 shows an example of the locus of light rays that are reflected twice from a point on the object and incident on the camera. (A) of FIG. 7 is observed from the side, and (b) is observed from right above. When the optical axis of the camera is coincident with the central axis of the cylindrical mirror, in order for a light ray to enter the camera, the light beam must pass through the central axis of the cylindrical mirror. When the three-dimensional object measurement apparatus of the present invention is viewed from above as shown in FIG. 7B, the two tangents of the cylindrical mirrors facing each other across the central axis of the cylinder are always as shown in FIG. 7B. Become parallel. Therefore, the light beam passing through the center of the cylinder always enters the mirror surface having parallel tangents perpendicularly when observed from directly above. Since light rays starting from the same point always exist on the same straight line, the epipolar line of the virtual camera is a straight line passing through the center of the cylinder.
[0041]
  FIG. 8 shows a simulation image obtained by photographing a virtual object.
An image in which an image directly visible from the camera at the center of the captured image and an image reflected by a cylindrical mirror are present around the image is obtained. Further, the reflected images are arranged concentrically from the image center in the order of the small number of reflections.
  Here, when the object shown in FIG. 8A has a conical shape, the light ray incident on the camera from the point A on the object surface moves only on the broken line connecting the center of the image and the point A. For this reason, the search range of the set of corresponding points existing in the photographed image can be limited to a straight line passing through the center of the image, so that the amount of calculation can be reduced and erroneous detection of the corresponding points can be reduced.
  Similarly, when the object shown in FIG. 8B has a quadrangular pyramid shape, the light ray incident on the camera from the point B on the object surface moves only on the broken line connecting the image center and the point B. Become.
  Further, even when the object in FIG. 8C has a conical shape and is photographed with the position shifted from the center, the light ray incident on the camera from the point C on the object surface is the center of the image. And move only on the broken line connecting the point C.
[0042]
  Next, procedure 3 for generating 3D image data based on stereo measurement processing in the 3D object measurement apparatus 100 of the present invention will be described.
  In step 3, stereo measurement processing of the direct image and the reflected image is performed based on the correspondence relationship between the feature point and the corresponding feature point obtained in step 2, and the 3D image data of the 3D object is obtained from the direct image and the reflected image. Is generated. For example, the calculation is performed as three-dimensional image data by calculating the difference between the direct image data and the reflected image data or inferring calculation. In the three-dimensional object measurement apparatus 100 of the present invention, the procedure 3 is not particularly limited, and any algorithm that performs stereo measurement processing can be widely applied.
[0043]
  FIG. 12 is a block diagram showing components of the 3D image data processing apparatus 40 in the 3D object measuring apparatus 100 of the present invention. Note that the hardware may be a specific implementation of 3D image data generation processing based on the stereo measurement method using an information processing organization combined with general-purpose personal computer resources and software modules. It may be realized by using dedicated hardware coded.
[0044]
  Reference numeral 41 denotes feature point extraction means for extracting and determining feature points on a photographed image photographed by the photographing recording means 32 of the camera 30. Although it is not particularly limited and can be widely applied as long as it is an effective feature point extraction algorithm, for example, image data is converted into a frequency domain by DCT transform, and singular points and edges in the image are extracted to extract feature points. Use a selection algorithm.
[0045]
  Corresponding feature point search means 42 is means for searching and determining corresponding feature points on the reflection image corresponding to the feature points on the direct image extracted by the feature point extraction means 41 on the reflection image, or features This is means for searching for and determining corresponding feature points on the direct image corresponding to the feature points on the reflected image extracted by the point extraction means 41 directly on the image. The corresponding feature point search means 42 incorporates an algorithm for executing a search using the above-described geometric relationship, that is, searching for a corresponding feature point by searching on an extension line connecting the center of the image and the feature point. deep.
[0046]
  43 is a three-dimensional image data generating means for performing a stereo measurement process between the direct image and the reflected image based on the correspondence between the feature points obtained by the corresponding feature point searching means 42 and the corresponding feature points, This is a part for generating three-dimensional image data of an object from a direct image and a reflection image which are two-dimensional images.
[0047]
  Here, as an example of the algorithm of the corresponding point search means 42, the details of a method using SSD (Sum of Squared Difference) will be described. As described above, the corresponding point search means 42 can apply an existing stereo measurement processing algorithm, and can apply a wide range of algorithms such as DP matching which is a non-linear matching method in addition to SSD.
  First, in the algorithm of the corresponding feature point search means 42, the captured image is expanded in polar coordinates for simplification of processing, and then a small area centered on a certain point is extracted as a search source area from the captured image expanded in polar coordinates. Thus, a process for searching for an area considered to be most similar to the search source area is performed.
  In this embodiment, an SSD (Sum of Squared Difference) is calculated between two regions, and it is determined that the regions are more similar as the SSD value is smaller. However, since the image reflected by the cylindrical mirror surface and incident on the camera includes distortion in the radial direction, the distortion of the reflected image should be corrected when determining the similarity between the regions. However, since the magnitude of the distortion of the reflected image depends on the normal direction of the object plane, which is an unknown parameter, it is impossible to correct the distortion analytically. Therefore, in this embodiment, in addition to the movement of the search region, the center point of the most similar region is calculated by calculating the SSD value while dynamically scaling the reflection image for examining the similarity in the radial direction. We decided to make the set of the corresponding points.
  In the following, with respect to the algorithm of the corresponding feature point search means 42, (1) polar coordinate development processing of the captured image, (2) normalization processing of the size of the search target region, (3) search target region limitation processing, (4) SSD This will be described separately for the value calculation process.
[0048]
(1) Polar coordinate expansion processing of captured images
  As a pre-process for simplifying the calculation of the SSD, a polar coordinate expansion process of the captured image is performed. FIG. 10A shows an example of an image obtained as a result of polar coordinate development. As described above, in an image photographed by the three-dimensional object measuring apparatus of the present invention, a set of corresponding points always exists on the same straight line passing through the image center. Therefore, the range when searching for a point corresponding to a certain point in the image by expanding the captured image into a polar coordinate image having the vertical axis as the distance from the image center and the horizontal axis as the angle is shown in FIG. It will be limited to the area of the broken line shown in (b).
  When the coordinate value in the image before polar coordinate development is (u, v) and the coordinate value after polar coordinate development is (t, w), the correspondence between the pixel values in the conversion is expressed by the following mathematical formulas 2 and 3. Here, t and w are integer values. U represents the number of pixels in the u and v-axis directions in the image before polar coordinate development, and T and W represent the number of pixels in the t and w-axis directions in the image after polar coordinate development, respectively. The image before the polar coordinate development is taken so that the center axis of the cylinder exists at the center of the image.
  Note that in the image before polar coordinate development, coordinate values (u, v) that do not satisfy the condition of Equation 4 below are not subject to conversion.
[0049]
[Expression 2]
[0050]
[Equation 3]
[0051]
[Expression 4]
[0052]
(2) Normalization processing of search target area size
  Next, a process for normalizing the size of the search target area will be described. As described above, the size of the reflected image in the w-axis direction in the captured image changes depending on the incident angle of the light beam and the normal direction of the object plane. For example, in FIGS. 10A and 10B, the size in the w-axis direction of the image that is reflected twice by the mirror and incident on the camera is smaller than the image that is reflected once and incident on the camera. . However, since the normal direction of the object plane is unknown at the measurement stage, it is impossible to analytically correct the change in the image size in the w-axis direction. Therefore, in the three-dimensional object measurement apparatus of the present invention, the SSD value is calculated while dynamically scaling the local region whose similarity is to be examined in the w-axis direction. Here, in order to calculate the SSD value, the size of the reference region (Source) and the evaluation target region (Target) in the search must be the same.
  In general, the change in image size due to the incident angle of light rays and the normal direction of the object surface is non-linear, but the distortion of minute images in a local region is small even if approximated by linear scaling. For this reason, in this three-dimensional object measurement apparatus, the local region is enlarged or reduced by linear interpolation in order to simplify the processing.
  In addition, the color of each pixel is expressed in the RGB color system. R, G, and B luminance values l at the point (t, w) in the normalized region_i(t, w) (i = r, g, b) is the luminance value o of the point before normalization_i Using (i = r, g, b) and the target scale s when Source is 1.0, it can be calculated from Equation 5 below. Here, trunc (x) is a function for truncating the decimal part of x.
[0053]
[Equation 5]
[0054]
(3) Search target area limiting process
Next, the search target area limiting process will be described. In an nth-order reflection image of an image taken with the three-dimensional object measurement apparatus of the present invention, a point (t_S, W_S) The area Target corresponding to the source Source has a spatial continuity of the image even if it is reflected by the mirror when no occlusion occurs. t, wⁿ _g). Where wⁿ _minAnd wⁿ _maxRespectively indicate the upper limit value and the lower limit value at which the nth reflected image exists, and w^S _minAnd w^S _maxIndicates the upper and lower values where the Source exists.
[0055]
[Formula 6]
[0056]
  Here, the point w obtained by the above equation 6ⁿ _gIs the estimation of the position of the target corresponding to the source based on the linear ratio. As described above, the distortion of the reflected image in the w-axis direction is generally non-linear, but the error when approximating the non-linear distortion with a linear ratio is small when viewed locally.
  So the point (t_S, W_S) To search for an area corresponding to the area Source, the range in the w-axis direction to be searched is wⁿ _gThe search area is reduced by limiting the range to a certain range centered on.
  Center point (t_S, W_S) When searching for the Target corresponding to the area Source from the n-th order reflected image, the upper end w of the range of w as the center point of the searchⁿ _gminAnd lower end wⁿ _gmaxIs expressed by Equation 7 below. When the image reflected by the mirror is used as the source region, the upper end wⁿ _gminAnd lower end wⁿ _gmaxIs expressed by Equation 8 below. Here, a is a constant representing the width of the search range, and the larger the value, the wider the search range.
[0057]
[Expression 7]
[0058]
[Equation 8]
[0059]
(4) SSD value calculation processing
  Next, an SSD value calculation process will be described. In order to evaluate the similarity between the source and the target normalized by linear interpolation, the three-dimensional object measurement apparatus uses the SSD value between the source and target areas as an evaluation amount. SSD value d between this Source area S and Target area T_ST Is obtained by the following formula 9. In the following formula, l^S _iAnd l^T _i(T, w) (i = r, g, b) is the luminance of each pixel in Source and Target, respectively. Also, l^S _iAnd l^T _iThe range of values that (t, w) (i = r, g, b) can take is 0 or more and 255 or less.
[0060]
[Equation 9]
[0061]
  Here, a flow of searching for a target corresponding to a certain source will be described with reference to FIG. FIG. 11 shows an image directly observed from a camera as a whole set S of Sources._allRepresents an example. S_allSource area Sa (Sa is S_allThe search candidate area corresponding to (which belongs to the set of) can be limited to the area indicated by the broken line in FIG. The Target corresponding to the Source is considered to exist in the above-described local region, and the region is cut out while changing the window size in the search candidate region, and the SSD value of Sa is calculated. Then, the region Ta having the smallest SSD value is selected as a region corresponding to Sa, and the respective center points are set as a set of corresponding points used for stereo viewing. Similarly, S_allThe corresponding points are searched for all the pixels included in.
[0062]
  As described above, according to the three-dimensional object measuring apparatus 100 of the present invention shown in the first embodiment, a direct image obtained by directly photographing a three-dimensional object placed on a pedestal facing the fisheye lens, and a mirror surface of the side wall of the mirror cylindrical body The reflected image of the reflected three-dimensional object can be photographed at the same time, and the feature point and the corresponding feature point corresponding to the feature point can be easily matched between the direct image and the reflected image, and stereo. Based on the measurement process, 3D image data can be generated from the 2D image data.
[Example 2]
[0063]
  An example of the three-dimensional object measurement apparatus 100a according to the second embodiment of the present invention will be described.
  The three-dimensional object measurement apparatus 100a according to the second embodiment forms a pedestal 10 of a transparent material with respect to the three-dimensional object measurement apparatus 100 according to the first embodiment, and as shown in FIG. The second camera 30b is provided with two cameras, and the fisheye lens 31a of the first camera 30a and the fisheye lens 31b of the second camera 30b are arranged so as to face each other with the pedestal 10 interposed therebetween.
  This is intended to capture a plurality of images from different viewpoints at the same time not only for the upper image of the object 1 but also for the lower image.
[0064]
  In the configuration in which one camera 30 is provided on the upper part of the specular cylindrical body 20 shown in the first embodiment, what is obtained as a captured image is a direct image D0 on the upper surface of the object 1 and a reflected image Rn on the upper surface of the object 1. The direct image D0 on the lower surface of the object 1 and the reflected image Rn on the lower surface of the object 1 cannot be taken together.
[0065]
  Therefore, the three-dimensional object measuring apparatus 100a according to the second embodiment has a configuration in which the camera 30b is provided not only at the upper part of the mirror cylinder 20 but also at the lower part of the mirror cylinder 20, and only the upper image of the object 1 is provided. Not only the lower image but also the direct image D0 and the reflected image Rn are taken at the same time. The pedestal 10 is made of a transparent material such as a glass plate so that the object 1 can be photographed from below, and the mirror cylindrical body 20 needs to be extended below the pedestal 10.
  Since the principle of photographing the direct image D0 and the reflected image Rn on the lower surface of the object 1 using the lower camera 30b and the principle of the stereo measurement processing are the same as those in the first embodiment, description thereof is omitted here.
[Example 3]
[0066]
  An example of the three-dimensional object measurement apparatus 100b according to the third embodiment of the present invention will be described.
  The three-dimensional object measurement apparatus 100b according to the third embodiment is configured such that the camera 30 is movable with respect to the mirror cylindrical body 20 with respect to the three-dimensional object measurement apparatus 100 according to the first embodiment. The camera 30 can move up and down along the central axis of the mirror cylindrical body 20, and the distance between the base 10 and the fisheye lens 31 is variable.
  The camera 30 may be moved by the user by hand, but the camera case may be moved in accordance with a user operation input by combining the camera case and the stepping motor mechanism. With a controllable configuration, the distance between the fisheye lens 31 and the object on the pedestal 10 can be adjusted accurately.
[0067]
  Considering the distance between the base 10 and the fisheye lens 31, the following can be said.
  There are advantages and disadvantages when the distance between the pedestal 10 and the fisheye lens 31 is small (when the distance between the object 1 and the camera 30 is short), and conversely, when the distance between the pedestal 10 and the fisheye lens 31 is large (the object 1 and the camera 30). (When the distance of 30 is far) has advantages and disadvantages, and it is necessary to adjust the distance between the pedestal 10 and the fisheye lens 31 in combination.
[0068]
  FIG. 15 is a diagram schematically illustrating merits and demerits when the distance between the base 10 and the fisheye lens 31 is small.
  When the distance between the pedestal 10 and the fisheye lens 31 is reduced, the object can be enlarged by the camera 30, and a detailed direct image D0 on the object surface can be obtained. In other words, the resolution of the image directly increases. This is a merit.
[0069]
  On the other hand, the incident angle to the fisheye lens 31 of the optical path A1 that forms the one-time reflected image R1 of the object reflected on the mirror surface of the inner wall of the mirror cylindrical body 20 increases. That is, the height of the shooting viewpoint F1 is reduced, and the once reflected image R1 corresponds to an image obtained by shooting the object from a viewpoint at a low position in the oblique direction. The incident angle to the fisheye lens 31 of the optical path A2 that forms the twice reflected image R2 of the object reflected on the inner wall mirror surface of the mirror cylindrical body 20 is further increased, and the twice reflected image R2 shows the object at a lower position in the oblique direction. However, the difference between the two-reflection image R2 and the one-time reflection image R1 is small, so that the stereo-measurement is used. A two-dimensional image to be used has a small amount of information. This is a disadvantage.
[0070]
  Next, FIG. 16 is a diagram schematically illustrating merits and demerits when the distance between the base 10 and the fisheye lens 31 is large.
  When the distance between the pedestal 10 and the fisheye lens 31 is increased, the object cannot be taken close-up by the camera 30 as compared with the case of FIG. 15, and the resolution of the direct image D0 on the object surface is lowered. This is a disadvantage.
[0071]
  On the other hand, the incident angle to the fisheye lens 31 of the optical path A1 forming the one-time reflected image R1 of the object reflected on the mirror surface of the inner wall of the mirror cylindrical body 20 is smaller than that in FIG. In other words, the height of the shooting viewpoint F1 is increased, and the once reflected image R1 corresponds to an image obtained by shooting the object from the viewpoint at a high position in the oblique direction. The incident angle to the fisheye lens 31 of the optical path A2 forming the twice reflected image R2 of the object reflected on the inner wall mirror surface of the mirror cylindrical body 20 is increased, and the twice reflected image R2 is taken from the viewpoint of the object at a low position in the oblique direction. The change in the incident angle of the once reflected image R1 and the incident angle of the twice reflected image is larger than that in the case of FIG. That is, the difference between the twice reflected image R2 and the once reflected image R1 is larger than that in the case of FIG. 15, and the amount of information is larger than that in the case of FIG. 15 as a two-dimensional image used for stereo measurement. This is a merit.
[0072]
  Thus, there are trade-offs between the advantages and disadvantages that occur when the distance between the pedestal 10 and the fisheye lens 31 is large, and the advantages and disadvantages that occur when the distance between the pedestal 10 and the fisheye lens 31 is small. Furthermore, since the size (height) of the object to be measured three-dimensionally affects the actual variable amount, the distance between the fisheye lens 31 and the object 1 is considered, not the distance between the pedestal 10 and the fisheye lens 31. There is a need.
  Therefore, the three-dimensional object measuring apparatus 100b according to the third embodiment is configured such that the camera 30 is movable up and down with respect to the mirror cylindrical body 20, and the camera 30 moves up and down along the central axis of the mirror cylindrical body 20. The distance between the pedestal 10 and the fisheye lens 31 is variable.
[0073]
  Note that, instead of making the camera 30 movable, considering the configuration in which the position of the camera 30 is fixed and the camera 30 includes a zoom mechanism, the size of the direct image D0, in particular, if the optical zoom mechanism is mounted, the direct image Although the resolution of D0 can be manipulated, the height of the photographing viewpoint F1 of the reflected image R1 and the photographing viewpoint F2 of the reflected image R2 cannot be manipulated. In addition, when the zoom mechanism is used, the peripheral image cannot be captured instead of the central image appearing large, and the higher-order reflected image Rn, and in some cases, the twice-reflected image R2 and the once-reflected image R1 are captured in the captured image. There is also a risk of disappearing. Therefore, the three-dimensional object measuring apparatus 100b according to the third embodiment places importance on manipulating the heights of the shooting viewpoint F1 of the reflected image R1 and the shooting viewpoint F2 of the reflected image R2 while considering the height of the object 1. The camera 30 can move up and down along the central axis of the mirror cylindrical body 20, and the distance between the base 10 and the fisheye lens 31 is variable.
However, the above description is not intended to exclude the camera 30 equipped with an optical zoom mechanism, but as the camera 30 of the three-dimensional object measuring apparatus 100 of the present invention, a camera equipped with an optical zoom mechanism or a digital zoom mechanism. Of course, it is possible to perform zoom shooting within a range where the reflected image that is the peripheral image can be captured in the image.
[Example 4]
[0074]
  An example of a three-dimensional object measurement apparatus 100c according to the present invention according to a fourth embodiment will be described.
  The three-dimensional object measurement apparatus 100c according to the fourth embodiment enables relative movement of the mirror cylindrical body 20 and the camera 30 with respect to the base 10 with respect to the three-dimensional object measurement apparatus 100 according to the first embodiment. As shown in the figure, with the pedestal 10 fixed, the center of the pedestal 10 is the center of rotational movement, and the mirror cylinder 20 and the camera 30 can be freely rotated together in a three-dimensional space.
  The mirror cylinder 20 may be rotated and moved by the user by hand. However, the mirror cylinder 20 is combined with the mirror cylinder 20 and the stepping motor mechanism, and the mirror cylinder 20 is combined with the user's operation input. If the movement can be controlled, the direction of the photographing viewpoint of the camera 30 can be accurately adjusted so as to obtain an angle that allows photographing of a part that could not be photographed in the basic posture.
[0075]
  The three-dimensional object measuring apparatus 100c according to the fourth embodiment has a specular cylindrical body 20 even when a blind spot is generated depending on the shape and direction of the object 1 to be three-dimensionally measured in the basic posture and a portion where the direct image D0 or the reflected image Rn cannot be obtained. The camera 30 is integrated to change the angle with respect to the pedestal 10 so that another angle is obtained, and a device that directly obtains the image D0 or the reflected image Rn is also obtained for a portion where a captured image cannot be obtained due to a blind spot.
[0076]
  When a blind spot is formed depending on the shape and direction of the object 1 and a part where the image D0 or the reflected image Rn cannot be obtained directly is generated, the following three cases are generally assumed.
[0077]
  The first is a case where the object 1 has a slightly deep concave shape. For example, when the concave shape is on the upper surface, the image D0 can be obtained directly for the part, but the reflected image R1 or the reflected image R2 obtained from the oblique photographing viewpoint F1 or the photographing viewpoint F2 has a concave bottom surface portion. May become blind spots at the periphery and may not be visible. Further, for example, when the concave shape is on the side surface, the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and the photographing viewpoint F2 facing the part can be obtained, but in the direct image D0, the concave shape is obtained. The bottom part may become a blind spot of the peripheral part and may not appear.
[0078]
  The second is a case where the side surface of the object 1 is nearly vertical. In this case, the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and the photographing viewpoint F2 facing the part can be obtained, but the direct image D0 is too large to obtain an effective image. There is a case. Even if a plurality of reflection images Rn can be obtained, since the direct image D0 is important information in the three-dimensional stereo measurement, it can be said that it is recommended to devise the shooting direction so as to obtain the direct image D0.
[0079]
  The third is a case where the shape of the object 1 is complicated and a part thereof is shaded by other parts. Typically, it is a site like the base of an ankle joint in insects. For example, although the image D0 can be obtained directly for the part, the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and photographing viewpoint F2 may not be reflected behind the other parts.
  Even in the case where there is a blind spot depending on the shape and direction of the object 1 and a part where the image D0 or the reflected image Rn cannot be directly obtained is obtained, the three-dimensional object measuring apparatus 100c of the fourth embodiment integrates the mirror cylindrical body 20 and the camera 30 together. The angle is changed with respect to the pedestal 10, and an image of another angle for obtaining the necessary direct image D0 or reflected image Rn is obtained for the part.
[0080]
  Here, as an example, a case where the side surface of the object 1 is nearly vertical will be described.
  FIG. 18 is a diagram schematically showing light reception of the direct image and the reflected image in the camera 30 in the longitudinal section of the three-dimensional object measuring apparatus 100c, as in FIG. As shown in FIG. 18, the optical path B0 corresponding to the direct image D0 cannot directly enter the fisheye lens 31 at the point B on the side surface portion of the object 1. That is, the direct image D0 does not include the image of the point B on the side surface portion, or even if it is included, effective information cannot be obtained because the photographing angle is extremely shallow. On the other hand, the one-reflection optical path B1 corresponding to the reflection image R1 is incident on the fisheye lens 31, and the image of the point B on the side surface portion is included in the one-reflection image R1. Similarly, the twice-reflection optical path B2 corresponding to the reflection image R2 is incident on the fisheye lens 31, and the image of the point B on the side surface portion is included in the twice-reflection image R2. As described above, the plurality of reflection images R1 and R2 include image information regarding the point B, but information regarding the point B is not obtained directly in the image D0.
[0081]
  Next, in FIG. 19, from the basic posture of FIG. 18, with the pedestal 10 fixed, the center of the pedestal 10 is the center of rotational movement, and the mirror cylindrical body 20 and the camera 30 are integrally rotated in the vertical plane in the clockwise direction. Shows the state. As shown in FIG. 19, the point B on the side surface of the object 1 is a position that can be seen well from the shooting viewpoint of the camera 30, and the optical path B <b> 0 corresponding to the direct image D <b> 0 directly enters the fisheye lens 31. That is, the image of the point B on the side surface portion is effectively included in the direct image D0. The once reflected optical path B1 corresponding to the reflected image R1 is also directly incident on the fisheye lens 31, and the image of the point B on the side surface portion is included in the once reflected image R1. Similarly, the image of the point B on the side surface portion is also included in the twice reflected image R2.
  As described above, sufficient image information can be obtained in the direct image D0 in the posture of FIG. 19 for the point B of the side surface portion in which sufficient image information cannot be obtained in the direct image D0 in the basic posture in FIG.
[0082]
  Note that the posture in FIG. 19 is an example of rotation for obtaining a direct image D0 of the point A on the right side of the object 1 that stands out, but the left side, the front (front side), and the back (back side) of the object 1. When the direct image D0 of the other side surface such as the side surface is obtained, the mirror cylindrical body 20 and the camera 30 may be separately rotated so that the target side surface faces the fisheye lens 31 separately. Here, the rotation control means is not limited. For example, a stepping motor or the like may be used.
  The three-dimensional object measurement device of the present invention is intended to obtain a plurality of images used for three-dimensional stereo measurement with a small number of photographing times. In addition, the image is taken one more time at different angles. However, it is affected by the shape and posture of the object, and multiple images can be obtained for the part that was originally difficult to shoot. It can be said that this is in line with the object of the present invention to obtain a plurality of images used for three-dimensional stereo measurement.
  While preferred embodiments of the invention have been illustrated and described, it will be appreciated that various changes can be made without departing from the scope of the invention.
[Example 5]
[0083]
  In the fifth embodiment, a prototype of the three-dimensional object measuring apparatus of the present invention is manufactured and the shape of an actual object is measured. The shape measurement data result using the actual measurement image is shown below.
  An object measurement environment according to the fifth embodiment will be described. The camera used for the measurement was Opteon DepictD1E, and an image of 1024 × 1024 (pixels) was cut out from the image taken at 1392 × 1040 (pixels). Also, a ring light was used as a light source, and light was projected from the camera periphery to the object. The inner diameter of the cylindrical mirror is 90 (mm) and the height is 100 (mm). However, since the camera used in the present embodiment can obtain only a grayscale image,^S _i(T, w) and l^T _i(T, w) (i = r, g, b) is represented by the following formula 10. Where l^S _gray(T, w) and l^T _gray(T, w) is the brightness value of the captured image obtained from the camera at Source and Target, respectively. Moreover, the range of possible values is 0 or more and 255 or less in both cases.
[0084]
[Expression 10]
[0085]
  In this embodiment, a cone-shaped object shown in FIG. 20 is used as a measurement target. The cone-shaped object has a bottom surface diameter of 56 (mm) and a height of 34 (mm), and the cone surface has a gray scale scene image as a texture.
  When the cone-shaped object shown in FIG. 20 is photographed by the three-dimensional object measuring apparatus of the present invention, the photographed image shown in FIG. 21 is obtained. In this embodiment, the shape of the cone object is measured from this image.
  Here, the SSD window size in this embodiment is 5 × 5 (pixels). In addition, the scale s in Equation 5 was changed by 0.1 from 0.5 to 2.0, and the shape was measured in stereo using an image directly observed by the camera and an image reflected once by the cylindrical mirror. FIG. 22 shows the result of cone shape measurement by the three-dimensional object measurement apparatus of the present invention.
  FIG. 23 shows a scatter diagram in which the horizontal axis represents the distance from the image center and the vertical axis represents the height of the cone shape measured by the three-dimensional object measurement apparatus of the present invention.
[0086]
  As described above, as a result of an attempt to measure the shape of an actual object, it has been shown that the entire shape of the actual object can be measured with a single image using the three-dimensional object measurement apparatus of the present invention. According to the fifth embodiment, the three-dimensional object measuring apparatus can measure the entire circumference of an object with a simple apparatus configuration including only a camera and a cylindrical mirror and a simple imaging process for capturing only one image. It has been shown that it is useful for the measurement of the entire circumference.
[Industrial applicability]
[0087]
  The three-dimensional object measuring apparatus of the present invention can obtain a multi-viewpoint image from all directions of an object by one shooting by putting an object in a cylindrical mirror and shooting the object from above with a camera. Can measure the three-dimensional shape of an object from a single image, so it can be used for content creation in educational fields such as electronic pictorial books for insects and small animals. , Can improve the description of the video. This 3D object measuring apparatus can be used not only in the education field, but also for various purposes such as the medical field and the academic research field.
  The three-dimensional object measurement apparatus of the present invention allows the user to easily measure the shape of personal belongings or to efficiently measure the entire circumference of many objects because of the simplicity of the device configuration and the photographing process. Useful for applications. In addition, since the entire shape of the object can be measured from a single image, it is possible to record the entire shape of the object along with the movement of the animal by continuously capturing images. Can be improved.
[Brief description of the drawings]
[0088]
FIG. 1 is a diagram schematically illustrating a basic configuration of a three-dimensional object measurement apparatus according to a first embodiment.
FIG. 2 is a diagram schematically showing light reception of a direct image and a reflected image in a vertical section of a three-dimensional object measuring apparatus.
FIG. 3 is a diagram schematically showing an image obtained by photographing an object having a cone shape placed on a pedestal with a camera;
FIG. 4 is a diagram schematically showing a state in which the photographing viewpoints of the direct image D0, the once reflected image R1, and the twice reflected image R2 with respect to the point A of the object are equivalently developed.
FIG. 5 is a view showing an image obtained by combining a direct image of a pyramid-shaped object placed at the center of a pedestal and a reflection image reflected on an inner wall mirror surface of a mirror cylindrical body;
FIG. 6 is a diagram showing that the geometric relationship between feature points is maintained even when the object is placed at a position deviated from the center of the pedestal.
FIG. 7 is a diagram showing an example of a locus of light rays that are reflected twice from a point of an object and incident on a camera.
FIG. 8 is a diagram showing a simulation image obtained by photographing a virtual object.
FIG. 9 is a relationship diagram between the angle φ and the distance d from the image center.
FIG. 10 is a photographed image obtained as a result of polar coordinate expansion.
FIG. 11 is a diagram showing search target areas;
FIG. 12 is a block diagram showing components that execute stereo measurement processing in the three-dimensional object measurement apparatus of the present invention.
FIG. 13 is a diagram schematically illustrating a basic configuration of a three-dimensional object measurement apparatus according to the second embodiment.
FIG. 14 is a diagram schematically illustrating a basic configuration of a three-dimensional object measurement apparatus according to a third embodiment.
FIG. 15 is a diagram schematically illustrating advantages and disadvantages when the distance between the pedestal and the fisheye lens is small.
FIG. 16 is a diagram schematically illustrating advantages and disadvantages when the distance between the pedestal and the fisheye lens is large.
FIG. 17 is a diagram schematically illustrating a basic configuration of a three-dimensional object measurement apparatus according to a fourth embodiment.
FIG. 18 is a diagram schematically showing an optical path of a direct image and a reflected image in a basic posture in a longitudinal section of a three-dimensional object measuring apparatus.
FIG. 19 is a view showing a state in which the mirror cylinder and the camera are integrally rotated 45 degrees clockwise in the vertical plane with the center of the pedestal as the center of rotational movement.
FIG. 20 is a diagram showing a real object having a conical shape used as a measurement target.
FIG. 21 is a photographed image of a real object having a conical shape used as a measurement target.
FIG. 22 is a diagram showing a cone shape measurement result using an actual measurement image;
FIG. 23 is a diagram showing the relationship between the distance from the center of the cone and the height.
[Explanation of symbols]
[0089]
  1 object
  10 pedestal
  20 mirror cylindrical body
  30, 30a, 30b camera
  31 Fisheye Lens
  32 Shooting and recording means
  40 Image data processing device
  41 Feature point extraction means
  42 Corresponding feature point search means
  43 Three-dimensional image data generation means
  100, 100a, 100b, 100c 3D object measuring device

Claims (4)

A pedestal on which a three-dimensional object to be imaged is placed; a pedestal that surrounds the pedestal with an inner wall surface of the cylinder; and a mirror cylinder that has the inner wall surface of the cylinder as a mirror surface; and a lens optical axis that faces the pedestal and has a lens optical axis A camera equipped with a fisheye lens arranged so as to coincide with the cylindrical central axis of the lens, and a photographing recording means for recording an image obtained through the fisheye lens, the photographing recording means of the camera, on the pedestal A three-dimensional object measurement device that captures both a direct image obtained by directly viewing a three-dimensional object and a reflected image of the three-dimensional object on the pedestal reflected on the mirror surface of the inner wall of the mirror cylinder,
Feature point extracting means for extracting and determining feature points on a photographed image photographed by the photographing recording means of the camera;
The corresponding feature point on the reflection image corresponding to the feature point on the direct image extracted by the feature point extraction unit is searched and determined on the reflection image, or extracted by the feature point extraction unit Means for searching and determining a corresponding feature point on the direct image corresponding to a feature point on the reflected image on the direct image, on an extension line connecting the center of the image and the feature point in the captured image A corresponding feature point searching means for searching for the corresponding feature point by searching for, and performing a stereo measurement process of the direct image and the reflected image based on a correspondence relationship between the feature point and the corresponding feature point, 3D object comprising 3D image data generating means for generating 3D image data of the 3D object from the direct image and the reflection image Measuring apparatus.   The pedestal is formed of a transparent material, the object is visible from below the pedestal, the camera includes two cameras, a first camera and a second camera, and the fisheye lens of the first camera and the The three-dimensional object measurement device according to claim 1, wherein the fish-eye lens of the second camera is disposed so as to face each other with the pedestal interposed therebetween.   The camera can move along the cylindrical central axis with respect to the mirror cylinder, and the distance between the pedestal and the fisheye lens is set to L, the height of the three-dimensional object to be imaged, and the base. The three-dimensional object measurement apparatus according to claim 1, wherein the three-dimensional object measurement apparatus is variable in a range of L to 10 L from the pedestal.   The mirror cylinder and the camera in a three-dimensional space with the center of the pedestal being the center of rotational movement in a state where the pedestal, the mirror cylinder and the camera can be moved relative to each other, and the pedestal is fixed. The three-dimensional object measuring device according to any one of claims 1 to 3, wherein the unit can freely rotate and move integrally.