JP6907566B2 - 3D shape measuring device and program - Google Patents

Description

The present invention relates to a technique for measuring a three-dimensional shape of an object.

Various tools have been proposed to acquire the three-dimensional shape of an object, but in recent years, due to improvements in camera resolution and computer computing performance, triangulation technology has been applied to create a large number of images. Techniques for generating shapes and surface textures have become widely used. The feature of this method is that it is possible to generate a shape and surface texture by inputting only a large number of images taken from multiple viewpoints, and due to improvements in image processing technology, it is more than a three-dimensional shape measuring device using a laser scanner. Stable measurement is becoming possible. An example of such a method is described in Patent Document 1.

Japanese Unexamined Patent Publication No. 8-254409

Since a sensor that acquires a three-dimensional shape in a non-contact manner uses light, it cannot measure an object surrounded by an object that refracts light, such as an aquarium. That is, even if the object in the aquarium is photographed with a camera, the image of the object is distorted by the refraction of light, so that the recent image-based three-dimensional shape digitization technology cannot be applied.

On the other hand, as one of the needs of digital archives, it is required to record the appearance of a deteriorating object at the time of shooting. Causes of deterioration include ultraviolet rays to paintings, damage caused by disasters and accidents, and the decay of living tissues. Currently, it is sealed and stored in an antiseptic solution typified by formalin as a preservation of living tissue.

As described above, in the digitization of the object stored in the container in a sealed state, the container cannot be opened for measurement, and therefore the measurement must be performed in the container in a sealed state. However, in the prior art, when measuring an object in a tank, the captured image is distorted by the refraction of light on the surface of the water tank, so that the relationship of triangulation is not satisfied and the three-dimensional shape is restored. There was a problem that it could not be done.

An object of the present invention is to provide a three-dimensional shape measuring device capable of removing distortion of an image caused by refraction and measuring a three-dimensional shape of an object.

From one aspect of the present invention, the three-dimensional shape measuring device for measuring the three-dimensional shape of the object housed in the transparent container to which the marker is attached is the transparent container photographed at a plurality of imaging positions. An image acquisition means for acquiring an image of the above, a shooting position specifying means for specifying the shooting position based on the marker portion included in the shot image, and the transparency based on the shooting position and the marker portion. A transparent surface specifying means for specifying the position of the transparent surface of the body container, and a generating means for generating an object image in which distortion caused by the transparent surface is removed based on the photographing position and the position of the transparent surface. A measuring means for measuring the three-dimensional shape of the object based on the image of the object from which the distortion has been removed is provided , and the transparent surface specifying means specifies the shape of the marker portion based on the photographing position. Then, the surface defined by the marker portion is determined to be the transparent surface .

The above-mentioned three-dimensional shape measuring device first acquires images of transparent containers photographed at a plurality of photographing positions, and identifies the photographing position based on a marker portion included in the photographed image. Next, the position of the transparent surface of the transparent container is specified based on the photographing position and the marker portion, and the object image from which the distortion caused by the transparent surface is removed is obtained based on the photographing position and the position of the transparent surface. Generate. Then, the three-dimensional shape of the object is measured based on the image of the object from which the distortion has been removed. Here, the transparent surface specifying means identifies the shape of the marker portion based on the photographing position, and determines the surface defined by the marker portion as the transparent surface. That is, a marker is attached to the transparent surface so that the shape of the transparent surface can be specified, and the transparent surface is determined based on the portion of the marker. This makes it possible to measure the three-dimensional shape of the object contained in the transparent container together with the liquid or the like.

In another aspect of the three-dimensional shape measuring device, the generation means is refracted on the transparent surface based on the imaging position, the position of the transparent surface, and the refractive index on the transparent surface. A means for recalculating the shooting position of the image from which the distortion has been removed, and an object image from which the distortion has been removed by converting the image shot at the shooting position into an image viewed from the recalculated shooting position. It is provided with a means for generating. In this aspect, an object image from which distortion has been removed is obtained by obtaining a shooting position of an image from which distortion due to refraction on a transparent surface has been removed.

Preferably, the marker is provided along the outer circumference of the transparent surface. In a preferred example, the marker is an opaque tape attached to the transparent container.

In another aspect of the invention, a program executed by a three-dimensional shape measuring device equipped with a computer and measuring the three-dimensional shape of an object housed in a transparent container to which a marker is attached may have a plurality of photographs. An image acquisition means for acquiring an image of the transparent container photographed at a position, a photographing position specifying means for specifying the photographing position based on a portion of the marker included in the photographed image, the photographing position, and the marker. An object image from which distortion caused by the transparent surface is removed based on a transparent surface specifying means for specifying the position of the transparent surface of the transparent body container, the photographing position, and the position of the transparent surface based on the portion. The computer is made to function as a generation means to be generated, a measurement means for measuring a three-dimensional shape of the object based on the object image from which the distortion has been removed, and the transparent surface specifying means is based on the shooting position. The shape of the marker portion is specified, and the surface defined by the marker portion is determined to be the transparent surface . By executing this program on a computer, the above-mentioned three-dimensional shape measuring device can be realized.

The configuration of the three-dimensional shape measuring apparatus according to the Example is shown. An example of a transparent container containing an object is shown. It is a flowchart of 3D shape measurement processing. It is a figure explaining the image processing in 3D shape measurement processing. It is another figure explaining the image processing in 3D shape measurement processing. It is a figure explaining the method of removing the distortion caused by refraction.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Constitution]
FIG. 1 shows the configuration of the three-dimensional shape measuring device according to the embodiment. The three-dimensional shape measuring device (hereinafter, simply referred to as “measuring device”) 20 is a device that measures the three-dimensional shape of the object 12 housed in the transparent container 10 and acquires a texture. The measuring device 20 can be configured by, for example, a computer device that executes a program prepared in advance.

The measuring device 20 is connected to two cameras 30a and 30b installed at different positions. The cameras 30a and 30b take images of the transparent container 10, respectively, and supply the photographed images Ia and Ib to the measuring device 20. The measuring device 20 performs image processing on the captured images Ia and Ib acquired from the cameras 30a and 30b, and measures the three-dimensional shape of the object 12 in the transparent container 10. Although two cameras 30a and 30b are shown in FIG. 1 for convenience of explanation, in reality, a plurality of captured images taken at different shooting positions by one or a plurality of cameras 30 are supplied to the measuring device 20. do it. For example, a system configuration may be configured in which one camera is fixedly arranged, the transparent container 10 is arranged on the turntable, and the turntable is rotated to take a plurality of captured images.

In the above configuration, the cameras 30a and 30b are examples of the image acquisition means in the present invention, and the measuring device 20 is an example of the photographing position specifying means, the transparent surface specifying means, the generating means and the measuring means in the present invention.

FIG. 2 shows the configuration of the transparent container 10. The transparent container 10 is, for example, a 10 cm square cubic glass cube filled with water and an object 12. The transparent container 10 may be made of transparent acrylic or the like instead of glass. In this embodiment, the object 12 is a spherical ball. The object 12 is located approximately in the center of the transparent container 10, but in FIG. 2, the object 12 is located slightly below the center due to refraction on the glass surface 14 constituting the transparent container 10. It looks like it is.

For example, an opaque masking tape 13 having a width of 1 cm is attached to all sides of the transparent container 10. The masking tape 13 is provided to specify the glass surface 14 of the transparent container 10, and is an example of a marker in the present invention.

[Three-dimensional shape measurement method]
Next, a method of measuring the three-dimensional shape of the object 12 will be described. FIG. 3 is a flowchart of the three-dimensional shape measurement process. This process is executed by the measuring device 20.

First, the measuring device 20 acquires captured images from a plurality of camera positions (step S10). Specifically, the measuring device 20 acquires captured images Ia and Ib from the two cameras 30a and 30b.

Next, the measuring device 20 calculates the camera position from the pixels of the portion (hereinafter, referred to as “masking tape portion”) to which the masking tape 13 is attached in the captured image (step S11). FIG. 4A shows an example of images Ia and Ib taken by the two cameras 30a and 30b. Due to the relative positional relationship between the transparent container 10 and the cameras 30a and 30b, the positions of the objects 12 are slightly shifted to the left and right in the images Ia and Ib. First, the measuring device 20 extracts only the pixels of the masking tape portion from the images Ia and Ib. FIG. 4B shows an example of an image in which only the pixels of the masking tape portion are taken out. Image Ia1 is obtained by extracting only the pixels of the masking tape portion from the image Ia, and image Ib1 is obtained by extracting only the pixels of the masking tape portion from the image Ib. Then, the measuring device 20 uses a known technique for restoring three dimensions from a plurality of images, and the positions of the cameras 30a and 30b that have taken these two images from the images Ia1 and Ib1 and the masking tape portion. The shape is calculated. FIG. 5A schematically shows the positions of the obtained cameras 30a and 30b and the shape of the masking tape portion in space. For this process, for example, commercially available software "PhotoScan" or the like can be used.

Next, the measuring device 20 utilizes the two camera positions and the shape of the masking tape portion, performs plane fitting on the assumption that the shapes of the masking tape portions are the same plane, and performs flat fitting, and the glass defined by the masking tape portion. The position of the surface 14 is calculated (step S12). Specifically, the measuring device 20 calculates the least squares approximate plane with respect to the point cloud forming the shape of the masking tape portion, and obtains the position of the glass surface 14. FIG. 5B schematically shows the position of the obtained glass surface 14.

In this way, when the position of the glass surface 14 is obtained, the positions of the two cameras and the position of the glass surface 14 are determined, so that the positions and orientations of the cameras 30a and 30b with respect to the glass surface 14 are determined. Therefore, the measuring device 20 recalculates the camera position for each of the images Ia and Ib obtained from the cameras 30a and 30b from the pixels of the object 12 from which the distortion of the image is removed in the region other than the masking tape portion ( Step S13). Hereinafter, processing relating to one camera position will be described.

FIG. 6 is a plan view of the camera position Po and the glass surface 14 as viewed from above. When the glass surface 14 is photographed from the camera position Po, the photographed image is obtained as a set of a plurality of pixels on the imaging plane So. Because the ray (light ray) Ro emitted from the camera position Po and passing through the position of each pixel on the imaging plane So reaches the glass surface 14, and the inside of the glass surface 14 (inside of the transparent container 10) is water. Refracted on the glass surface 14. That is, the ray Ro enters the glass surface 14 at the incident angle α, is refracted at the glass surface 14, becomes the ray Ro ′ emitted at the exit angle β, and travels inside the glass surface 14 to reach the object 12. Here, when the line segment Rx is created by extending the ray Ro'inside the glass surface 14 in the direction opposite to the traveling direction, a plurality of rays corresponding to the plurality of rays Ro that have passed the positions of each pixel on the imaging plane So. The line segment Rx of is gathered at the imaging position Px. This imaging position Px corresponds to a camera position where the same image as the captured image can be obtained when refraction does not occur on the glass surface 14. In other words, the imaging position Px is a camera position recalculated from the image of the object 12 (hereinafter, referred to as “object image”) after removing the distortion due to refraction on the glass surface 14.

Specifically, since the camera position Po and the position and orientation of the glass surface 14 are known in step S12, the measuring device 20 obtains the direction of the ray Rx passing through the position of each pixel on the imaging plane So. , The intersection and the intersection angle (= incident angle α) between the ray Rx and the glass surface 14 are obtained. Since the medium inside the glass surface 14 is water, the refractive index on the glass surface 14 is known. Therefore, the measuring device 20 uses the refractive index on the glass surface 14 to determine the emission angle β from the incident angle α. Find and find the direction of the line segment Rx. The measuring device 20 performs this process on all the pixels on the photographing plane So, and obtains the intersection of the obtained plurality of line segments Rx as the imaging position Px. In this way, the measuring device 20 obtains the recalculated camera position (imaging position) Px.

When the camera position Px recalculated in this way is obtained, the measuring device 20 generates an object image in which distortion due to refraction on the glass surface 14 is removed (step S14). Specifically, the measuring device 20 determines the imaging plane So based on the positional relationship between the camera position Po and the recalculated camera position Px and the positional relationship between the camera position Po and each pixel on the imaging plane So. Each of the above pixels is moved onto the imaging plane Sx corresponding to the recalculated camera position Px to generate an object image on the imaging plane Sx. At this time, the measuring device 20 generates an object image for the captured image excluding the masking tape portion.

When the above processing is performed for the plurality of camera positions and a plurality of object images in which the distortion due to refraction on the glass surface 14 is removed are obtained, the measuring device 20 obtains the object from the obtained plurality of object images. Twelve three-dimensional shapes and textures are generated (step S15). This process can be performed using a known technique for restoring three dimensions from the plurality of images used in step S11 described above.

As described above, in the three-dimensional shape measurement process of the present embodiment, an object image in which distortion due to refraction on the glass surface 14 of the transparent container 10 is removed is generated, and the three-dimensional shape and texture of the object 12 are obtained. be able to. As a result, it is possible to obtain the three-dimensional shape and texture of an object enclosed with a liquid in a transparent case such as a glass case or an acrylic case without taking it out of the case. For example, it is possible to obtain a three-dimensional shape and texture of an object such as a formalin-pickled sample stored in a museum or a fish and shellfish in an aquarium tank.

[Modification example]
In the above embodiment, the masking tape 13 is attached to all sides of the glass surface 14 (four sides of one glass surface) as a marker for specifying the glass surface 14, but the method of adding the marker is to this. Is not limited. For example, if the glass surface is a flat surface, in order to identify one glass surface, if markers are added to at least three places at positions that do not overlap with the object 12, one glass surface is assigned based on those markers. Can be identified.

10 Transparent container 12 Object 13 Masking tape 14 Glass surface 20 Three-dimensional shape measuring device 30a, 30b Camera

Claims (5)

A three-dimensional shape measuring device that measures the three-dimensional shape of an object housed in a transparent container to which a marker is attached.
An image acquisition means for acquiring an image of the transparent container taken at a plurality of imaging positions, and an image acquisition means.
A shooting position specifying means for specifying the shooting position based on the marker portion included in the shot image, and
A transparent surface specifying means for specifying the position of the transparent surface of the transparent container based on the photographing position and the marker portion.
A generation means for generating an object image from which distortion caused by the transparent surface is removed based on the shooting position and the position of the transparent surface.
A measuring means for measuring the three-dimensional shape of the object based on the image of the object from which the distortion has been removed, and
Equipped with a,
The transparent surface specifying means is a three-dimensional shape measuring device, characterized in that the shape of the marker portion is specified based on the photographing position, and the surface defined by the marker portion is determined as the transparent surface. The generation means
A means for recalculating the shooting position of an image from which distortion due to refraction on the transparent surface has been removed, based on the shooting position, the position of the transparent surface, and the refractive index on the transparent surface.
A means for generating an object image from which the distortion is removed by converting an image taken at the shooting position into an image viewed from the recalculated shooting position.
The three-dimensional shape measuring device according to claim 1, wherein the three-dimensional shape measuring device is provided. The three-dimensional shape measuring device according to claim 1 or 2 , wherein the marker is provided along the outer periphery of the transparent surface. The three-dimensional shape measuring device according to any one of claims 1 to 3 , wherein the marker is an opaque tape attached to the transparent container. A program executed by a three-dimensional shape measuring device equipped with a computer and measuring the three-dimensional shape of an object housed in a transparent container to which a marker is attached.
An image acquisition means for acquiring an image of the transparent container taken at a plurality of imaging positions,
A shooting position specifying means for specifying the shooting position based on the marker portion included in the shot image.
A transparent surface identifying means for specifying the position of the transparent surface of the transparent container based on the photographing position and the marker portion.
A generation means for generating an object image from which distortion caused by the transparent surface is removed based on the shooting position and the position of the transparent surface.
A measuring means for measuring the three-dimensional shape of the object based on the image of the object from which the distortion has been removed.
It makes the computer function as,
The transparent surface specifying means is a program characterized in that the shape of the marker portion is specified based on the photographing position, and the surface defined by the marker portion is determined as the transparent surface.

Images