JP2021185372A - Device and method for shape measurement - Google Patents

Abstract

To measure the shape of an object with high accuracy by using the luminance of the object observed at two wavelengths via a medium.SOLUTION: A shape measuring device includes: a light source 80 which radiates light on a surface of an object 60 having reflectance coefficients with respect to first and second wavelengths, via a medium 10 having absorption coefficients at the first and second wavelengths; a sensor 21 which measures the light of each wavelength that has come from the surface of the object and has propagated through the medium 10; and a measuring unit which calculates a distance from the surface of the medium 10 to the surface of the object 60 on the basis of the measured intensities at each wavelength and the absorption coefficients at each wavelength, and measures the shape of the object 60 on the basis of the calculated distance. Each of the wavelength is selected such that a difference between the intensity of a minimum distance at the first wavelength and the intensity of a maximum distance at the second wavelength is maximized, a difference between the reflectance coefficient at the first wavelength and the reflectance coefficient at the second wavelength is minimized, and a difference between the absorption coefficient at the first wavelength and the absorption coefficient at the second wavelength is maximized.SELECTED DRAWING: Figure 1A

Description

The present invention relates to an optical shape measuring device and a method for measuring the shape of an object to be measured.

Patent Document 1 discloses a three-dimensional imaging system using absorption. In the 3D imaging system, the fluorescent medium between the object and the sensor is excited, and the light beam of two wavelengths is emitted using the excitation light from the excitation light source. In the 3D imaging system, the sensor is reflected by the object. Receives the light beam. The computer connected to the sensor determines the thickness of the medium based on the ratio of the intensity of the first wavelength to the intensity of the second wavelength, and determines a plurality of additional thicknesses within the region of the object. Then, the three-dimensional image of the area of the object is reconstructed.

Further, Patent Document 2 discloses a shape measuring device that accurately measures the surface shape of an object in a short time. Here, the shape measuring device includes an illumination unit that illuminates the subject through a medium, an optical sensor for receiving light beams of two wavelengths reflected by the object, and light intensity of the two wavelengths. It is characterized by having a measuring unit for measuring the distance to the surface of the object to be measured based on the transmittance of the medium and the transmittance of the medium.

U.S. Pat. No. 8,107,806 Japanese Unexamined Patent Publication No. 2008-256504

However, in the three-dimensional imaging system disclosed in Patent Document 1 described above, a fluorescent medium is provided between the object and the sensor in order to excite light beams having two wavelengths in response to the excitation light from the excitation light source. There was a limitation that it had to be provided.

Further, in the shape measuring device disclosed in Patent Document 2, two wavelengths having the same reflectance coefficient of an object are selected, and the reflectance coefficient of an object needs to be known before measurement.

An object of the present invention is to provide a shape measuring device and a method for measuring the shape of an object with higher accuracy than the prior art by using the brightness of the object observed at two wavelengths through a medium. ..

Another object of the present invention is to provide a shape measuring device and a method for measuring the shape of an object with higher accuracy while avoiding the limitation of known reflection characteristics in the prior art by using the luminance of an object having three wavelengths. To do.

The shape measuring device according to the first invention is
An object having reflectance coefficients of the first wavelength and a second wavelength via a medium having an absorption coefficient at each first wavelength and an absorption coefficient at a second wavelength longer than the first wavelength. A light source that irradiates the surface with light having a first wavelength and a second wavelength,
A sensor that receives light coming from the surface of the object and propagating through the medium, and measures the intensities of the first wavelength and the second wavelength.
The distance from the surface of the medium to the surface of the object based on the measured intensities of the first and second wavelengths and the absorption coefficients of the medium at the first and second wavelengths. A shape measuring device including a measuring unit that calculates and measures the shape of the object based on the calculated distance.
In the first wavelength and the second wavelength, the difference between the intensity of the shortest distance at the first wavelength and the intensity of the longest distance at the second wavelength becomes larger than the predetermined first value, and the first wavelength is used. The difference between the reflectance coefficient at the wavelength and the reflectance coefficient at the second wavelength is selected to be smaller than a predetermined second value.
Here, the shortest distance means the shortest distance among a plurality of corresponding distances at a plurality of positions on the surface of the object.
The longest distance is characterized by referring to the longest of the corresponding distances at a plurality of positions on the surface of the object.

The shape measuring device according to the second invention is
A medium in which the third wavelength is between the first wavelength and the second wavelength, which is longer than the first wavelength, and has absorption coefficients at the first wavelength, the second wavelength, and the third wavelength. A light source that irradiates the surface of an object having a reflectance coefficient at each of the first wavelength, the second wavelength, and the third wavelength through the light source.
A sensor that receives light coming from the surface of the object and propagating through the medium, and measures the intensities of the first wavelength, the second wavelength, and the third wavelength.
The distance from the surface of the medium to the surface of the object is calculated based on the intensities of the first wavelength and the second wavelength and the absorption coefficients of the medium at the first and second wavelengths, and the calculation is performed. Under the condition that the reflection coefficient characteristics of the surface of the object have a substantially linear relationship over the wavelength range from the first wavelength to the second wavelength via the third wavelength based on the distance. It is characterized by having a measuring unit for measuring the shape of the object.

Therefore, according to the present invention, the shape can be measured using light of two wavelengths arriving from an object with higher accuracy than the prior art. In addition, it is possible to measure the shape of an object by using light of three wavelengths coming from the object with higher accuracy than the prior art.

These and other objects and features of the invention will become apparent from the following description relating to the embodiments thereof, with reference to the accompanying drawings.

It is a schematic block diagram which shows the structure of the optical shape measuring apparatus which concerns on 1st basic embodiment of this invention. It is a flowchart which shows the shape measurement process executed by the measuring apparatus 70 of FIG. 1A. It is a schematic block diagram which shows the structure of the optical shape measuring apparatus which concerns on the 2nd basic embodiment of this invention. 2 is a flowchart showing a shape measurement process executed by the measuring device 70A of FIG. 2A. It is a schematic block diagram which shows the structure of the optical shape measuring apparatus which concerns on 3rd basic embodiment of this invention. Correction to correct the distance l from the medium surface 10s to the object surface 60s when the incident light L80 and the reflected light R60 are inclined at angles φ and θ with respect to the direction perpendicular to the surface (water surface) of the medium 10. It is a schematic block diagram which shows the method. It is a graph which shows the absorption ratio of light by water in 400 nm to 1400 nm. It is a photographic image showing the difference in the appearance of water between visible light and near infrared light, and is a photographic image showing the appearance of water taken by an infrared camera without using a filter. It is a photographic image showing the difference in the appearance of water between visible light and near infrared light, and is a photographic image showing the appearance of water taken by an infrared camera through a 950 nm filter. It is a schematic side view of the apparatus for showing the amount of light according to Lambert-Beer's law. For example, it is a schematic side view of a shape measuring apparatus when the optical axes of the sensors 21 and 22 which are cameras and the light source 80 are parallel and have a positional relationship perpendicular to the water surface. It is a graph which shows the relationship between the relative reflectance error and the estimated distance error. It is a graph which shows the reflection spectrum of "24 color checker board". It is a graph which shows the reflection spectrum error of the "color checker board" in one wavelength pair. It is a spectrum diagram which shows the reflection spectrum of "wood". It is a spectrum diagram which shows the reflection spectrum of "cloth". It is a spectrum diagram which shows the reflection spectrum of "leather". It is a spectrum diagram which shows the reflection spectrum of "metal". It is a graph which shows the reflection spectrum error of "wood color checker board" in one wavelength pair. It is a graph which shows the reflection spectrum error of the "cloth color checker board" in one wavelength pair. It is a graph which shows the reflection spectrum error of "leather color checker board" in one wavelength pair. It is a graph which shows the reflection spectrum error of the "metal color checker board" in one wavelength pair. It is a schematic side view of the shape measuring apparatus when the optical axes of the sensors 21 and 22 and the light source 80 are not parallel and have a positional relationship not perpendicular to the water surface. It is a photographic image which shows the appearance of the coaxial spectral imaging system (shape measuring apparatus) which concerns on 1st Embodiment of this invention. FIG. 3 is a spectral diagram showing a response function of the camera of FIG. 15 (manufactured by POINT-GREY, GS3-U3-41C6NIR type) having a wavelength from 300 nm to 1100 nm. It is a spectrum diagram which shows the transmission function of the two bandpass filters of FIG. 15, and the response function of a camera. It is a spectrum diagram which shows the spectrum of the light source of FIG. It is a spectral diagram which shows the absorption coefficient of water. It is a graph which shows the distance accuracy in "cyan-colored tile", and is the graph which shows the comparison between the estimated distance and the true value. It is a graph showing the distance accuracy in the "cyan-colored tile", and is a graph showing the relative error of the estimated distance. It is a graph showing the distance accuracy in the "red plastic board", and is a graph showing the comparison between the estimated distance and the true value. It is a graph showing the distance accuracy in the "red plastic board", and is a graph showing the relative error of the estimated distance. It is a graph which shows the distance accuracy in "white marble", and is the graph which shows the comparison between the estimated distance and the true value. It is a graph which shows the distance accuracy in "white marble", and is the graph which shows the relative error of the estimated distance. It is a graph which shows the distance accuracy in "black marble", and is the graph which shows the comparison between the estimated distance and the true value. It is a graph which shows the distance accuracy in "black marble", and is the graph which shows the relative error of the estimated distance. It is a photographic image of a "shellfish" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of a "shellfish" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the "shellfish" distance image (after black-and-white conversion) estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the "shellfish" estimated shape image estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the "shellfish" RGB image (after black-and-white conversion) estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of a "stone" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of a "stone" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "stone" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "stone" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "stone" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of "a squirrel object made of wood" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of "a squirrel object made of wood" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "squirrel object made of wood" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "squirrel object made of wood" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "squirrel object made of wood" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of a "monkey object made of pottery" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. It is a photographic image of "a monkey object made of pottery" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of "a monkey object made of pottery" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "monkey object made of pottery" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "monkey object made of pottery" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of a "cup object having a colorful color" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of a "cup object having a colorful color" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "object of a cup having a colorful color" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "object of a cup having a colorful color" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "object of a cup having a colorful color" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of a "paper cup object having a texture" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. It is a photographic image of a "paper cup object having a texture" taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "paper cup object having a texture" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of "the object of the paper cup having a texture" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "paper cup object having a texture" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of a "cherry blossom object" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. It is a photographic image of a "cherry blossom object" taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "cherry blossom object" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "cherry flower object" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "cherry blossom object" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of "another cherry blossom object" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of "another cherry blossom object" taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of "another cherry blossom object" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of "another cherry blossom object" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of "another cherry blossom object" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of "an object which is translucent and has a color gradation" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of "an object which is translucent and has a color gradation" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "object which is translucent and has a color gradation" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "object which is translucent and has a color gradation" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of "the object which is translucent and has a color gradation" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of "a semi-transparent heart-shaped object with specular reflection" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. It is a photographic image of "a semi-transparent heart-shaped object with specular reflection" taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. It is a photographic image which shows the distance image (after black-and-white conversion) of the "semi-transparent heart-shaped object by specular reflection" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the estimated shape image of the "semi-transparent heart-shaped object by specular reflection" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image which shows the RGB image (after black-and-white conversion) of the "semi-transparent heart-shaped object by specular reflection" estimated by the shape measuring apparatus which concerns on 1st Embodiment. It is a photographic image of a "moving hand (first image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. In contrast to the "moving hand (first image)" in FIG. 34A, the photographic image of the "moving hand (first image)" whose shape is estimated by the shape measuring device according to the first embodiment. be. It is a photographic image of a "moving hand (second image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. It is a photographic image of the "moving hand (second image)" whose shape is estimated by the shape measuring device according to the first embodiment with respect to the moving hand (second image) of FIG. 34C. It is a photographic image of a "moving hand (third image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. In contrast to the "moving hand (third image)" in FIG. 34E, the photographic image of the "moving hand (third image)" whose shape is estimated by the shape measuring device according to the first embodiment. be. It is a photographic image of a "moving hand (fourth image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. In contrast to the "moving hand (fourth image)" in FIG. 34G, the photographic image of the "moving hand (fourth image)" whose shape is estimated by the shape measuring device according to the first embodiment. be. It is a photographic image of a "goldfish (first image)" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of the "goldfish (first image)" whose shape is estimated by the shape measuring device according to the first embodiment with respect to the "goldfish (first image)" of FIG. 35A. It is a photographic image of a "goldfish (second image)" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of the "goldfish (second image)" whose shape is estimated by the shape measuring device according to the first embodiment with respect to the "goldfish (second image)" of FIG. 35C. It is a photographic image of a "goldfish (third image)" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of the "goldfish (third image)" whose shape is estimated by the shape measuring device according to the first embodiment with respect to the "goldfish (third image)" of FIG. 35E. It is a photographic image of a "goldfish (fourth image)" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. It is a photographic image of the "goldfish (fourth image)" whose shape is estimated by the shape measuring device according to the first embodiment with respect to the "goldfish (fourth image)" of FIG. 35G. It is a photographic image of an RGB image (after black-and-white conversion) of "an object in which the surface of an egg-shaped object having a transparent upper part is not painted". It is a photographic image of the “object without paint” whose shape is estimated by the shape measuring device according to the prior art with respect to the “object without paint” in FIG. 36A. It is a photographic image of an RGB image (after black-and-white conversion) of "an object in which the surface of an egg-shaped object having a transparent upper part is painted". It is a photographic image of the "object with paint" whose shape is estimated by the shape measuring device according to the first embodiment with respect to the "object with paint" in FIG. 36C. It is a photographic image of the “object without paint” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “object without paint” in FIG. 36A. It is a graph which shows the linear approximation method of the reflected radiation spectrum in the wavelength range of _{[λ 1} , λ ₂ ] used in the shape measuring apparatus which concerns on 2nd Embodiment of this invention. It is a spectrum diagram which shows the result of having performed the linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to “wood”. It is a spectrum diagram which shows the result of having performed the linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to “cloth”. It is a spectrum diagram which shows the result of having performed the linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to "leather". It is a spectrum diagram which shows the result of having performed the linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to “metal”. It is a top view which shows the structural example of the shape measuring apparatus (corresponding to the 3rd basic embodiment) when the optical axis of a light source 80 (parallel light) and the optical axis of a spectroscope sensor 24 are coaxial. For example, it is a top view which shows the configuration example of the camera system of the same angle of view by three cameras 21C, 22C, 23C used in the shape measuring apparatus which concerns on 2nd basic embodiment. It is a photographic image which shows the input image of the wavelength 905 nm about the "egg-shaped object" for the shape estimation by the shape measuring apparatus which concerns on 2nd Embodiment. It is a photographic image which shows the estimation result image of the shape estimation by the shape measuring apparatus which concerns on 2nd Embodiment with respect to the "egg-shaped object" of FIG. 41A. It is a photographic image which shows the input image of the wavelength 905 nm about the "food sample of a roll cake" for the shape estimation by the shape measuring apparatus which concerns on 2nd Embodiment. It is a photographic image which shows the estimation result image of the shape estimation by the shape measuring apparatus which concerns on 2nd Embodiment with respect to the "food sample of a roll cake" of FIG. 42A. It is a photographic image which shows the input image of the wavelength 905 nm about the "turnip object" for the shape estimation by the shape measuring apparatus which concerns on 2nd Embodiment. It is a photographic image which shows the estimation result image of the shape estimation by the shape measuring apparatus which concerns on 2nd Embodiment with respect to the "Turnip object" of FIG. 43A.

Hereinafter, the basic embodiment will be described with reference to the drawings. In the following basic embodiments, similar components are designated by the same reference numerals.

1st Basic Embodiment FIG. 1A is a schematic block diagram showing a configuration of an optical shape measuring device according to a first basic embodiment of the present invention. Referring to FIG. 1A, the state measuring device according to the first basic embodiment measures the shape of the object 60, and includes a tank 50 including a medium 10, a light source 80, a half mirror 40, and a bandpass filter. It includes 31, 32, sensors 21 and 22, such as an image pickup camera, a measuring device 70 having an internal memory of 70 m and consisting of a digital computer, an operation unit 71, and a display 72. Here, the on / off of the operation of the light source 80 is controlled by the control signal S80 from the measuring device 70.

As shown in FIG. 1A, the inside of the tank 50 is filled with a medium 10 which is at least one of, for example, water, liquid, gas, solid, and gel, and has an object surface 60s (hereinafter, an object). 60 is placed, and the object 60 is parallel to, for example, the bottom surface of the tank 50 by the moving support portion 61 based on the control signal S61 from the measuring device 70 at the time of shape measurement and three-dimensional imaging. It is moved while being supported on a two-dimensional plane. The operation of the moving support portion 61 is the same in the basic embodiment and the embodiment described later.

Medium 10 has an absorption coefficient characteristic for the wavelength, for example, the absorption coefficient at the first wavelength lambda ₁ alpha (lambda _1), the absorption coefficient at a second wavelength different from lambda ₂ and the first wavelength lambda ₁ alpha It has (λ ₂ ) and. The object 60 is arranged on the bottom surface of the tank 50 as described above, and the object surface 60s, which is a target surface such as the upper surface of the object 60, has a reflectance characteristic with respect to a wavelength, for example, the first wavelength λ _1. It has a reflectance coefficient s (λ ₁ ) at the second wavelength λ ₂ and a reflectance coefficient s (λ ₂ ) at the second wavelength λ 2. The light source 80 emits wide-band parallel light including the first and second wavelengths λ ₁ and λ _2, and irradiates the object surface 60s through the medium 10 as incident light L80. Here, the arrow of the incident light L80 indicates the direction in which the incident light L80 propagates.

The incident light L80 is substantially reflected along the optical axis 40a as reflected light R60 substantially parallel to the incident light L80 by the object surface 60s of the object 60. Here, the arrow of the reflected light R60 indicates the direction in which the reflected light R60 propagates. Of the reflected light R60, the light _{having a passing wavelength of the first wavelength λ 1} propagates to the sensor 21 via the medium 10, the half mirror 40, and the bandpass filter 31. The half mirror 40 reflects the reflected light R60 and propagates the light toward the sensor 22 via the bandpass filter 32. Here, since the bandpass filter 32 is inserted, the light of the second wavelength λ ₂ of the reflected light R60 is incident on the sensor 22. The sensors 21 and 22 can be configured by, for example, an image pickup camera. Here, the sensor 21 _{detects the intensity of the reflected light R60 having the first} wavelength λ 1 of the reflected light R60, and outputs the intensity data to the measuring device 70. Further, the sensor 22 _{detects the intensity of the reflected light R60 having the second} wavelength λ 2 of the reflected light R60, and outputs the data of the intensity to the measuring device 70.

The operation unit 71 is connected to the measuring device 70 and is provided for inputting data and instruction commands necessary for performing shape measurement processing by the user. The display 72 is operably connected to the measuring device 70 and is provided for displaying the result of the shape measuring process.

The measuring device 70 has an internal memory 70 m in which the following data is input and stored in advance using, for example, the operation unit 71: the absorption coefficient characteristic of the medium 10 with respect to the wavelength, for example, the first wavelength λ _1. Data of the absorption coefficient α (λ ₁ ) in the first wavelength λ ₁ and the absorption coefficient α (λ ₂ ) at the second wavelength λ _{2 different from the first wavelength λ 1.} The measuring device 70 estimates and measures the three-dimensional shape of the object surface 60s of the object 60 by executing the shape measuring process of FIG. 1A described later.

The measuring device 70 is conceptually similar to the invention disclosed in Patent Document 2, and has a plurality of locations on the object surface 60s of the object 60 based on the respective intensities of _{the first and second wavelengths λ 1} and λ _2. By calculating the distance l (meaning the distance from the surface of the medium 10s to the surface of the object 60s), the one-dimensional, two-dimensional, or three-dimensional shape of the object is measured. The shape measuring device can further reconstruct a three-dimensional image of the object based on the measured shape of the object to perform three-dimensional imaging of the object 60.

In the first basic embodiment, the distance l is calculated with a higher accuracy than the device accuracy of Patent Document 2 by using the following method different from the shape measuring device described in Patent Document 2, and the object 60 is calculated. The shape of the can be measured. That is, the method of the first basic embodiment is significantly different from the shape measuring device of Patent Document 2 in that a three-dimensional image of the object 60 can be reconstructed even if the translucent object 60 is provided. ..

First, first, s (λ) is the reflected radiation spectrum of the object surface 60s of the object 60 (in the case of a transparent object, some light enters and exits the inside of the object, so the light arriving from the object surface is the reflection spectrum. And at least one of the radiation spectrum, hereinafter referred to as "reflection radiation spectrum"; however, when it is clearly not a transparent object, it is defined as "reflection spectrum"), that is, the present embodiment is defined as. It is not limited to the reflective surface. Defining s (λ) as the ratio of radiance at a surface point to incident irradiance at a particular wavelength λ, the same theory has more complex materials such as translucency, subsurface scattering, volume scattering, and interreflection. Applicable to objects. In other words, the first basic embodiment can perform three-dimensional imaging on the translucent object 60.

Next, wavelength selection for accurate estimation will be described below. For accurate shape reconstruction, it is important to properly select the _{two wavelengths λ 1} and λ 2 for shape measurement. The main factors that affect the shape measurement accuracy are as follows.

(1) Spectral sensitivity of sensors 21 and 22
(2) Difference between the absorption coefficient α (λ ₁ ) and the absorption coefficient α (λ ₂ ) of the medium 10,
_{(3) The difference between the reflectance coefficient s (λ 1} ) and the reflectance coefficient s (λ ₂ ) of the object surface 60s of the object 60, and (4) the size of the object 60 such as the thickness of the object 60.

The conditions to be met for wavelength selection can be summarized as follows.
(A) In order to make the best use of the dynamic range of sensors 21 and 22 such as a camera, it is necessary to select _{two wavelengths λ 1} and λ 2 under the following conditions.
Condition: The difference between exp (-2α (λ ₂ ) −α (λ ₁ ) l _min ) and exp (-2α (λ ₂ ) −α (λ ₁ ) l _max ) is from the predetermined threshold value D _th1 . It is preferable that the difference is maximized so that the difference is also large. "
Here, l _max represents the longest distance among the plurality of corresponding distances l at a plurality of positions on the object surface 60s measured by the shape measuring device, and l _min represents the shortest distance thereof. In the embodiment, water is used as the medium 10, but the medium 10 is not limited to water.

The above condition is that when λ ₂ > λ ₁ , the intensity of the pixel at the point (l _{1 ) at the distance l 1} on the object surface 60s _{of the λ 1} image, which is the image of the first wavelength λ ₁ , should be as high as possible. It means that the intensity at the pixel of the point (l _{2 ) at the distance l 2} on the object surface 60s _{of the λ 2} image which is the image of the second wavelength λ _{2 should be as small as possible.} In this case, the difference between the intensity at the shortest distance l _{min at the first wavelength λ 1} and the intensity at the longest distance l _{max at the second wavelength λ 2} is preferably larger than the _{threshold value D th1.} The difference is the maximum. Here, the shortest distance l _min corresponds to the position having the minimum depth on the object surface 60s of the object 60, and the longest distance l _max corresponds to the position having the maximum depth on the object surface 60s of the object 60. do.

In our experiments, the ratio of the intensity at the shortest distance l _{min at the first wavelength λ 1 to} the intensity at the longest distance l _{max at} the second wavelength λ ₂ is greater than about 4 or 5. preferable.

(B) In order to minimize the difference between the reflectance coefficients s (λ ₁ ) and s (λ ₂ ) at the first and second wavelengths λ ₁ and λ ₂ and improve the accuracy, a plurality of objects 60 are used. Based on the statistical data of the reflectance coefficient _{, the difference between the reflectance coefficients s (λ 1} ) and s (λ ₂ ) is as small as possible, preferably smaller _{than the predetermined threshold D th2} , so that the two wavelengths λ Select ₁ , λ _2.

(C) In order to reduce noise at the outputs of the sensors 21 and 22, the spectral sensitivity and intensity of the sensors 21 and 22 at _{the first and second wavelengths λ 1} and λ _{2 are sufficiently high, and a predetermined threshold value is obtained.} It is preferably larger than D _th3.

FIG. 1B is a flowchart showing a shape measurement process executed by the measuring device 70 of FIG. 1A.

In FIG. 1B, first, in step S1, the _{intensities of the first wavelength λ 1} and the second wavelength λ ₂ measured by the sensors 21 and 22 are received from the sensors 21 and 22 and 23. Next, in step S2, the received _{intensities of the first wavelength λ 1} and the second wavelength λ ₂ and the absorption coefficient α of the medium 10 at the _first wavelength λ 1 stored in advance in the internal memory 70 m ( The distance l from the medium surface 10s to the object surface 60s is calculated based on the absorption coefficient α (λ _{2 ) of the medium 10 at the second} wavelength λ 2) and λ _1). Then, in step S3, the shape of the object surface 60s of the object 60 is measured based on the calculated distance l, and in step S4, the shape of the object 60 is three-dimensionally measured based on the shape of the measured object surface 60s of the object 60. Reconstruct an image with a shape. Further, in step S5, an image having a three-dimensional shape of the reconstructed object 60 is displayed on the display 72, and the shape measurement process is completed.

FIG. 4 is a schematic block diagram showing a correction method for correcting the distance l. When the incident light L80 and the reflected light R60 are inclined at angles φ and θ with respect to the surface of the medium 10 (the medium surface perpendicular to the optical axis 40a), the distance l is as shown in FIG. It should be corrected to (l _m1 / cosφ + l _m2 / cosθ) / 2. Here, l _m1 and l _m2 indicate the actually measured lengths (distances) of the incident light L80 and the reflected light R60, respectively. The correction method can be applied to the second and third basic embodiments described later.

As described above, according to the first basic embodiment, two wavelengths λ ₁ and λ ₂ are selected as described above, and the shape measuring device measures the shape of the object 60 with higher accuracy than the prior art. can do.

2nd Basic Embodiment FIG. 2A is a schematic block diagram showing a configuration of an optical shape measuring device according to a second basic embodiment of the present invention. Referring to FIG. 2A, the shape measuring apparatus according to the second basic embodiment uses _{three wavelengths λ 1} , λ ₂ and λ ₃ which are different from each other of _{λ 1} <λ ₃ <λ ₂ and thus the first basic. It is characterized in that the shape of the object 60 is measured with a higher accuracy than the accuracy of the embodiment. The shape measuring device includes a medium 10, a light source 80, half mirrors 41, 42, bandpass filters 31, 32, 33, sensors 21, 22, 23 such as an imaging camera, and an internal memory 70 m. It includes 70A, an operation unit 71, and a display 72. Hereinafter, the differences between the first basic embodiment and the second basic embodiment will be described.

Referring to FIG. 2A, the reflected light R60 propagates substantially along the optical axis 40a to the sensor 21 via a medium 10, a half mirror 41, and a bandpass filter 31 having a passing wavelength of _{the first wavelength λ 1.} do. The half mirror 41 reflects the reflected light R60 and propagates it toward the sensor 23 via the half mirror 42 and the bandpass filter 33 having a passing wavelength of _{the third wavelength λ 3.} The half mirror 42 reflects the reflected light R60 and propagates it toward the sensor 22 through a bandpass filter 32 having a passing wavelength of _{the second wavelength λ 2.}

The sensors 21, 22, and 23 can be configured by, for example, an image pickup camera. The sensor 21 receives the reflected light R60, _{detects the intensity of the reflected light R60 having the first} wavelength λ 1, and outputs the intensity data to the measuring device 70A. The sensor 22 receives the reflected light R60, _{detects the intensity of the reflected light R60 having the second} wavelength λ 2, and outputs the intensity data to the measuring device 70A. The sensor 23 receives the reflected light R60, _{detects the intensity of the reflected light R60 having the third} wavelength λ 3, and outputs the intensity data to the measuring device 70A.

The measuring device 70A calculates the distance l based on the intensities of the first wavelength, the second wavelength, and the third wavelengths λ ₁ , λ ₂ , λ _{3 having the relationship of λ 1} <λ ₃ <λ _2. By calculating the distance l of a plurality of points on the object surface 60s of the object 60, the one-dimensional, two-dimensional, and three-dimensional shapes of the object 60 can be further measured. In addition, the measuring device 70A reconstructs the three-dimensional image of the object 60 based on the measured shape of the object 60, and images the three-dimensional image of the object 60.

In the second basic embodiment, the following coefficients are defined for measurement.
(A) _{Reflection coefficient s (λ 1} ) of the object surface 60s of the object 60 at the _{first wavelength λ 1;}
(B) _{Reflection coefficient s (λ 2} ) of the object surface 60s of the object 60 at the _{second wavelength λ 2;}
(C) _{Reflection coefficient s (λ 3} ) of the object surface 60s of the object 60 at the _{third wavelength λ 3;}
(D) _{Absorption coefficient α (λ 1} ) of the medium 10 at the _{first wavelength λ 1;}
(E) _{Absorption coefficient α (λ 2} ) of the medium 10 at the _second wavelength λ 2; and (F) _{Absorption coefficient α (λ 3} ) of the medium 10 at the _{third wavelength λ 3.}

In the second basic embodiment, the reflection coefficient s (λ ₁ ) and the reflection coefficient s (λ ₂ ) are such that the difference between the reflection coefficient s (λ ₁ ) and the reflection coefficient s (λ _{2) is the reflection coefficient s (λ 2).} It is preferable to set it to be smaller than a predetermined small value such as ₁ ) or 1/100 of the reflection coefficient s (λ _2). In this case, the wavelength characteristic of the reflection coefficient of the object surface 60s of the object 60 is the wavelength range _{from the first} wavelength λ 1 to the second wavelength λ ₂ via the third wavelength λ _{3 as shown in the following equation.} It is preferable to have a substantially linear relationship (linear approximation) with respect to the wavelength λ.

s (λ) = a × λ + b

Here, a and b are constants. It is expressed by the following equation for each wavelength λ ₁ , λ ₂ , and λ _3.

s (λ ₁ ) = a × λ ₁ + b
s (λ ₂ ) = a × λ ₂ + b
s (λ ₃ ) = a × λ ₃ + b

FIG. 2B is a flowchart showing a shape measurement process executed by the measuring device 70A of FIG. 2A.

In FIG. 2B, first, in step S11, the _{intensities of the first wavelength λ 1} , the second wavelength λ ₂ , and the third wavelength λ ₃ measured by the sensors 21, 22 and 23 are measured by the sensors 21 and 22. , 23. Then, in step S12, the received _{intensities of the first wavelength λ 1} , the second wavelength λ ₂ , and the third wavelength λ _{3 and the first wavelength λ 1} stored in the internal memory 70 m in advance. Based on the absorption coefficient α (λ ₁ ) of the medium 10 in, the absorption coefficient α (λ ₂ ) of the medium 10 at the second wavelength λ _2, and the _{absorption coefficient α (λ 3} ) of the medium 10 at the _{third wavelength λ 3.} , The distance l from the medium surface 10s to the object surface 60s is calculated. In step S13, based on the calculated distance l, the reflectance coefficient characteristic of the object surface 60s is substantially over the wavelength range _{from the first} wavelength λ 1 to the second wavelength λ ₂ through the third wavelength λ _3. The shape of the object surface 60s of the object 60 is measured under the condition of having a linear relationship. Further, in step S14, an image having the three-dimensional shape of the object 60 is reconstructed based on the shape of the object surface 60s of the measured object 60, and the object 60 has the three-dimensional shape reconstructed in step S15. The image is displayed on the display 72, and the shape measurement process is completed.

The measuring device 70A is higher than the first basic embodiment in consideration of the error between the _{reflection coefficients s (λ 1} ) and s (λ ₂ ) on the premise of a substantially linear relationship of the wavelength characteristics of the reflection coefficient. The distance l can be calculated with accuracy. Further, the measuring device 70A can calculate the distance l in a shorter time than the above method by using the nonlinear least squares method and the parallel processing. Further, the measuring device 70A may calculate the distance l in a shorter time than the above method by using the approximate condition of the following equation.

(Case 1) α (λ ₃ ) −α (λ ₁ ) = α (λ ₂ ) −α (λ ₃ )
(Case 2) 2α (λ ₃ ) -2α (λ ₁ ) = α (λ ₂ ) −α (λ ₃ )

As described above, according to the second basic embodiment, the measuring device 70A has a substantially linear relationship (linear approximation) with respect to the wavelength in the wavelength characteristic of the reflection coefficient of the object surface 60s of the object 60. Using assumptions, shape measurements can be made with higher accuracy than in the prior art. In this case, by further performing parallel processing using the nonlinear least squares method, the length calculation can be executed in a shorter time than the above method. In addition, by using the above two conditions for approximation, the length can be calculated in a shorter time as compared with the above method.

Third Basic Embodiment FIG. 3 is a schematic block diagram showing a configuration of an optical shape measuring device according to a third basic embodiment of the present invention. The shape measuring device of the second basic embodiment shown in FIG. 2A includes three sensors 21, 22, 23, but in the third basic embodiment, as shown in FIG. 3, three sensors 21, 22, 23 Alternatively, a spectroscope sensor 24 such as a spectroscope camera may be used. In this case, instead of the three bandpass filters 31, 32, 33, _{only one bandpass filter 34 that allows light of three wavelengths λ 1} , λ ₂ , λ 3 to pass is provided, and the measuring device 70A of FIG. 2A is provided. Instead, a measuring device 70B is provided.

Referring to FIG. 3, the reflected light R60 is substantially reflected along the optical axis 40a and is received by the spectroscope sensor 24 via the bandpass filter 34. The spectrometer sensor 24 outputs not only the intensities of the first, second, and third wavelengths λ ₁ , λ ₂ , λ ₃ but also the RGB image data for photography to the measuring device 70B. In response to this, the measuring device 70B measures the shape of the object 60, performs three-dimensional imaging, and outputs the RGB image data for photography, which is the imaging data, to the display 72 for display.

In the third basic embodiment, the calculation method for high speed and high accuracy in the second basic embodiment can also be used.

Hereinafter, the first embodiment corresponding to the first basic embodiment and the second embodiment corresponding to the second and third basic embodiments will be described in detail with reference to the examples.

First Embodiment In the following first embodiment, chapter numbers and section numbers will be added and described.

1. 1. Shape estimation using the light absorption characteristics of the medium Here, for example, distance estimation using the light absorption characteristics of the medium 10 which is water will be described. First, the light absorption characteristics will be briefly explained, and then the principle of distance estimation from the light absorption characteristics of the _{two wavelengths λ 1} and λ 2 will be described. Next, the accuracy of the estimated distance is considered, and the reflected emission spectrum of the object between the _{two wavelengths λ 1} and λ 2 in the near-infrared region used last is verified.

1.1 Light Absorption Characteristics When light passes through a medium 10, light has the characteristic of being absorbed by the medium 10. However, it is not absorbed at a constant rate in the entire wavelength range of light to reduce its intensity, and light is absorbed according to the absorption characteristics depending on the medium. In general, the medium 10 has a wavelength range in which light is strongly absorbed and a wavelength range in which light is hardly absorbed. In the present embodiment, we focus on water as an example of the medium 10, and consider estimating the shape of the object 60 by utilizing the light absorption characteristics of water.

FIG. 5 is a graph showing the absorption rate of light by water at 400 nm to 1400 nm. That is, in FIG. 5, as an example, a graph showing the ratio of light absorbed in the wavelength range of 400 nm to 1400 nm when light passes through water having a length of 12 mm is shown. As is clear from FIG. 5, it can be seen that almost no light is absorbed in the visible light region of 400 nm to 750 nm. This shows the fact that water looks transparent to the human eye, which is sensitive to this wavelength range. On the other hand, the absorption of light begins to increase from the wavelength range where the wavelength is longer than the visible light range, and the absorption rate rapidly increases from the wavelength range of 900 nm. It can be seen that the light in the near infrared region of 1200 nm to 1400 nm is well absorbed.

FIG. 6A is a photographic image showing the difference in the appearance of water between visible light and near-infrared light, and is a photographic image showing the appearance of water taken by an infrared camera without using a filter. Further, FIG. 6B is a photographic image showing the difference in the appearance of water between visible light and near infrared light, and is a photographic image showing the appearance of water taken by an infrared camera through a 950 nm filter.

That is, FIGS. 6A and 6B are comparative images for confirming that almost no light is absorbed by water in the visible light region, but light is strongly absorbed in the wavelength region where near infrared rays are strongly absorbed. .. These images are taken of a container containing water with an imaging camera that is sensitive to near infrared rays of 1100 nm in addition to the visible light region. Here, FIG. 6A is taken without using a filter, whereas FIG. 6B is taken with a bandpass filter of 950 nm, which is a wavelength at which light is strongly absorbed by water, is installed in front of the camera lens. Since the bandpass filter has a property of transmitting only the light of the band, only the light of 950 nm reaches the image pickup camera, and as a result, a spectroscopic image of 950 nm is obtained.

Further, it can be seen that the image of FIG. 6B absorbs light more strongly than the image of FIG. 6A, and the water looks black. We have confirmed that light is actually absorbed, but this light absorption characteristic follows Lambert-Beer's law.

FIG. 7 is a schematic side view of a device for showing the amount of light according to Lambert-Beer's law. _{Assuming the situation of the device of FIG. 7, the relationship between the light intensity I 0} before being incident on the medium 10 at a certain wavelength λ and the light intensity I after being incident on and transmitted through the medium 10 is determined by Beer-Lambert. According to the law, it can be expressed by the following equation.

Here, l is a distance (mm), α (λ) is an absorption coefficient (mm ^-1 ) depending on the wavelength λ, and e ^{−α (λ) l} is −α (λ) of the Napier number. It is the l-th power. In this way, if the amount of light when incident and the amount of light when emitted are known, it is possible to determine how much light is absorbed, and if the absorption coefficient is known, the length of the medium through which the light has passed is also known. It is possible to estimate.

1.2 Distance estimation using absorption of light of two wavelengths In order to estimate the distance l from the water surface (medium surface 10s) to the object surface 60s in water, the characteristic that the amount of light absorption differs completely depending on the wavelength. Think about using it. As a assumed situation, a scene in which parallel light is applied to the object surface 60s of the object 60 underwater from a light source at infinity is arranged coaxially with the optical axis of the light source 80, for example, positive. The image is taken using the sensors 21 and 22 which are projection cameras. Since it is not assumed that Lambertian reflection, diffuse reflection, or specular reflection occurs on the object surface 60s, it can be considered that the brightness value of the obtained image can be separated into the reflection characteristics of the geometric shape and the wavelength. That is, the reflectance function f (ω, λ) = r (ω) s (λ) is the geometrical shape characteristic r (ω) of the object 60 (ω means the angle of incident light, synchrotron radiation, etc.) and the wavelength in reflection. It is a loose assumption that it can be separated into the characteristic s (λ). Nevertheless, this assumption can be applied to many real objects. A few exceptions are cases where the surface shape of an object is as fine as the wavelength scale of light, such as a CD-ROM.

In order to eliminate the need to consider the influence of the water surface when actually measuring, it is recalled to install an imaging system underwater, but due to the equipment that can be used at present, the sensors 21 and 22 in this embodiment. And the light source 80 is not put into the water of the medium 10. Since it is assumed that the directional light source 80 and the sensors 21 and 22 which are orthographic projection cameras use wavelength ranges in which two wavelengths are close to each other, the influence of specular reflection of light on the water surface can be ignored. This will be described below. Also, if the specular reflection component on the water surface is strong, take a picture of only the specular reflection component and subtract it from the image containing it, or use a polarizing plate to avoid shooting the specular reflection component. There are conceivable ways to deal with this.

1.2.1 Two-wavelength distance image FIG. 8 is a schematic side view of a shape measuring device when, for example, the optical axes of the sensors 21 and 22 and the light source 80 are parallel and have a positional relationship perpendicular to the water surface. As shown in FIG. 8, consider the positional configuration of the light source 80 and the sensors 21 and 22 which are ideally arranged in a straight line. Here, the optical axis of the sensors 21 and 22 and the optical axis of the directional light source 80 are perpendicular to the flat water surface. Monochrome light having a wavelength λ ₁ and a light intensity I ₀ is incident from the water surface, passes through the water of the medium 10, and reaches a matte point 60p at a distance l. The amount of light reflected from the point 60p and sensed by the sensors 21 and 22 is expressed by the following equation.

Here, 2l is twice the distance l, and since the light reciprocates from the water surface (medium surface 10s) to the point 60p on the object surface 60s, it passes through the length twice the distance l. Become. The geometrical shape characteristic r (ω) of the object surface 60s and the wavelength characteristic s (λ) due to reflection are unknown because they naturally depend on the formation and material of the object surface 60s. Another kind of monochrome light is used to cancel this unknown element. Assuming that the wavelength of the light is λ ₂ and the light source 80 has the same light amount I _{0 as the first} light (wavelength λ _{1), the light amount I (wavelength λ 2} ) of the second light sensed by the sensors 21 and 22 Is expressed by the following equation.

Here, by dividing the equation (2) by the equation (3), the distance l can be calculated as the following equation.

となっている２つの波長域の光源８０を選定することができれば、距離ｌは次式のように近似可能となる。 It should be noted that the geometric shape characteristic r (ω) can be canceled no matter how complicated the shape of the object surface 60s is. Here, the _{reflectances at wavelengths λ 1} and λ ₂ are the same, that is,

If the light sources 80 in the two wavelength regions can be selected, the distance l can be approximated as shown in the following equation.

Equation (5) is an equation showing the core algorithm in this embodiment. As a result, even if the material, surface reflection characteristic, and geometric shape characteristic of the object 60 are unknown, it is easy to measure the difference in the brightness value of each pixel of the two camera images taken with respect to the appropriately selected wavelength. The distance l can be estimated.

3.2.2 Estimated distance accuracy Consider, for example, the relative distance estimation error Δl of the following equation for the error Δs of the surface reflection characteristics s (λ ₁ ) and s (λ _{2 ) at the two wavelengths λ 1} and λ _2. Try.

The estimated distance error Δl is calculated from the equations (4) and (5) as follows.

FIG. 9 is a graph showing the relationship between the relative reflectance error and the estimated distance error. That is, FIG. 9 is a _{plot of the relative distance error with respect to the relative reflectance error when the luminance ratio I (λ 1} ) = I (λ ₂ ) is changed (for example, the difference between the two selected wavelengths is increased). Is. As is clear from the curve of FIG. 9, it can be seen that the estimated distance l has almost no effect on the error of the reflected emission spectrum. This is an important criterion for selecting a wavelength in distance estimation using two wavelengths. In particular, in the present embodiment, the maximum estimation accuracy can be obtained by selecting so that the difference in water absorption coefficient is maximized and the difference in reflectance is minimized at the two wavelengths.

3.2.3 Surface reflection characteristics FIG. 10 is a graph showing the reflection spectrum of the "24-color color checker board". That is, FIG. 10 shows the reflection spectra of a color checker board in which 24 patch tiles (24 colors) having different colors are arranged in a grid shape, and the 24 graphs in FIG. 10 show 24 colors. Corresponds to the reflection spectrum of. As can be seen from FIG. 10, the amount of light absorbed in water changes rapidly in the wavelength range of 900 nm to 1000 nm. Furthermore, we will demonstrate that the reflection spectra of various materials tend to be flat in this wavelength range. First, as shown in FIG. 10, when the reflection spectrum of a standard color checker board was measured, the change in the reflection spectrum was significantly reduced in the long wavelength region after 900 nm in all patch tiles. It turned out.

FIG. 11 is a graph showing the reflection spectrum error of the “color checker board” in one wavelength pair. In the example of FIG. 11, as a wavelength pair, a pair of 900 nm and 950 nm and a pair of 900 nm and 920 nm were examined. Although there are some patch tiles with large differences, the relative average of the difference in the reflection spectra between the wavelength pairs of 900 nm and 950 nm is 5.7%, and the relative average of the difference in the reflection spectra between the wavelength pairs of 900 nm and 920 nm is. It has decreased further to 2.1%.

As a further investigation, we collected basic materials such as wood, cloth, leather, and metal, measured the reflection spectrum, and created a database.

FIG. 12A is a spectrum diagram showing a reflection spectrum of “wood”. FIG. 12B is a spectrum diagram showing the reflection spectrum of the “cloth”. FIG. 12C is a spectrum diagram showing a reflection spectrum of “leather”. FIG. 12D is a spectrum diagram showing a reflection spectrum of “metal”.

FIG. 13A is a graph showing the reflection spectrum error of the “wood color checker board” in one wavelength pair. FIG. 13B is a graph showing the reflection spectrum error of the “cloth color checker board” in one wavelength pair. FIG. 13C is a graph showing the reflection spectrum error of the “leather color checker board” in one wavelength pair. FIG. 13D is a graph showing the reflection spectrum error of the “metal color checker board” in one wavelength pair.

In the graphs of FIGS. 12A to 13D, data of about 20 different materials are included in each material. For example, the 20 graphs in FIG. 12A correspond to the data of about 20 different materials of wood. Then, as in the case of the color checker board described above, when the difference in the reflection spectrum between the wavelength pair of 900 nm and 950 nm and the wavelength pair of 900 nm and 920 nm was evaluated, the average error in the wavelength pair of 900 nm and 950 nm was found in wood, cloth, and the like. The average error in the wavelength pairs of 900 nm and 920 nm was 1.4%, 1. It was 1%, 1.9%, and 5.0%. Although this database may have a small sample size of material, the average error evaluation results show that the error in the reflection spectra of two adjacent wavelength pairs in the near infrared region is fairly small.

2. 2. Algorithm of "Shape from Water" The algorithm of "Shape from Water" (hereinafter referred to as "SFW algorithm") for practical setup based on the distance estimation using two wavelength images will be described below. The SFW algorithm aims to correct the camera position deviation and the error of the distance estimation result due to the transmittance of the actual bandpass filter, which are problems in constructing the imaging system.

2.1 Sensor position So far, the directional light source 80 and, for example, the sensors 21 and 22 which are orthographic projection cameras are both installed on the same optical axis, and the optical axis is perpendicular to the water surface. I was expecting it. When actually installing the device of the imaging system, it is difficult to design and adjust the optical axis perpendicular to the water surface, and the light source 80 or the sensors 21 and 22 or both are slightly tilted with respect to the vertical direction. Is expected. Therefore, when estimating the distance of a certain point, we will introduce a method of correcting the error of the estimation result due to the deviation of the device position.

倍になっているので、センサ２１，２２の光軸２１ａと、光源８０の光軸８０ａが傾いているときの距離ｌとセンサ２１，２２で計測した輝度値と水の吸収係数の関係は次式のように表現できる。 FIG. 14 shows that the optical axis 21a of the sensors 21 and 22 (the optical axes of the two sensors 21 and 22 are the same, so one reference numeral 21a is attached) and the optical axis 80a of the light source 80 are not parallel and perpendicular to the water surface. It is a schematic side view of the shape measuring apparatus when it has no positional relationship. As shown in FIG. 14, at a certain point 60p (reflection point on the object surface 60s) in water where the medium 10 is water, for example, the light of the sensors 21 and 22 with respect to the optical axis 40a (virtual line) perpendicular to the water surface. Let θ be the inclination of the axis 21a, and let φ be the inclination of the optical axis 80a of the light source 80. Since the refractive index of water is almost constant in the near-infrared region, the inclinations θ and φ of the optical axes 21a and 80a of these sensors 21 and 22 and the light source 80 are two wavelengths close to each other in the near-infrared region. It can be assumed that there is no change between them. However, although twice the distance l is 2l in the equation (5), the optical axis 21a of the sensors 21 and 22 and the optical axis 80a of the light source 80 are slightly from the reference optical axis 40a from the direction perpendicular to the water surface. By being out of alignment

Since it is doubled, the relationship between the distance l when the optical axis 21a of the sensors 21 and 22 and the optical axis 80a of the light source 80 is tilted, the brightness value measured by the sensors 21 and 22, and the water absorption coefficient is as follows. It can be expressed like an expression.

を簡単に推定することができる。 By using the equation (7), in the distance estimation at one point, the rate of change of the length twice the distance l due to the inclination

Can be easily estimated.

4.2 Narrowband Path Filter In order to implement an imaging system using the SFW algorithm, it is ideal to install and use a light source with sufficient light intensity in a wide wavelength range and two narrowband pass filters in front of the camera. It is a target. So far, we have assumed that the transparency function of the filter is a delta function (a perfect narrowband filter), but this is not realistic. Therefore, it is necessary to consider and correct the transmittance function of the bandpass filter used there. Consider the transmittance functions of the narrow band filter at the two wavelengths λ ₁ and λ _{2 actually used as β 1} (λ) and β ₂ (λ), respectively.

Assuming that the bandpass filter transmits only a sufficiently narrow band, it can be considered that the reflection spectrum of a point of the object 60 is flat between the two bands. As a result, when the transparency function of the bandpass filter is taken into consideration in equation (2), it is expressed by the following equation.

Similarly, the equation (3) is also expressed by the following equation.

By solving the following equations obtained from these two equations, it is possible to estimate the distance l corresponding to the optical path length.

So far, the spectrum of the light source 80 and the response function of the camera have not been considered, but they can be considered in the same way as the function of the filter, and are included in the transmission function of the filter.

3. 3. Coaxial camera system and experiment The present inventors constructed a coaxial system as an imaging system for estimating the shape of the object to be measured 60 by using the SFW algorithm. It is possible to arrange two sensors 21 and 22 coaxially and take two wavelength images in real time at a video rate. The shapes of a complex geometrical object 60 and a dynamic object 60 underwater are estimated from continuously captured images. First, the outline of the imaging system will be described, and the accuracy of the distance actually estimated by the system will be examined.

5.1 Outline and Adjustment of Imaging System FIG. 15 is a photographic image showing the appearance of a coaxial spectral imaging system (shape measuring device) according to the first embodiment of the present invention. As shown in FIG. 15, a coaxial dual-wavelength imaging system was constructed using a beam splitter and two grayscale cameras (POINT-GREYGS3-U3-41C6NIR).

FIG. 16 is a spectral diagram showing a response function of the camera of FIG. 15 (manufactured by POINT-GREY, GS3-U3-41C6NIR type) having a wavelength from 300 nm to 1100 nm. As is clear from FIG. 16, although the sensitivity of near infrared rays is lower than that in the visible light region, it is actually used by increasing the amount of light of the light source 80 or slightly lengthening the exposure time of the sensors 21 and 22. Images with near-infrared wavelengths can be taken clearly. Then, by installing two narrow bandpass filters of 905 nm and 950 nm in front of the sensors 21 and 22, the wavelength image can be taken by the camera.

FIG. 17 is a spectral diagram showing the transmission function of the two bandpass filters of FIG. 15 and the response function of the camera. That is, in FIG. 17, both are plotted in order to compare the curves of the response function of the sensors 21 and 22 in the near infrared region actually used and the transmission function of the bandpass filter. Here, the light source 80 uses an incandescent light bulb having a sufficient amount of light in the near infrared region.

FIG. 18 is a spectrum diagram showing the spectrum of the light source 80 of FIG. Here, the sensors 21 and 22, which are actually two infrared cameras, are synchronized, and the beam splitter and the sensor 21 are capable of capturing two wavelength images with the same optical axis for the object 60 to be photographed at the same angle of view. , 22 positions and angles were adjusted to construct a camera system. Further, in order to estimate the shape using the SFW algorithm, data on the absorption coefficient of water is required, but it can be easily measured. The spectrum of the white target is measured with the white target fixed in water at a known depth. Since the spectrum absorbs light according to the distance, the absorption coefficient of water was calculated by applying the data to Lambert-Beer's law.

FIG. 19 is a spectral diagram showing the absorption coefficient of water. That is, in FIG. 19, the water absorption coefficient measured and corrected at a plurality of depths in order to improve the accuracy is shown.

3.2 Accuracy of the estimated distance In order to evaluate the accuracy of the estimated distance, the distance was estimated by the method according to the present embodiment using a plurality of plates on a plane made of different materials. The accuracy was evaluated by changing the distance from 10 mm to 40 mm with the value measured with a ruler as the true value. In each distance situation, the distance is estimated from the equation (5) by using the images of two wavelengths taken by using the coaxial imaging system as input data. In order to evaluate the effectiveness of the algorithm of the method according to the present embodiment, the estimation results were corrected from the equations (7) and (10).

(1) FIG. 20A is a graph showing the distance accuracy of "cyan-colored tiles", and is a graph showing a comparison between the estimated distance and the true value. FIG. 20B is a graph showing the distance accuracy of the “cyan-colored tile” and is a graph showing the relative error of the estimated distance.
(2) FIG. 21A is a graph showing the distance accuracy of the “red plastic board”, and is a graph showing a comparison between the estimated distance and the true value. FIG. 21B is a graph showing the distance accuracy of the “red plastic board”, and is a graph showing the relative error of the estimated distance.
(3) FIG. 22A is a graph showing the distance accuracy in "white marble", and is a graph showing a comparison between the estimated distance and the true value. FIG. 22B is a graph showing the distance accuracy of "white marble", and is a graph showing the relative error of the estimated distance.
(4) FIG. 23A is a graph showing the distance accuracy in "black marble", and is a graph showing a comparison between the estimated distance and the true value. FIG. 23B is a graph showing the distance accuracy of "black marble", and is a graph showing the relative error of the estimated distance.

That is, the graphs from FIGS. 20A to 23B are the results of evaluating the accuracy of the estimated distance using four kinds of materials: cyan tile, red plastic board, white marble, and black marble. 121 points (pixels) were randomly extracted for each material, and the average of the distance estimation results of the points, the correction results and the true values are shown in the graphs from FIGS. 20A to 23A. Further, in order to evaluate the spatial consistency at each distance, the distribution of the relative error in the estimation result corrected for the distance of the 121 points is shown in the graphs of FIGS. 20B to 23B. In the graphs of FIGS. 20B to 23B, the values in the 25th to 75th percentiles are shown by the box, and the average value is shown by the horizontal line in the center of the rectangular box.

From these results, it can be seen that the correction algorithm plays an important role in improving the estimation accuracy. By correcting, the average value of the distance estimation is a value that is quite close to the true value, and the relative error is about 3%. In order to evaluate the spatial consistency, the corrected estimation result measured and calculated at 121 points is consistent without depending on the texture of the material.

6. Shape estimation experiment Using the SFW algorithm, we tried to estimate the shape of an object with complicated reflection characteristics and an object that moves while changing its shape. Since it is difficult to obtain the true value of the shape data of the object itself, the estimated shape was qualitatively evaluated.

6.1 Shape estimation result of an object with complicated reflection characteristics A drawing of a photographic image showing the shape estimation result is shown below.

(1) FIG. 24A is a photographic image of a "shellfish" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 24B is a photographic image of a “shell” taken with a wavelength of 950 nm by the shape measuring device according to the first embodiment. FIG. 24C is a photographic image showing a “shell” distance image (after black-and-white conversion) estimated by the shape measuring device according to the first embodiment. FIG. 24D is a photographic image showing a "shellfish" estimated shape image estimated by the shape measuring device according to the first embodiment. FIG. 24E is a photographic image showing a “shell” RGB image (after black-and-white conversion) estimated by the shape measuring device according to the first embodiment.

(2) FIG. 25A is a photographic image of a "stone" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 25B is a photographic image of a "stone" taken with a wavelength of 950 nm by the shape measuring device according to the first embodiment. FIG. 25C is a photographic image showing a distance image (after black-and-white conversion) of a “stone” estimated by the shape measuring device according to the first embodiment. FIG. 25D is a photographic image showing an estimated shape image of a “stone” estimated by the shape measuring device according to the first embodiment. FIG. 25E is a photographic image showing an RGB image (after black-and-white conversion) of the “stone” estimated by the shape measuring device according to the first embodiment.

(3) FIG. 26A is a photographic image of a "squirrel object made of wood" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 26B is a photographic image of a “squirrel object made of wood” taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. FIG. 26C is a photographic image showing a distance image (after black-and-white conversion) of a “squirrel object made of wood” estimated by the shape measuring device according to the first embodiment. FIG. 26D is a photographic image showing an estimated shape image of a “squirrel object made of wood” estimated by the shape measuring device according to the first embodiment. FIG. 26E is a photographic image showing an RGB image (after black-and-white conversion) of a “squirrel object made of wood” estimated by the shape measuring device according to the first embodiment.

(4) FIG. 27A is a photographic image of a "monkey object made of pottery" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 27B is a photographic image of a “monkey object made of pottery” taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. FIG. 27C is a photographic image showing a distance image (after black-and-white conversion) of a “monkey object made of pottery” estimated by the shape measuring device according to the first embodiment. FIG. 27D is a photographic image showing an estimated shape image of a “monkey object made of pottery” estimated by the shape measuring device according to the first embodiment. FIG. 27E is a photographic image showing an RGB image (after black-and-white conversion) of a “monkey object made of pottery” estimated by the shape measuring device according to the first embodiment.

(5) FIG. 28A is a photographic image of a "colorful cup object" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 28B is a photographic image of a “colorful cup object” taken with a wavelength of 950 nm by the shape measuring device according to the first embodiment. FIG. 28C is a photographic image showing a distance image (after black-and-white conversion) of a “cup object having a colorful color” estimated by the shape measuring device according to the first embodiment. FIG. 28D is a photographic image showing an estimated shape image of a “colored cup object” estimated by the shape measuring device according to the first embodiment. FIG. 28E is a photographic image showing an RGB image (after black-and-white conversion) of a “cup object having a colorful color” estimated by the shape measuring device according to the first embodiment.

(6) FIG. 29A is a photographic image of a "textured paper cup object" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 29B is a photographic image of a “textured paper cup object” taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. FIG. 29C is a photographic image showing a distance image (after black-and-white conversion) of a “paper cup object having a texture” estimated by the shape measuring device according to the first embodiment. FIG. 29D is a photographic image showing an estimated shape image of a “paper cup object having a texture” estimated by the shape measuring device according to the first embodiment. FIG. 29E is a photographic image showing an RGB image (after black-and-white conversion) of a “textured paper cup object” estimated by the shape measuring device according to the first embodiment.

The photographic images from FIGS. 24A to 29E are the result of shape estimation of an opaque object having texture or having complicated reflection characteristics. These objects are difficult to estimate by the conventional shape estimation method, but as can be seen from the estimation results, the imaging system constructed in this embodiment and the method according to this embodiment are textured objects 60 or a strong mirror surface. It can be seen that it is also useful for an object 60 having a reflection characteristic.

In particular, the colorfully colored cups of FIGS. 28A-28E show the remarkable advantages of the SFW algorithm. The colorful colors create a complex texture that is difficult to estimate with conventional methods, but as you can see from the images at 905 nm and 950 nm, the reflectances of all colors are about the same in this wavelength range. And can be treated as an object without texture.

In addition, the monkey object in FIGS. 27A to 27E is a craft with a strong specular reflection component, and if the sensors 21 and 22 cause saturation at the pixel at the point where specular reflection occurs, the distance estimation result will be inaccurate. Become. However, it can be avoided by setting the sensors 21 and 22 so as not to cause saturation. The surfaces of the shells of FIGS. 24A to 24E and the stones of FIGS. 25A to 25E are considerably uneven, but it can be seen from the estimation results that the uneven portion can also be estimated.

4.2 Shape estimation result of semi-transparent object It is quite difficult to estimate the shape of an object with semi-transparent reflection characteristics by the conventional method. The SFW algorithm, which cancels the reflection characteristics and estimates the shape, can estimate the shape of a translucent object.

The photographic image of the shape estimation result is shown below.

(1) FIG. 30A is a photographic image of a "cherry blossom flower object" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 30B is a photographic image of a “cherry blossom object” taken with a wavelength of 950 nm by the shape measuring device according to the first embodiment. FIG. 30C is a photographic image showing a distance image (after black-and-white conversion) of the “cherry blossom object” estimated by the shape measuring device according to the first embodiment. FIG. 30D is a photographic image showing an estimated shape image of the “cherry blossom object” estimated by the shape measuring device according to the first embodiment. FIG. 30E is a photographic image showing an RGB image (after black-and-white conversion) of the “cherry blossom object” estimated by the shape measuring device according to the first embodiment.

(2) FIG. 31A is a photographic image of "another cherry blossom object" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 31B is a photographic image of "another cherry blossom object" taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. FIG. 31C is a photographic image showing a distance image (after black-and-white conversion) of "another cherry blossom object" estimated by the shape measuring device according to the first embodiment. FIG. 31D is a photographic image showing an estimated shape image of "another cherry blossom object" estimated by the shape measuring device according to the first embodiment. FIG. 31E is a photographic image showing an RGB image (after black-and-white conversion) of "another cherry blossom object" estimated by the shape measuring device according to the first embodiment.

(3) FIG. 32A is a photographic image of a "translucent object having a color gradation" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 905 nm. FIG. 32B is a photographic image of a "translucent object having a color gradation" taken by the shape measuring apparatus according to the first embodiment using a wavelength of 950 nm. FIG. 32C is a photographic image showing a distance image (after black-and-white conversion) of an “object that is translucent and has a color gradation” estimated by the shape measuring device according to the first embodiment. FIG. 32D is a photographic image showing an estimated shape image of an “object that is translucent and has a color gradation” estimated by the shape measuring device according to the first embodiment. FIG. 32E is a photographic image showing an RGB image (after black-and-white conversion) of an “object that is translucent and has a color gradation” estimated by the shape measuring device according to the first embodiment.

(4) FIG. 33A is a photographic image of a "mirror-reflected and translucent heart-shaped object" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 33B is a photographic image of a “mirror-reflected, translucent heart-shaped object” taken by the shape measuring device according to the first embodiment using a wavelength of 950 nm. FIG. 33C is a photographic image showing a distance image (after black-and-white conversion) of a “specular reflection and translucent heart-shaped object” estimated by the shape measuring device according to the first embodiment. FIG. 33D is a photographic image showing an estimated shape image of a “specular reflection and translucent heart-shaped object” estimated by the shape measuring device according to the first embodiment. FIG. 33E is a photographic image showing an RGB image (after black-and-white conversion) of a “specular reflection and translucent heart-shaped object” estimated by the shape measuring device according to the first embodiment.

That is, each of the objects of FIGS. 30A to 33E is translucent and has strong specular reflection, and a strong glossiness can be seen from the RGB image. Further, the objects of FIGS. 32A to 32E have a specular reflection characteristic and a gradation in color, and have a considerably complicated shape. By comparing the RGB images of FIGS. 30E to 33E with the estimation result images, it can be seen that the shapes of these objects can be estimated correctly.

2.3 Dynamic object shape estimation result The coaxial imaging system constructed in this embodiment is suitable for real-time shape estimation of a dynamic object.

The photographic image of the shape estimation result is shown below.

(1) FIG. 34A is a photographic image of a "moving hand (first image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 34B shows the “moving hand (first image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “moving hand (first image)” of FIG. 34A. It is a photographic image.
(2) FIG. 34C is a photographic image of a "moving hand (second image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 34D is a photographic image of the “moving hand (second image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the moving hand (second image) of FIG. 34C. Is.
(3) FIG. 34E is a photographic image of a "moving hand (third image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 34F shows the “moving hand (third image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “moving hand (third image)” of FIG. 34E. It is a photographic image.
(4) FIG. 34G is a photographic image of a "moving hand (fourth image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 34H shows the “moving hand (fourth image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “moving hand (fourth image)” of FIG. 34G. It is a photographic image.

FIGS. 34A to 34H are the results of shape estimation by photographing a scene in which the hand is moving in water. From this result, it can be seen that the shape of a dynamic object can be estimated correctly. The actual shooting is performed at about 30 FPS, and the shape of the hand is estimated in each frame, but here, a frame whose movement is easy to understand is selected.

Next, FIG. 33 shows the shape estimation result of the goldfish swimming in the aquarium.

(1) FIG. 35A is a photographic image of a "goldfish (first image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 35B is a photographic image of the “goldfish (first image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “goldfish (first image)” of FIG. 35A.
(2) FIG. 35C is a photographic image of a "goldfish (second image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 35D is a photographic image of the “goldfish (second image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “goldfish (second image)” of FIG. 35C.
(3) FIG. 35E is a photographic image of a "goldfish (third image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 35F is a photographic image of the “goldfish (third image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “goldfish (third image)” of FIG. 35E.
(4) FIG. 35G is a photographic image of a "goldfish (fourth image)" taken by the shape measuring device according to the first embodiment using a wavelength of 905 nm. FIG. 35H is a photographic image of the “goldfish (fourth image)” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “goldfish (fourth image)” of FIG. 35G.

Although it was not possible to estimate the scene in which a fish is swimming by the conventional method, the method using the SFW algorithm can estimate more than a predetermined value, which shows that it is effective. This also selects a frame that is easy to understand the movement. The image at 905 nm is considerably dark, because the scales of the goldfish have strong specular reflection characteristics, and the sensors 21 and 22 are adjusted so as not to cause saturation.

5. Consideration Although the distance estimation by two wavelengths using the SFW algorithm according to the present embodiment cannot directly deal with the ambient light at present, it is actually illuminated only by the ambient light with the light source 80 turned off. It can be dealt with by taking a picture of the state and subtracting it from the image of the state illuminated by both the ambient light and the light source.

Further, when the imaging system using the SFW algorithm is installed outside the water, the surface reflection on the water surface causes an estimation error. It is known that this problem can be alleviated by taking a single picture without placing the object 60 in the water. This is because when the object is not placed in water, the near infrared rays incident from the water surface are absorbed by the water, the amount of light is almost eliminated, and the light does not return to the sensors 21 and 22. Since the brightness value of the image taken in this state is only the surface reflection component on the water surface, the reflection component on the water surface can be obtained from the image of the object 60 in the water without installing the imaging system in the water. By subtracting the captured image, surface reflection on the water surface can be removed.

In Section 1.2.3 of this embodiment, it was shown that the reflection spectra of various materials hardly change in the wavelength range of 900 nm to 1000 nm in which the absorption of the water used is rapidly strengthened. However, as an exception, we have confirmed that there are materials whose reflectance changes slightly. In the method according to the present embodiment, the difference in reflectance between the two wavelength regions of the object 60 directly leads to an error in the shape estimation result. To deal with this problem, after linearly approximating the spectrum between the two wavelengths, the reflectance error of the object is corrected by using the image of the other wavelength in addition to the two images currently in use. This enables more accurate shape estimation.

Three-dimensional shape estimation of a transparent object 60 was difficult with a non-contact shape estimation method, but the SFW algorithm uses a near-infrared material even for an object made of a transparent material such as glass or plastic. If it does not absorb light, the shape can be estimated. However, when the bottom surface of the object 60 is transparent, there is a problem that the shape cannot be estimated because the water after the light has passed through the bottom surface is also calculated as a distance.

Next, the photographic image to be referred to is as follows.

FIG. 36A is a photographic image of an RGB image (after black-and-white conversion) of "an object in a state where the surface of an egg-shaped object having a transparent upper part is not painted". FIG. 36B is a photographic image of the “object without paint” whose shape is estimated by the shape measuring device according to the prior art with respect to the “object without paint” in FIG. 36A. FIG. 36C is a photographic image of an RGB image (after black-and-white conversion) of "an object in a state where the surface of an egg-shaped object having a transparent upper part is painted". FIG. 36D is a photographic image of the “painted object” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “painted object” of FIG. 36C. FIG. 36E is a photographic image of the “object without paint” whose shape is estimated by the shape measuring device according to the first embodiment with respect to the “object without paint” in FIG. 36A.

36A to 36E show the results of an attempt to estimate the shape of an egg-shaped object made of transparent plastic on the upper surface, although the bottom is not transparent. The laser three-dimensional shape measuring device according to the prior art cannot deal with a completely transparent portion, resulting in inaccurate results (see FIG. 36B). However, the shape can be estimated correctly by using the SFW algorithm. For reference, I painted this object and estimated the state where the surface was not transparent with a laser three-dimensional measuring device, but since it is almost the same as the result of the method according to this embodiment, it is a transparent object by the SFW algorithm. It can be seen that the shape estimation result of is correct.

As shown in Sections 1.2.2 and 1.2.3 of this embodiment, the wavelengths are selected so that the absorption coefficient difference is large between the two near-infrared wavelengths so that the accuracy of shape estimation is improved. However, because the wavelength image having the larger absorption coefficient (950 nm image in this embodiment) absorbs too much light, the image becomes too dark for an appropriate SNR. As a solution to this problem, it is conceivable to lengthen the exposure time or increase the amount of light from the light source at the shooting stage with the camera. Regarding the shape estimation result described in this embodiment, the light amount of the light source is not adjusted due to the limitation of the equipment, but the exposure time of the sensors 21 and 22 is adjusted.

6. Summary As described above, in this embodiment, we have proposed an imaging system using the SFW algorithm, which is a completely new distance estimation method using light absorption. The SFW algorithm estimates the shape by using the difference in the degree of light absorption between the wavelengths in the two near-infrared regions without being affected by the surface reflection characteristics of the object. Using low-cost off-the-shelf hardware, we built a coaxial imaging system and captured two wavelength images at the same time, enabling real-time shape estimation. From the results of actual shape estimation, it was found that the SFW algorithm can accurately estimate the shape of an object with complicated reflection characteristics or a dynamically deforming object.

Second Embodiment In the following second embodiment, chapter numbers will be added and described.

1. 1. Distance estimation using absorption of light of three wavelengths In the first embodiment described above, the _{reflectances of the target objects at the two wavelengths λ 1} and λ _{2 to be} photographed are the same, that is, s (λ ₁ ) ≈ s (λ _2). ), The distance estimation using the absorption of light of two wavelengths has been described. This assumption is fine for most materials, as can be seen from the database of reflection spectra of the basic material (see, eg, FIG. 10). However, there are some substances, such as metals, for which this assumption does not hold. Therefore, ₁ two wavelengths _lambda, in addition to the lambda _2, by using the captured information about the wavelength of the third _{λ 3 (λ 1 <λ 3} <λ 2) that is between lambda ₁ and lambda _2, The method for dealing with the problem that the assumption does not hold is explained below.

FIG. 37 is a graph showing a linear approximation method of the reflected radiation spectrum in the wavelength range of _{[λ 1} , λ ₂ ] used in the shape measuring apparatus according to the second embodiment of the present invention. As shown in FIG. 37, it can be assumed that the reflection spectrum between _{the two wavelengths λ 1} and λ _{2 in the near-infrared region has an unknown slope but is almost linear.} The reflectance s (λ _{3 ) at the third} wavelength λ 3 can be calculated by the following equation.

であり、

であるとき

となる。 here,

And

When

Will be.

Equations (2) and (3) in the first embodiment are formulations of the amount of light sensed by the sensors 21 and 22, but the same applies to the _{third wavelength λ 3 as in the following equation.} Can be represented.

は次式のように表すことができる。 From equations (12) and (13), the ratio of the reflectance of the target object

Can be expressed as the following equation.

By substituting the equation (15) into the equation (13) and the equation (14), the relational expression of the amount of light at the following three wavelengths can be obtained.

From this equation (16), even when this assumption that the reflectances of the target objects are the same at two wavelengths does not hold, it is possible to estimate the accurate distance l by using the three wavelengths.

2. 2. In the reflectance database, reflectance error evaluation by estimation using three wavelengths By using three wavelengths when the reflectances of the target objects are not the same in the wavelength range between the _{two wavelengths λ 1} and λ _2. It is evaluated from the reflectance database that the estimation considering the difference in reflectance is possible. Here, the _{reflectance error is calculated by setting λ 1} , λ ₂ , and λ ₃ to 900 nm, 950 nm, and 925 nm, respectively. In the case of estimation using three wavelengths, the reflectance s (λ _{3 ) at wavelength λ 3 is the two points [λ 1} , s (λ ₁ )] and [λ 1] on the graph, as shown in FIG. 37. ₂ , s (λ ₂ )] is assumed to exist on the straight line of the linear function, and _{the reflectance at the estimated wavelength λ 3} is e (λ ₃ ). In order to evaluate the error between the estimated reflectance e (λ ₃ ) and the actual reflectance s (λ ₃ ), the following equation was calculated.

Here, a spectral diagram for an object made of four materials is shown below.

FIG. 38A is a spectrum diagram showing the result of performing a linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to “wood”. FIG. 38B is a spectrum diagram showing the result of performing a linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to the “cloth”. FIG. 38C is a spectrum diagram showing the result of performing a linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to “leather”. FIG. 38D is a spectrum diagram showing the result of performing a linear approximation of the reflection spectrum in the wavelength range of _{[λ 1} , λ ₂ ] with respect to “metal”.

From the evaluation results shown in FIGS. 38A to 38D, it can be seen that the assumption that the reflection spectra between the _{two wavelengths λ 1} and λ _{2 are almost linear is valid.} When calculating the average error of the reflectance when estimated at two wavelengths, it was 3.8%, 2.1%, 6.0% and 11.1% for wood, cloth, leather and metal, respectively. On the other hand, with the three wavelengths, the error can be significantly reduced to 0.1%, 0.4%, 0.4%, and 0.6%.

3. 3. Speeding up the distance estimation process using the absorption of light of three wavelengths The following equation is obtained by modifying the equation (16) showing the relational equation of the amount of light at the three wavelengths described above.

By solving this equation (16a) for the distance l, the distance l can be obtained. However, estimating the distance l for each pixel of the image has a very high calculation cost, and it is very difficult to estimate the distance of the image in real time even by using the nonlinear least squares method. Therefore, we devised an algorithm that can be estimated in real time by limiting the selection of the three wavelengths to be used.

(1) Case 1:
Select the wavelength at which α (λ ₃ ) −α (λ ₁ ) = α (λ ₂ ) −α (λ _3). Next, if α (λ ₃ ) −α (λ ₁ ) = α (λ ₂ ) −α (λ ₃ ) = k, then equation (16) can be expressed as the following equation.

Multiplying both sides of equation (17) by e ^2kl gives the following equation.

Here, if e ^2kl = A, the equation (18) can be expressed as a quadratic equation for A as in the following equation.

By solving this equation (19) for A, the distance l can be calculated. Since this is a process of simply solving a quadratic equation, the calculation cost is small and it is possible to estimate the distance in real time.

(2) Case 2:
Select the wavelength at which 2 (α (λ ₃ ) −α (λ ₁ )) = α (λ ₂ ) −α (λ _3). Then, as in Case I, multiplying both sides of the equation (16) by ^{e4 kl} gives the following equation.

Here, if e ^2kl = B, the equation (19a) can be expressed as a cubic equation for B as in the following equation.

By solving this equation (20) for B, the distance l can be calculated. Similarly, since it only solves a simple cubic equation, it is possible to estimate the distance in real time.

4. Measuring device whose optical axis is coaxial with the light source (parallel light) For distance estimation and object shape estimation using light absorption, the light source 80, sensors 21 and 22, or both are relative to the water surface. It is expected to be slightly tilted from the vertical direction (see FIG. 14). As described in the first embodiment, it is possible to compensate for the tilting effect. However, in order to make a more accurate estimation, it is desirable that the optical axes of the light source 80 and the sensors 21 and 22 are coaxial. Therefore, we have developed a measuring device that can take pictures with the optical axis of the light source 80, which is parallel light, and the optical axis of the sensors 21, 22, and 23 coaxial.

FIG. 39 is a plan view showing a configuration example of a shape measuring device (corresponding to the device of FIG. 3 of the third basic embodiment) when the optical axis of the light source 80 (parallel light) and the spectroscope sensor 24 are coaxial. be. Here, the spectrometer 24 is a sensor 21, 22, 23 configured by one device.

In FIG. 39, a light source 80, an iris diaphragm IRD, three filters FR, FG, FB, an automatic moving stage MS for filters, cylindrical lenses CL1 to CL4, a spectroscope sensor 24, and a half mirror (beam splitter) 40 are shown. By arranging the object 60 as shown above, the object 60 arranged on the upper side of FIG. 39 can be photographed by the spectroscope sensor 24 in a coaxial state using the light of the light source 80. Here, the filter FR is a red filter that passes only the red wavelength, the filter FG is a green filter that passes only the green wavelength, and the filter FB is a blue filter that passes only the blue wavelength. The automatic moving stage MS selects one of the three filters and moves and arranges it on the optical axis.

In the measuring device of FIG. 39, the light incident from the light source 80 first passes through the cylindrical lens CL1 to become parallel light. Further, only the light having the wavelength specified by one of the filters FR, FG, and FB is transmitted. Then, the light reflected at a right angle by the half mirror 40 travels toward the upper object 60 in FIG. 39. The light radiated to the object 60 is reflected and passes through the half mirror 40 again, but by passing through the half mirror 40 here, it is incident on the spectroscope sensor 24. With this configuration, it is possible to take a picture in a coaxial state between the optical axis of the spectroscope sensor 24 and the light source 80. In FIG. 39, since a small half mirror 40 is assumed, the light is once aggregated by arranging the cylindrical lenses CL2 to CL4 before and after the light is incident on the half mirror 40. That is, when the half mirror 40 has a size equal to or larger than a predetermined light width, the cylindrical lenses CL1 to CL4 may be omitted.

5. Three-wavelength coaxial imaging system In order to estimate the distance using the absorption of light of three wavelengths for a moving object 60, it is necessary to acquire images of each of the three wavelengths in real time. Therefore, using three cameras 21C, 22C, and 23C, a coaxial camera system with the same angle of view was constructed.

FIG. 40 shows a configuration example of a camera system having the same angle of view using three cameras 21C, 22C, and 23C used in the shape measuring device according to the second basic embodiment (in the device of FIG. 2A of the second basic embodiment). It is a plan view which shows (corresponding).

In FIG. 40, two half mirrors 41 and 42 are used to separate light in three directions. By arranging the three cameras 21C, 22C, and 23C so that the separated light can be photographed, it becomes possible to acquire images of three wavelengths in real time. The amount of light incident on the camera 22C that captures the light separated by the half mirror 42 is less than half the amount of light incident on the camera 21C that is incident on the half mirror 41 and captures the separated light, but the light source. By adjusting the amount of light of 80 and the sensitivity of the cameras 21C, 22C, 23C, the difference in brightness is corrected in each of the three cameras 21C, 22C, 23C. In FIG. 40, the RL is a relay lens, and bandpass filters 31, 32, and 33 are arranged on the front surfaces of the cameras 21C, 22C, and 23C, respectively.

Further, the shape estimation result by the shape measuring device according to the second embodiment will be described below.

(1) FIG. 41A is a photographic image showing an input image having a wavelength of 905 nm for an “egg-shaped object” for shape estimation by the shape measuring device according to the second embodiment. FIG. 41B is a photographic image showing an estimation result image whose shape is estimated by the shape measuring device according to the second embodiment with respect to the “egg-shaped object” of FIG. 41A.
(2) FIG. 42A is a photographic image showing an input image having a wavelength of 905 nm for a “food sample of roll cake” for shape estimation by the shape measuring device according to the second embodiment. FIG. 42B is a photographic image showing an estimation result image whose shape is estimated by the shape measuring device according to the second embodiment with respect to the “food sample of roll cake” of FIG. 42A.
(3) FIG. 43A is a photographic image showing an input image having a wavelength of 905 nm for a “turnip object” for shape estimation by the shape measuring device according to the second embodiment. FIG. 43B is a photographic image showing an estimation result image whose shape is estimated by the shape measuring device according to the second embodiment with respect to the “turnip object” of FIG. 43A.

As is clear from FIGS. 41A to 43B, according to the shape measuring device according to the second embodiment, it is estimated and measured with high accuracy as compared with the shape estimation result of the shape measuring device according to the first embodiment. I know I can do it.

As described in detail above, according to the present invention, it is possible to measure the shape using light of two wavelengths arriving from an object with higher accuracy than the prior art. In addition, it is possible to measure the shape of an object by using light of three wavelengths coming from the object with higher accuracy than the prior art.

Although the invention has been fully described in the context of its embodiments, it should be noted that various changes and modifications will be apparent to those skilled in the art with reference to the accompanying drawings. Such changes and amendments should be understood to be within the scope of the invention as defined by the appended claims.

10 Medium 21,22,23 Sensors 21C, 22C, 23C Sensor Camera 24 Spectroscope Sensor 31,32,33 Bandpass Filter (BPF)
40,41,42 Half mirror 40a Optical axis 50 Tank 60 Object to be measured (object)
60s Surface of the object to be measured (object surface)
61 Moving support 70, 70A Measuring device 70m Internal memory 71 Operation 72 Display 80 Light source CL1 to CL4 Collimating lens CAL Camera lens FR, FG, FB Color filter IRD Iris diaphragm L80 Incident light MS Automatic moving stage R60 Reflected light S61, S80 Control signal

Claims (6)

The surface of an object having reflectance coefficients of the first wavelength and the second wavelength via a medium having an absorption coefficient at the first wavelength and an absorption coefficient at a second wavelength longer than the first wavelength. In addition, a light source that irradiates light having a first wavelength and a second wavelength,
A sensor that receives light coming from the surface of the object and propagating through the medium, and measures the intensities of the first wavelength and the second wavelength.
The distance from the surface of the medium to the surface of the object based on the measured intensities of the first and second wavelengths and the absorption coefficients of the medium at the first and second wavelengths. A shape measuring device including a measuring unit that calculates and measures the shape of an object based on the calculated distance.
For the first wavelength and the second wavelength, the difference between the intensity of the shortest distance at the first wavelength and the intensity of the longest distance at the second wavelength is the maximum, and the reflectance coefficient and the second wavelength at the first wavelength are the second. It is selected so that the difference from the reflectance coefficient at the first wavelength is the minimum and the difference between the absorption coefficient at the first wavelength and the absorption coefficient at the second wavelength is the maximum.
Here, the shortest distance means the shortest distance among a plurality of corresponding distances at a plurality of positions on the surface of the object.
The longest distance means the longest distance among a plurality of corresponding distances at a plurality of positions on the surface of the object.
_{Let the first} wavelength be λ 1 and the second wavelength be λ ₂ .
_{Let I (λ 1} ) be the intensity of the measured first wavelength.
_{Let I (λ 2} ) be the intensity of the measured second wavelength.
_{Let α (λ 1} ) be the absorption coefficient of the medium at the first wavelength.
_{Let α (λ 2} ) be the absorption coefficient of the medium at the second wavelength.
_{Let s (λ 1} ) be the reflectance coefficient at the first wavelength.
When the reflectance coefficient at the second wavelength is s (λ ₂ ),
The distance l from the surface of the medium to the surface of the object is determined by using the following equation.

A shape measuring device characterized by calculation. The shape measuring apparatus according to claim 1, wherein the first and second wavelengths are selected so that the spectral response of the sensor is larger than a predetermined third value. The shape measuring device according to claim 1 or 2, wherein the measuring unit further reconstructs a three-dimensional image of the object based on the measured shape of the object. The shape measuring device according to any one of claims 1 to 3, wherein the medium contains at least one of water, liquid, gas, solid, and gel. The shape measuring device according to any one of claims 1 to 4, wherein the sensor is a spectrometer sensor. The light source has a reflectance coefficient of each of the first wavelength and the second wavelength via a medium having an absorption coefficient at the first wavelength and an absorption coefficient at a second wavelength longer than the first wavelength. A step of irradiating the surface of an object with light having a first wavelength and a second wavelength,
A step in which the sensor receives light coming from the surface of the object and propagating through the medium to measure the intensities of the first wavelength and the second wavelength.
The measuring unit calculates the distance from the surface of the medium to the surface of the object based on the measured intensities of the first wavelength and the second wavelength and the absorption coefficients at the first and second wavelengths. A shape measuring method including a step of measuring the shape of an object based on the calculated distance.
For the first wavelength and the second wavelength, the difference between the intensity of the shortest distance at the first wavelength and the intensity of the longest distance at the second wavelength is the maximum, and the reflectance coefficient and the second wavelength at the first wavelength are the second. It is selected so that the difference from the reflectance coefficient at the first wavelength is the minimum and the difference between the absorption coefficient at the first wavelength and the absorption coefficient at the second wavelength is the maximum.
Here, the shortest distance means the shortest distance among a plurality of corresponding distances at a plurality of positions on the surface of the object.
The longest distance means the longest distance among a plurality of corresponding distances at a plurality of positions on the surface of the object.
_{Let the first} wavelength be λ 1 and the second wavelength be λ ₂ .
_{Let I (λ 1} ) be the intensity of the measured first wavelength.
_{Let I (λ 2} ) be the intensity of the measured second wavelength.
_{Let α (λ 1} ) be the absorption coefficient of the medium at the first wavelength.
_{Let α (λ 2} ) be the absorption coefficient of the medium at the second wavelength.
_{Let s (λ 1} ) be the reflectance coefficient at the first wavelength.
When the reflectance coefficient at the second wavelength is s (λ ₂ ),
The distance l from the surface of the medium to the surface of the object is determined by using the following equation.

A shape measuring method characterized by calculation.

Images