JP6364777B2 - Image data acquisition system and image data acquisition method - Google Patents

Description

  The present invention relates to an image data acquisition system and an image data acquisition method for acquiring internal scattering characteristics by the three-dimensional shape, normal line, and spectrum of a translucent object surface.

In recent years, CG (computer graphics) for creating images using computer resources has been put into practical use by improving the calculation speed in computer systems and developing image processing algorithms.
For example, the three-dimensional CG expresses an optical phenomenon when a three-dimensional object is illuminated by a predetermined light source with a mathematical model. In addition, the 3D CG generates a 3D two-dimensional image to be displayed on a 2D screen by adding shading or shading to the surface of the 3D object and further pasting a pattern based on a mathematical model.

In order to create this three-dimensional CG, three-dimensional shape, spectral, and normal image data of the substance are required. In addition, when the object to be reproduced by 3D CG is a translucent object (hereinafter referred to as a translucent object), scattering characteristics (BSSRDF: Bidirectional Subsurface Scattering Distribution Function, bidirectional scattering surface reflectivity) Scattering data showing the distribution function is required.
Currently, various measuring devices are commercially available, but few are compatible with translucent objects. Those for translucent objects are still in the research stage and specialized in shape measurement. Thus, at present, there is no measuring apparatus that can collectively measure the internal scattering characteristics by a three-dimensional shape, normal line, and spectrum of a translucent object.

As a method of measuring only the shape of a semi-transparent object, a direct shift component (direct reflection light) and indirect reflection of reflected light are performed by a phase shift method in which a sine wave pattern (pattern light) is projected from a projector onto a semi-transparent object. There is a method for separating the reflection component (indirect reflected light) (see, for example, Non-Patent Document 1).
In addition, there is a method of reducing the influence of scattered light from a translucent object and improving the accuracy of shape measurement by combining a polarizing plate with a phase shift method (for example, Non-Patent Document 2, Non-Patent Document 3, and Non-Patent Document 3). (See Patent Document 4).
In addition, as object shape measurement, there are a method using a computer tomography technique (for example, see Patent Document 4) and a method for measuring using a confocal pattern projection method (for example, see Patent Document 5).

The three-dimensional shape measurement method can measure the three-dimensional shape of an object only with the accuracy of the shape information of the resolution of the pattern light to be projected in the case of an apparatus that projects and measures the pattern light.
A illuminance difference stereo method is representative as a method for acquiring normal information that is a local surface inclination in a three-dimensional shape. The illuminance difference stereo method is usually a three-dimensional shape measurement method that requires three or more light sources, but there is a method that can acquire normal line information with two light sources (see Non-Patent Document 5, for example) The three-dimensional shape cannot be measured.

Japanese Patent Application Laid-Open No. H10-228561 describes a method for simultaneously acquiring the three-dimensional shape of an object and the spectral information of the target surface. That is, the spectral pattern is projected onto the surface of the object to be measured, the acousto-optic tunable filter attached in front of the camera is controlled, and the spectral pattern warped from the surface of the object is imaged for each wavelength. Then, the three-dimensional shape of the object is acquired from the captured spectral pattern image by triangulation.
In Patent Document 1, white light is projected onto the surface of the object to be measured, the transmission wavelength of the acousto-optic tunable filter is controlled, and the spectrum pattern reflected from the object surface is imaged once or a plurality of times. Then, spectral information on the surface of the object is obtained from an image of the obtained spectral pattern by an image arithmetic circuit.

Further, Patent Document 2 describes a method in which a three-dimensional shape measuring device using a slit light projection method (light cutting method) and a spectroscopic information measuring device by switching and photographing a large number of filters having different spectral regions are described. Has been. Thereby, the three-dimensional shape of the object and the spectral information of the target surface can be acquired simultaneously.
Furthermore, Patent Document 3 describes a technique for acquiring spectral reflectance as an absolute value. However, the above-described methods for measuring a three-dimensional object are all limited to diffuse reflection objects and cannot be applied to semi-transparent objects.

In addition, in order to reproduce a CG image of a translucent object, there are the following two methods for measuring and recording information on the internal scattering characteristics of the translucent object used for the reproduction.
The first method is to irradiate the surface of a semi-transparent object with light under general illumination and photograph an image in which reflected light of the light from the semi-transparent object is blurred. This is a technique for recording internal scattering characteristics called BSSRDF from the degree of blur of reflected light (see, for example, Patent Document 4 and Non-Patent Document 6).
Second, the black dot pattern is projected onto the target translucent object, and the internal scattering characteristics for determining BSSRDF are measured and recorded from the percentage of light scattered from the bright area into the black dot pattern area. (For example, refer nonpatent literature 7).

  In addition, in order to generate a CG image, a parameter of a dipole model, which is an approximate model of BSSRDF, is calculated, and rendering is performed using each of the three-dimensional shape, normal line, and spectral data and the dipole model parameter. Yes (Non-Patent Document 8).

JP 2003-294424 A JP 2000-009440 A JP2012-078221A Special table 2012-502262 gazette Special table 2012-530267 gazette

S. K. Nayer, G. Krishnan, M. D. Grossberg, R. Rasker, “Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination”, In (SIGGRAPH) (2006). Soshi Kamata, Ken Ohashi, Toshiro Ejima. Proposal of high-speed 3D measurement method using phase shift method. IEICE technical report. Pattern recognition and media understanding, Vol. 99, No. 515, pp. 43-50, 19991217. M. Gupta, A. Agrawal, A. Veeraraghavan and S. Narasimhan: “Structured light 3d scanning under global illumination”, Proc. Of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011). T. Chen, H. P. A. Lensch, C. Fuchs and H. P. Seidel: “Polarization and phase-shifting for 3d scanning of translucent objects”, Proc. Of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007). Q. Zhang, M. Ye, R. Yang, Y. Matsushita, B. Wilburn and H. Yu: “Edge-Preserving Photometric Stereo via Depth Fusion”, Proc. Of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ( 2012). Yasuhiro Mukakawa, Kazuya Suzuki, Yasushi Yagi, Analysis of subsurface scattering under general lighting, Image recognition and understanding symposium (MIRU2008). A. Ghosh, T. Hawkins, P. Peers, S. Frederiksen, and P. Debevec “Practical Modeling and Acquisition of Layered Facial Reflectance”, In (SIGGRAPH Asia) (2008). H. W. Jensen, S. R. Marschiner, M. Revoy, and P. Hanrahan: “A practical model for subsurface light transport”, In (SIGGRAPH) (2001).

However, according to the above-described method, it is impossible to acquire spatially dense scattering characteristics with the accuracy of accurately calculating a dipole model for rendering.
In addition, in order to determine the internal scattering characteristics of a semi-transparent object based on the three-dimensional shape, normal line, and spectrum, a pattern specialized for each measurement of the three-dimensional shape, normal line, and spectrum is applied to the semi-transparent object. It is necessary to project it.

  In addition, the illuminance difference stereo method does not consider the normal estimation method for the three-dimensional shape of the translucent object, and the estimation of the three-dimensional shape and the normal cannot be performed collectively. For this reason, the above-described method specialized in measuring the shape, normal line, and spectrum of a translucent object individually, and individually measuring the internal scattering characteristics of the translucent object based on these measurement results. It is a technique. Therefore, the shape, normal, spectral and internal scattering characteristics of these semi-transparent objects cannot be measured simultaneously and with a single system.

  For this reason, when measuring each of the three-dimensional shape, normal line, and spectrum of the semi-transparent object used when generating a CG image of the semi-transparent object by rendering, each apparatus is required, and the measuring apparatus is very expensive. It takes. In addition, it takes a lot of time to determine the internal scattering characteristics of a translucent object after several measurements are performed to determine each of the shape, normal line, and spectrum of the transparent object with a plurality of measuring devices.

  The present invention has been made in view of such a situation. All data on the shape, normal, spectral and internal scattering characteristics of a translucent object used when rendering and generating a CG image of the translucent object are made. Provide an image data acquisition system and an image data acquisition method that are obtained collectively by irradiating only a single sequence pattern with a single system.

In order to solve the above-described problems, an image data acquisition system according to the present invention includes a three-dimensional shape information on a surface of a translucent object, surface normal information, a primary scattering characteristic with shadows removed, and multiple scattering. A pattern light irradiating unit that irradiates the semi-transparent object with pattern light of two or more different phases, and a surface of the semi-transparent object irradiated with the pattern light. An image capturing unit that spectrally divides the reflected light and obtains spectral imaging data for each predetermined wavelength, and a phase component , the normal information, and the primary scattering from the spectral imaging data for each wavelength according to a predetermined arithmetic expression. a single scattering reflection component for determining characteristics, the separating each of the multiple scattering reflection component for obtaining multiple scattering characteristics, the phase image, the primary diffuse reflection component image, and each of the multiple scattering reflection component image Generate An image data analysis unit, a shape calculation unit that obtains three-dimensional shape information of the surface of the translucent object based on the phase image based on the phase image, and the primary scattered reflection component image by two or more different projectors From the multiple scattering reflection component image, a normal calculation unit for obtaining the normal information from the multiple scattering reflection component image, and a scattering calculation unit for obtaining multiple scattering characteristics represented by a predetermined function.

  The image data acquisition system according to the present invention is characterized in that the scattering calculation unit obtains the predetermined function indicating a multiple scattering characteristic for each pixel from the multiple scattered reflection component image.

  In the image data acquisition system according to the present invention, the normal calculation unit obtains the primary scattered reflection component image obtained from each of the two or more kinds of pattern lights by two or more different projectors, and each projector pixel. A shadow image of the translucent object is obtained from a ray image representing a direction from the object surface to the object surface, and a normal line of the translucent object and a primary scattering characteristic of the image albedo are obtained from the shadow information. To do.

  In the image data acquisition system of the present invention, the shape calculation unit includes: each pixel of the phase image in which the phase of each of the two or more types of pattern light is different; and each pixel of the pattern light projected from the projector Taking correspondence, the three-dimensional shape of the translucent object is estimated by triangulation.

An image data acquisition method according to the present invention acquires three-dimensional shape information of a surface of a translucent object, surface normal information, primary scattering characteristics from which shading has been removed, and multiple scattering characteristics. A pattern light irradiation process of irradiating the semi-transparent object with two or more types of pattern light of different phases, and the reflected light from the surface of the semi-transparent object irradiated with the pattern light, The primary scattering reflection for obtaining the phase component , the normal information, and the primary scattering characteristic from the spectral imaging data for each wavelength and the spectral imaging data for each wavelength by a predetermined arithmetic expression. a component, each of the multiple scattering reflection component for obtaining said multiple scattering properties were separated, the phase image, the primary diffuse reflection component image, and the imaging data analysis process of generating the respective multiple scattering reflection component image, the Phase image Based on the shape calculation process for obtaining the three-dimensional shape information of the surface of the translucent object by the phase shift method and the normal calculation for obtaining the normal information from the primary scattered reflection component images by two or more different projectors And a scattering calculation process for obtaining a multiple scattering characteristic represented by a predetermined function from the multiple scattered reflection component image.

  As described above, according to the present invention, the image data acquisition system performs all of the shape, normal, spectral and internal scattering characteristics of the translucent object used when rendering and generating a CG image of the translucent object. These data can be obtained collectively by irradiating only a single sequence pattern with a single system.

It is a figure which shows the structural example of the image data acquisition system 1 by this embodiment of this invention. It is sectional drawing of the semi-transparent object which shows the optical model which demonstrates the reflected light in a semi-transparent object. It is a block diagram which shows the structural example of the measurement process part 5 in the image data acquisition system 1 of FIG. This is an example of pattern light that the light irradiation control unit 101 causes the projector 3 or the projector 4 of the pattern light irradiation unit 7 to emit. It is a figure which shows the picked-up image of the translucent object 6 used as the analysis object, the phase image 11, the primary scattering reflected image 19, and the multiple scattered reflected image 20. FIG. It is a figure explaining the process which produces | generates the normal line (normal line information) of the normal line calculating part 105 in this embodiment. It is a figure which shows an example of the graph of the function of BSSRDF. 4 is a flowchart illustrating an operation example of acquiring spectral imaging data of a translucent object 6 by the image data acquisition system 1. 4 is a flowchart illustrating an operation example of image analysis processing of a translucent object in the image data acquisition system 1.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration example of an image data acquisition system 1 according to this embodiment of the present invention. The image data acquisition system 1 according to the present embodiment includes an imaging unit 2, a measurement processing unit 5, and a pattern light irradiation unit 7. The pattern light irradiation unit 7 includes two or more projectors that project a pattern, for example, two projectors 3 and 4 in this embodiment.

  Each of the projector 3 and the projector 4 irradiates light (for example, a sine wave pattern light described later) having a different phase (the phase of the pattern light in the phase shift method) onto the translucent object 6 as the object. To do. 1 has a configuration in which two projectors 3 and 4 are combined, but may be configured as one projector in which two or more light sources are combined.

That is, the projector 3 irradiates the semitransparent object 6 with the sine wave pattern LT1. The projector 4 irradiates the semitransparent object 6 with the sine wave pattern LT2 in a time zone different from the sine wave pattern LT1. The imaging unit 2 captures reflected light RLT1 and reflected light RLT2 in which the sine wave pattern light LT1 and sine wave pattern light LT2 irradiated from each of the projector 3 and the projector 4 are reflected by a translucent object.
Returning to FIG. 1, as will be described later, the measurement processing unit 5 uses the imaging data for each wavelength of the reflected light from the semitransparent object imaged by the imaging unit 2 to determine the three-dimensional shape, normal line, and spectrum of the semitransparent object 6. Measure the scattering characteristics by.

  FIG. 2 is a cross-sectional view of a translucent object showing an optical model for explaining reflected light in the translucent object. As shown in the sectional view of FIG. 2, the translucent object 6 of FIG. 1 reflects and reflects the sine wave pattern light (sine wave pattern light LT1 and sine wave pattern light LT2) emitted from the pattern light irradiation unit 7. Emits as light. The sine wave pattern light is separated into a plurality of components of a specular reflection component 8, a primary scattering reflection component 9, and a multiple scattering reflection component 10 in the translucent object 6.

Here, the specular reflection component 8 is component light of sine wave pattern light that is specularly reflected by the surface 60 of the translucent object 6 and reflected directly from the surface 60.
The primary scattered reflection component 9 is a component light of sine wave pattern light that is reflected from the internal reflection point 61 in the vicinity of the surface 60 of the translucent object 6 about once and emitted from the translucent object 6.

The multiple scattering reflection component 10 is reflected mutually between the plurality of internal reflection points 61 inside the semitransparent object 6, that is, the sine wave pattern light emitted from the semitransparent object 6 after being multiple scattered by the plurality of internal reflection points 61. Component light.
The captured image obtained by capturing the reflected light (reflected light RLT1 and reflected light RLT2) of the translucent object 6 by the irregular reflection (subsurface scattering) below the surface becomes a blurred image.

  That is, due to this subsurface scattering, at the internal reflection point 61 inside the translucent object 6, as shown in FIG. 2, unlike the specular reflection component 8 or the primary scattering reflection component 9, the multiple scattering reflection component 10 is The incident point of the sine wave pattern light does not coincide with the emission point from which the reflected light is emitted. Because of this subsurface scattering, the reflected light is observed even from a place where the sine wave pattern light is not incident, so the captured image becomes a blurred image. The image data acquisition system 1 according to the present embodiment irradiates the surface of the translucent object 6 with a sine wave pattern light, and captures image data for each wavelength of reflected light acquired by the imaging unit 2 (spectral imaging data described later). Thus, the three-dimensional shape of the translucent object 6, the normal line, and the scattering characteristics due to spectroscopy are measured.

FIG. 3 is a block diagram illustrating a configuration example of the measurement processing unit 5 in the image data acquisition system 1 of FIG. In FIG. 3, the measurement processing unit 5 includes a control unit 100, an operation unit 107, a display unit 108, an external interface 109, and a storage unit 110.
The control unit 100 is, for example, a CPU (Central Processing Unit). The control unit 100 is a functional unit that controls the operation of each unit in the measurement processing unit 5, and executes a program (for example, a predetermined translucent object measurement application program) that is previously written in the internal storage unit within the control unit 100. By doing so, control processing of each functional unit is performed.

The operation unit 107 includes, for example, a keyboard, a mouse, and a touch panel (not shown). The operation unit 107 accepts input of information including operation and processing commands and data for the image data acquisition system 1 and outputs the information to the control unit 100. To do.
The display unit 108 is, for example, a liquid crystal display or the like, and is an image that prompts an operator to input information by the operation unit 107 under the control of the control unit 100, or an image that shows calculation results and measurement data by each calculation process described later. Is displayed.

  The external interface 109 is a connection interface that transmits and receives data to and from the imaging unit 2 and the pattern light irradiation unit 7. That is, the control unit 100 transmits / receives data such as reception of spectral imaging data from the imaging unit 2 or transmission of control information for controlling emission of sine wave pattern light to the pattern light irradiation unit 7 via the external interface 109. Do it.

The storage unit 110 is a storage area in which information such as imaging data for each wavelength of reflected light captured by the imaging unit 2 is written and stored by the control unit 100. The storage unit 110 is configured by a mass storage device such as a hard disk drive (HDD) or a solid state drive (SSD), which is a general storage device.
The measurement processing unit 5 may be a general-purpose PC (Personal Computer) including the control unit 100, the operation unit 107, the display unit 108, the external interface 109, and the storage unit 110 described above.

The control unit 100 includes a light irradiation control unit 101, an imaging control unit 102, an imaging data analysis unit 103, a shape calculation unit 104, a normal calculation unit 105, and a scattering calculation unit 106.
The light irradiation control unit 101 controls which one of the projector 3 and the projector 4 in the pattern light irradiation unit 7 in FIG.

  FIG. 4 is an example of pattern light that the light irradiation control unit 101 emits to the projector 3 or the projector 4 of the pattern light irradiation unit 7. FIG. 4 shows the surface irradiated with the sine wave pattern light. Since the light intensity changes in a sine wave shape, a change in intensity is observed between the bright part and the dark part. The phase of the pattern light being out of phase means, for example, that the position of the highest portion in the light intensity is different, that is, the position of the intensity changing in a sine wave shape is different.

  Returning to FIG. 3, the light irradiation control unit 101 outputs a control signal to the pattern light irradiation unit 7, and from the projector 3 or the projector 4 in the pattern light irradiation unit 7, several tens of types of sine wave pattern light are translucent. The object 6 is irradiated. Specifically, the light irradiation control unit 101 alternates each of the sine wave pattern light LT1 and the sine wave pattern light LT2 in time series with respect to a plurality of projectors, each of the projector 3 and the projector 3 in the present embodiment. The semi-transparent object 6 is irradiated. That is, the light irradiation control unit 101 does not irradiate each of the sine wave pattern light LT1 and the sine wave pattern light LT2 to the semi-transparent object 6 at the same timing, and irradiates only one of them.

  Each of the projector 3 and the projector 4 emits sine wave pattern light LT1 and sine wave pattern light LT2 having wavelengths corresponding to white light having a flat spectral distribution characteristic (spectral distribution characteristic). Each of the projector 3 and the projector 4 uses, for example, a standard white plate that is a calibration standard plate for a spectrophotometer, and uses a sine wave pattern light at a wavelength at which the spectral distribution in the spectral distribution characteristic of the spectral distribution characteristic becomes flat. Calibration is performed so that LT1 and sine wave pattern light LT2 are emitted.

  The imaging unit 2 includes the reflected light RLT1 and the reflected light of the sine wave pattern light LT1 and the sine wave pattern light LT2 emitted from each of the projector 3 and the projector 4 in the pattern light irradiation unit 7 and reflected by the translucent object 6. RLT2 is condensed and received. Here, the imaging unit 2 has a spectral function of incident light (for example, a spectroscope (not shown)) inside, and spectrally separates and separates each of the incident reflected light RLT1 and reflected light RLT2 in time series. Imaging data representing the color of the translucent object 6 (hereinafter referred to as spectral imaging data) is sequentially received every time.

  That is, the imaging unit 2 converts each of the reflected light RLT1 and the reflected light RLT2 of the sine wave pattern light LT1 and sine wave pattern light LT2 that are received white light into three or more different wavelengths λ1, λ2, λ3, ..., Λn (n is an integer equal to or greater than 3) is divided into light (post-spectral method).

Then, the imaging unit 2 performs spectral processing for each of the reflected light RLT1 and the reflected light RLT2, and the imaging data of each of the wavelengths λ1, λ2, λ3,. Capture image data.
The imaging control unit 102 inputs spectral imaging data captured by the imaging unit 2 via the external interface 109.

  Then, the imaging control unit 102 writes and stores the light into the storage unit 110 for each of the wavelength components λ1, λ2, λ3,..., Λn, which are spectral components, in the units of the reflected light RLT1 and the reflected light RLT2. . In the present embodiment, “color” indicates the spectral reflectance of each of three or more wavelengths, that is, wavelengths λ1, λ2, λ3,. For this reason, hereinafter, in the present embodiment, the spectral reflectance is indicated as “color”.

  Further, in the present embodiment, a description will be given assuming that a spectroscope is provided inside the imaging unit 2 and the incident sine wave pattern light is dispersed. However, as a configuration in another embodiment, a spectroscope may be provided inside the pattern light irradiation unit 7 instead of the imaging unit 2. That is, the spectroscope inside the pattern light irradiation unit 7 spectrally separates the sine wave pattern light LT1 and the sine wave pattern light LT2 emitted from each of the projector 3 and the projector 4, and wavelength λ1, λ2, λ3,. It is good also as a structure (structure of a pre-spectral system) which irradiates the semi-transparent object 6 with every sine wave pattern light.

  In this case, the imaging unit 2 is supplied with information on which wavelength of the sine wave pattern light the pattern light irradiation unit 7 is radiating from the pattern light irradiation unit 7 for each wavelength to be dispersed. In addition, the imaging unit 2 captures spectral imaging data, which is a spectrally captured image for each of the wavelengths λ1, λ2, λ3,. Then, the imaging control unit 102 inputs information on which wavelength of the sine wave pattern light received from the pattern light irradiation unit 7 is supplied from the imaging unit 2. Based on the above information, the imaging control unit 102 divides the captured spectral imaging data for each of the wavelength components λ1, λ2, λ3,. Write to the storage unit 110 and store it.

The imaging data analysis unit 103 reads the spectral image data stored by the imaging control unit 102 from the storage unit 110 for each wavelength, and performs various image analysis described below.
The imaging data analysis unit 103 obtains a phase image (a phase image 11 described later), a primary scattered reflection image (a 1 described later) from the spectral imaging data for each wavelength of the translucent object 6 irradiated with the sine wave pattern by the phase shift method. A process of separating each of the next scattered reflection image 19) and the multiple scattered reflection image (multiple scattered reflection image 20 described later) is performed.

  As already described, when the imaging unit 2 images the semi-transparent object 6, the pattern light irradiation unit 7 irradiates the semi-transparent object 6, collects the reflected light reflected by the semi-transparent object 6, and Spectroscopic imaging is performed as spectral imaging data. Each of the spectral imaging data acquired by the imaging unit 2 is composed of a set of pixels arranged in a matrix, like general image data. However, since the spectral imaging data is captured for each wavelength, each pixel in the spectral imaging data has light intensity data at the wavelength (color) corresponding to each spectral imaging data.

Then, the spectral imaging data of all wavelengths is synthesized by superimposing the pixel positions by matching the intensity of each pixel with the intensity of the color corresponding to the wavelength, and overlapping the pixel positions. become.
Note that when the imaging unit 2 captures spectral imaging data and the imaging control unit 102 receives the spectral imaging data, the three-dimensional shape, normal, and spectral scattering characteristics of the semi-transparent object 6 are unknown. is there.

In the following description, each of the projector 3 and the projector 4 is controlled by the light irradiation control unit 101, and as an initial phase pattern image having three different phases, the sine wave pattern light having the luminance value shown in the following equation (1). The translucent object 6 is irradiated with LT1 and sine wave pattern light LT2. Then, the imaging data analysis unit 103 uses the pixel data of the obtained reflected light (reflected light RTL1 and reflected light RTL2) to perform the calculation of the following formula (2), thereby obtaining a phase image and primary scattering. Each of the phase value Φ (x, y), the direct reflection component L _d , and the indirect reflection component L _g for generating the reflection image and the multiple scattering reflection image is obtained.

In the above equation (1), R _i (x _p , y _p ) represents each coordinate (x on the radiation plane (two-dimensional plane formed from the x axis and the y axis) of the projector 3 or the projector 4 in the initial phase δ _i . axis coordinates x _p and y-axis input luminance value at the coordinates y _p) (a luminance value) of each coordinate in a sinusoidal pattern light irradiation control unit 101 controls (where, i is the phase number). X _u is the number of pixels corresponding to one cycle of the sine wave. R _max represents the maximum input luminance value in the sine wave pattern light.

Thereby, when the reflected light reflected from the translucent object 6 is imaged, the luminance value I _i (x, y) of each pixel of the spectral imaging data captured by the imaging unit 2 is expressed by the following equation (2). Where i is the phase number.

In the above equation (2), Φ (x, y) is a coded phase value on the light receiving plane (two-dimensional coordinates composed of the x axis and the y axis) of the image sensor in the imaging unit 2. L _d (x, y) is a direct reflection component of sinusoidal pattern light. L _g is an indirect reflection component of sinusoidal pattern light. If i is 3 or more, that is, if there are three coordinates having different phases, each of the phase value Φ (x, y), the direct reflection component L _d , and the indirect reflection component L _g can be estimated.

The calculations described above, in units of types of each of the sinusoidal pattern light LT1 and sinusoidal pattern light LT2, using a spectroscopic imaging data for each wavelength, by each wavelength (2), direct reflection component L _d and indirect The reflection component _Lg is obtained.
That is, in both the sine wave pattern light LT1 and the sine wave pattern light LT2, the phase value Φ (x, y), the direct reflection component L _d , and the indirect reflection component for each type of sine wave pattern according to the above equation (2). Estimate each of L _g .

The light receiving unit of the imaging unit 2 is provided with a polarizing filter, and the specular reflection component 8 is not incident on the imaging unit 2, and the primary scattering reflection component 9 and the multiple scattering reflection component 10 are incident on the imaging unit 2. .
In the present embodiment, a direct reflection component L _d to reflect about once surface shallow part of of a translucent objects 6 (near the surface) and the primary reflection component 9, reflected several times under the surface Thus, the indirect reflection component L _g that causes subsurface scattering is used as the multiple scattering reflection component 10.

  In the image data acquisition system 1 of the present embodiment, the sine wave pattern light LT1 and the sine wave pattern light LT2 are irradiated and projected onto the surface of the translucent object 6 by the phase shift method. Then, the image data acquisition system 1 captures the reflected light to obtain a phase image 11, a primary scattered reflected image 19 composed of the primary scattered reflected component 9, and a multiple scattered reflected image 20 composed of the multiple scattered reflected component 10. Can be sought.

  FIG. 5 is a diagram showing a captured image, a phase image 11, a primary scattered reflected image 19, and a multiple scattered reflected image 20 of the translucent object 6 to be analyzed. FIG. 5A is a captured image 21, for example, an image generated by combining spectral imaging data obtained by imaging only the sine wave pattern light LT <b> 1 as irradiation light over all phases and all wavelengths. FIG. 5B is a phase image 11, which is obtained by projecting sine wave pattern light (sine wave pattern light LT 1 and sine wave pattern light LT 2) onto the surface of a translucent object and obtaining a phase value obtained by a phase shift method. This is an image of reflected light (reflected light RTL1 or reflected light RTL2) obtained based on Φ (x, y).

FIG. 5 (c), a single scattering reflection image 19 is an image obtained based on direct reflection component L _d obtained by the phase shift method. 5 (d) is a multiple scattering reflection image 20 is an image obtained based on the indirect reflected components L _g obtained by the phase shift method.

  Returning to FIG. 3, the shape calculation unit 104 generates the phase image 11 of the phase value Φ (x, y) calculated by the imaging data analysis unit 103 for each different phase in the same projector (projector 3 or projector 4). . Here, when the sine wave pattern light (the chord wave pattern light LT1 and the sine wave pattern light LT2) is projected onto the translucent object 6, the shape calculation unit 104 includes a pixel of reflected light and a pixel of sine wave pattern light. The correspondence of is determined. At this time, in the sine wave pattern light, there is ambiguity in the repetition of one cycle of the sine wave (the accuracy of correspondence is low).

For this reason, the shape calculation unit 104 uses the phase image 11 of two types of sine wave pattern light having different phases for each of the reflected light RTL1 and the reflected light RTL2, and connects the phase images 11 having different phases.
Then, the shape calculation unit 104 determines the correspondence between the reflected light pixel and the sine wave pattern light pixel for each of the reflected light RTL1 and the reflected light RTL2, and outputs correspondence information indicating the corresponding pixels.
In addition, the shape calculation unit 104 obtains the three-dimensional shape (three-dimensional shape information) of the translucent object 6 by triangulation using the correspondence information of each pixel. That is, in the sine wave pattern light having two different phases, the shape calculation unit 104 connects each phase and takes correspondence between each pixel projected from the projector and a pixel imaged by the imaging unit 2. Then, the shape calculation unit 104 calculates the three-dimensional shape of the translucent object 6 from the correspondence information between each pixel projected from the projector and each pixel of the reflected light RTL1 and the reflected light RTL2, using a triangulation method. obtain.

  FIG. 6 is a diagram illustrating a process of generating a normal line of the normal line calculation unit 105 in the present embodiment. FIG. 6A shows a first-order scattered reflection image 19_1 and a light ray image 13_1 obtained from the spectral imaging data captured by the projector 3. FIG. 6B shows a primary scattered reflection image 19_2 and a light ray image 13_2 obtained from the spectral imaging data captured by the projector 4. FIG. 6C shows a primary scattered / reflected image 19_1 of the projector 5 (not shown), a primary scattered / reflected image 19_3, and a light image 13_3 of the projector 4 when three projectors are provided.

  FIG. 6D is a normal image 40 showing the normal line of the translucent object 6 calculated from the shadow information of the primary scattered reflection image 19_1, the primary scattered reflection image 19_2, and the primary scattered reflection image 19_3. . Here, each of the light image 13_1, the light image 13_2, and the light image 13_3 is a light vector in units of pixels obtained in advance from the geometric calibration of the projector, and for example, a direction vector is stored in an RGB value. Here, the light ray image is an image indicating from which direction each image of the projector is irradiated to each pixel of the captured image. In other words, the data structure of this light image is such that the XYZ values indicating the direction of the light vector of each pixel in the image are associated with each pixel as image data instead of the RGB value of each pixel in the image. It has become.

  Returning to FIG. 3, the normal calculation unit 105 refers to each of the light ray image 13_1 and the light ray image 13_2, and each pixel of the primary scattered reflection image 19_1 and the primary scattered reflection image 19_2 having different shadows and the light ray image. The positional relationship with each pixel of 13_1 and light image 13_2 is specified.

  Then, the normal line calculation unit 105 can obtain the normal line of the translucent object by the illuminance difference stereo method from the shadow information of each pixel of the primary scattered reflection image 19_1 and the primary scattered reflection image 19_2. In the illuminance difference stereo method, projectors having two or more locations are used, and two or more pieces of imaging data having different shadows are used. Therefore, in the present embodiment, the projector 3 and the projector 4 having different arrangement locations are used, and the normal is obtained using each of the primary scattered reflection image 19_1 and the primary scattered reflection image 19_2 having different shadows.

Since the illuminance difference stereo method is a method assuming a diffuse reflection object or a specular reflection object, it is impossible to obtain a normal line of a semi-transparent object from normal reflected light imaging data.
Therefore, in the present embodiment, the normal can be estimated by using the first-order scattered reflection image 19 obtained by the equation (2) from the spectral imaging data obtained by imaging the reflected light.

  That is, the primary scattered reflection image 19 used in the present embodiment is reflected light that is reflected at a shallow portion of the object, as already described. Therefore, the deviation between the incident position of the incident light and the emission position of the reflected light is so small as to be negligible, and can be regarded as the same as the diffuse reflection component in the reflected light from the diffuse reflection object used in the illuminance difference stereo method.

Further, the multiple scattering reflection image 20 can be regarded as an image of an indirect reflection component in a reflection component from a translucent object. However, this indirect reflection image is an image in a state where the light irradiated to each point on the object surface is scattered and convolved.
Therefore, in order to estimate the scattering characteristics at each point of the translucent object from this convolved image, it is assumed that scattering can be expressed by an arbitrary function (multiple scattering function) such as a point spread function. To do. Based on this assumption, the multiple scattering function at each point of the object can be obtained by estimating with an optimization method.

In the illuminance difference stereo method, the normal line calculation unit 15 uses the shadow information obtained from the primary scattered reflection image 19_1 or the primary scattered reflection image 19_2 and the corresponding ray image 13_1 or ray image 13_2 to detect the surface of the object. A primary scattered reflection image of the image albedo from which the shadow is removed is calculated. Here, the image albedo is data indicating the ratio of the total amount of reflected light to the total amount of incident light in each pixel.
The scattering calculation unit 16 uses the multiple scattered reflection image 20 calculated by the imaging data analysis unit 103 to obtain a function indicating the scattering characteristic for each pixel of the multiple scattered reflection image 20 as described below.
The multiple scattered reflection image 20 is an image in which the intensity of light scattered from an arbitrary point on the object surface is spatially convoluted on the object surface. Since the scattering characteristic under the surface of the sine wave pattern light irradiated to an arbitrary point is expressed, it can be generally expressed by a function called BSSRDF. That is, the scattering calculation unit 16 obtains a scattering function for each pixel that can reproduce the multiple scattered reflection image 20 by numerical calculation.

FIG. 7 is a diagram illustrating an example of a BSSRDF function graph. In FIG. 7, the horizontal axis indicates the scattering propagation distance (mm) from the pixel on which light is incident, and the vertical axis indicates the scattering intensity R (d).
The curve of the BSSRDF function in FIG. 7 shows that the scattering intensity decreases as the propagation distance due to scattering increases. By estimating the BSSRDF function, it is possible to express blur (light bleeding state in the object) when light is incident on the translucent object.

  Returning to FIG. 3, the scattering calculation unit 16 approximates the BSSRDF function by the following equation (3), and expresses blur (light bleeding state in the object) when the light enters the translucent object. To do. The following equation (3) is a dipole approximation model equation that approximates the BSSRDF function.

In the above equation (3), σ _s is a parameter specific to the object and a scattering coefficient. σ _a is an object-specific parameter and is an absorption coefficient. Η is a parameter specific to the object and is a refractive index. d is a distance from a point (an arbitrary point described above) where light is incident. g is an inner product of the scattering direction by the phase function and the light propagation direction, and is “0” in the case of isotropic scattering.
In the case of a general object, if the refractive index η is known, multiple scattering can be measured by estimating the scattering coefficient and the absorption coefficient. For example, the scattering intensity in multiple scattering can be estimated by the maximum likelihood estimation method using the dipole approximation model equation (3).

  Next, the spectral data acquisition operation in the image data acquisition system 1 will be described with reference to the drawings. FIG. 8 is a flowchart showing an operation example of acquiring spectral imaging data of the translucent object 6 by the image data acquisition system 1. In the following description, the sine wave pattern light LT1 from the projector 3 and the sine wave pattern light LT2 from the projector 4 are projected onto the translucent object 6 in the pattern light irradiation unit 7.

Step S1:
The light irradiation control unit 101 sets the wavelengths λ of the sine wave pattern light LT1 and the sine wave pattern light LT2 for each of the projector 3 and the projector 4, and advances the process to step S2.
Here, the light irradiation control unit 101 sequentially sets each of the wavelengths λ1, λ2, λ3,..., Λn in order to capture the spectral imaging data of “color” set in advance. For example, the magnitude relationship between the wavelengths is λ1 <λ2 <λ3 <. The light irradiation control unit 101 changes the wavelength λ1 in the first flow, the wavelength λ2 in the second flow chart,...

Step S2:
Next, the light irradiation control unit 101 sets whether to use the projector 3 or the projector 4, and advances the process to step S3.
In the present embodiment, for example, when the image data acquisition system 1 is operated, the projector 3 is first set (initial setting). In the subsequent flow, the setting is changed to the projector 4 for the projector 3, while the setting is changed to the projector 4 for the projector 4.

Step S3:
The light irradiation control unit 101 sequentially irradiates the set projector with a sine wave pattern in three different phases. For example, if the projector 3 is selected as the projector, the light irradiation control unit 101 causes the projector 3 to project the sine wave pattern LT1 onto the translucent object 6 with three different phases. On the other hand, if the projector 4 is selected as the projector, the light irradiation control unit 101 causes the projector 4 to project the sine wave pattern LT2 on the semitransparent object 6 with three different phases.

When the projector 3 is selected, the projector 3 sequentially changes the phase (three different phases) in a predetermined cycle, and projects the sine wave pattern LT1 onto the translucent object 6.
On the other hand, when the projector 4 is selected, the projector 4 sequentially changes the phase (three different phases) in a predetermined cycle, and projects the sine wave pattern LT2 onto the translucent object 6.

In any case, the light irradiation control unit 101 sequentially irradiates the set projector with a sine wave pattern in three different phases.
Further, when the light irradiation control unit 101 irradiates each projector with a sine wave pattern, the light irradiation control unit 101 adds an identification number of the projector to be irradiated, phase information indicating a phase at the time of imaging, and wavelength information indicating a wavelength λ to be dispersed. The identification number, phase information, and wavelength information are output to the imaging unit 2.

The imaging control unit 102 causes the imaging unit 2 to capture the reflected light from the translucent object 6 based on the projector irradiation control signal output from the light irradiation control unit 101 and the phase control signal for changing the phase.
Here, when the projector 3 projects the sine wave pattern LT1 onto the semitransparent object 6, the imaging control unit 102 acquires from the imaging unit 2 spectral imaging data obtained by imaging the reflected light RTL1 from the semitransparent object 6. . Then, the imaging control unit 102 adds the identification information of the projector 3, phase information indicating the phase at the time of imaging, and wavelength information to the acquired spectral imaging data, and writes and stores them in the storage unit 110.

On the other hand, when the projector 3 projects the sine wave pattern LT2 on the semitransparent object 6, the imaging control unit 102 acquires the spectral imaging data obtained by imaging the reflected light RTL2 from the semitransparent object 6 from the imaging unit 2. Then, the imaging control unit 102 adds the identification information of the projector 4, phase information indicating the phase at the time of imaging, and wavelength information to the acquired spectral imaging data, and writes and stores them in the storage unit 110.
The light irradiation control unit 101 causes the projector to irradiate sine wave patterns having three different phases, and then proceeds to step S4.

Step S4:
The light irradiation control unit 101 determines whether or not to change the projector that irradiates the semi-transparent object 6 with the sine wave pattern light.
At this time, if the currently selected projector is the projector 3, the light irradiation control unit 101 advances the process to step S2 in order to change the projector.
On the other hand, if the currently selected projector is the projector 4, the light irradiation control unit 101 does not change the projector, and thus advances the process to step S5.

Step S5:
The light irradiation control unit 101 determines whether or not to change the wavelength of the sine wave pattern light applied to the translucent object 6. That is, the light irradiation control unit 101 determines whether or not the currently selected wavelength λ is the wavelength λn. If the wavelength λ is the wavelength λn, it is assumed that all the wavelength changes have been completed.

At this time, if the currently selected wavelength λ is not the wavelength λn, the light irradiation control unit 101 advances the process to step S1 in order to change to the next largest wavelength λ.
On the other hand, if the currently selected wavelength λ is the wavelength λn, the light irradiation control unit 101 determines that the imaging processing for all the preset wavelengths λ has been completed, and advances the processing to step S1. .

  Next, the operation of the image analysis processing of the translucent object in the image data acquisition system 1 will be described with reference to the drawings. FIG. 9 is a flowchart illustrating an operation example of the image analysis processing of the translucent object in the image data acquisition system 1.

Step S11:
The imaging data analysis unit 103 sets the wavelength λ (the wavelength of the sine wave pattern light LT1 or the sine wave pattern light LT2) of the spectral data read from the storage unit 110, and advances the process to step S12.
Here, the imaging data analysis unit 103 sequentially sets each of the wavelengths λ1, λ2, λ3,..., Λn in order to read the spectral imaging data of “color” set in advance. For example, the magnitude relationship between the wavelengths is λ1 <λ2 <λ3 <. The light irradiation control unit 101 changes the wavelength λ1 in the first flow, the wavelength λ2 in the second flow chart,...

Step S12:
Next, the imaging data analysis unit 103 sets which spectral imaging data of the projector 3 or the projector 4 is read from the storage unit 110, and advances the process to step S13.
In the present embodiment, for example, when the image data acquisition system 1 is operated, the projector 3 is first set (initial setting). In the subsequent flow, the setting is changed to the projector 4 for the projector 3, while the setting is changed to the projector 4 for the projector 4.

Step S13:
Based on the wavelength information of the wavelength λ set in step S11 and the identification numbers of the projectors (projector 3 and projector 4) set in step S12, the imaging data analysis unit 103 converts the spectral imaging data having three phases different from each other. The data is read from the storage unit 110 together with the phase information.
Then, the imaging data analysis unit 103 separates the spectral imaging data read out for each of the projector 3 and the projector 4 into a direct reflection component and an indirect reflection component using Expression (2). At this time, the imaging data analysis unit 103 also calculates the phase value Φ (x, y) together with the direct reflection component (primary scattering reflection component 9) and the indirect reflection component (multiple scattering reflection component 10), and the process is performed in step S14. Proceed to
Here, the imaging data analysis unit 103 obtains the phase image from the phase value Φ, the primary scattered reflection image 19 from the direct reflection component (primary scattered reflection component 9), and the indirect reflection component (multiple scattered reflection component 10). A multiple scattering reflection image 20 is obtained.

Step S14:
The shape calculation unit 104 generates the phase image 11 of the phase value Φ (x, y) calculated by the imaging data analysis unit 103 for each different phase in the same projector (projector 3 or projector 4). Here, when the sine wave pattern light (the chord wave pattern light LT1 and the sine wave pattern light LT2) is projected onto the translucent object 6, the shape calculation unit 104 includes a pixel of reflected light and a pixel of sine wave pattern light. The correspondence of is determined.

Next, the shape calculation unit 104 uses the phase image 11 of the sine wave pattern light in which two types of phases overlap, and connects the phase image 11 having a different phase for each projector unit, and the reflected light pixel and the sine wave pattern light pixel And the correspondence information indicating the corresponding pixel is output.
Then, the shape calculation unit 104 obtains the three-dimensional shape of the translucent object 6 by triangulation using the correspondence information of each pixel.

Step S15:
The normal line calculation unit 105 performs geometric calibration of the camera and the projector in advance, calculates the direction of the light beam on the object, and stores it in the RGB value, thereby obtaining the spectral image data of the reflected light RTL1 and the reflected light RTL2. Corresponding light image 13_1 and light image 13_2 are generated.
The normal line calculation unit 105 refers to each of the light ray image 13_1 and the light ray image 13_2, and each pixel of the primary scattered reflection image 19_1 and the primary scattered reflection image 19_2 having different shadows, and the light ray image 13_1 and the light ray image 13_2. The positional relationship with each pixel is specified.
Then, the normal line calculation unit 105 obtains the normal line of the translucent object by the illuminance difference stereo method from the shadow information of each pixel of the primary scattered reflection image 19_1 and the primary scattered reflection image 19_2.

Step S16:
The imaging data analysis unit 103 determines whether to change the projector of the spectral imaging data to be read.
At this time, if the currently selected projector is the projector 3, the imaging data analysis unit 103 advances the process to step S12 in order to change the projector.
On the other hand, when the currently selected projector is the projector 4, the imaging data analysis unit 103 advances the process to step S17 because the projector is not changed.

Step S17:
The imaging data analysis unit 103 determines whether to change the wavelength of the spectral imaging data to be read. That is, the imaging data analysis unit 103 determines whether or not the currently selected wavelength λ is the wavelength λn. If the wavelength λ is the wavelength λn, it is assumed that all the wavelength changes have been completed.
At this time, if the currently selected wavelength λ is not the wavelength λn, the imaging data analysis unit 103 advances the process to step S11 in order to change to the next largest wavelength λ.
On the other hand, if the currently selected wavelength λ is the wavelength λn, the imaging data analysis unit 103 determines that imaging processing has been completed for all the wavelengths λ set in advance, and the process proceeds to step S18. .

Step S18:
The shape calculation unit 104 uses the three-dimensional shape data of all wavelengths of wavelengths λ1, λ2, λ3,..., Λn, and generates an improved three-dimensional shape by, for example, the least square method.
Then, the shape calculation unit 104 advances the process to step S19.

Step S19:
The normal line calculation unit 105 uses the normal data of all wavelengths of wavelengths λ1, λ2, λ3,..., Λn, and generates improved normal data by, for example, the least square method.
Then, the normal line calculation unit 105 advances the process to step S20.

Step S20:
Next, the normal line calculation unit 15 uses, for example, the shadow information obtained from the primary scattered reflection image 19_1 or the primary scattered reflection image 19_2 of the projector 3 and the projector 4 and the corresponding ray image 13_1 or ray image 13_2. First-order scattered reflection information (image albedo first-order scattered reflection image) of the image albedo without the shadow (with the shadow removed) is generated.

Step S21:
For example, the scattering calculation unit 106 generates a dipole approximation model formula for each wavelength from the average of the pixel values of the multiple scattered reflection image 20 for each wavelength of the projector 3 and the projector 4 using the formula (3) already described. . Here, generating the dipole approximation model formula for each wavelength is the estimation of the multiple scattering characteristics.
The scattering calculation unit 106 generates a multiple scattering spectroscopic image corresponding to the multiple scattering reflection image 20 of each wavelength by the generated dipole approximate model formula.

  As described above, according to the present embodiment, the analysis target is a translucent object, and the reflected light of the irradiated pattern light includes diffuse reflection (multiple scattered reflection) (even if mixed). The reflected light is separated into a primary scattering reflection component 9 and a multiple scattering reflection component 10. By separating the reflection components, according to the present embodiment, it is possible to estimate the multiple scattering characteristic (BSSRDF) of the translucent object.

In addition, according to the present embodiment, since the three-dimensional shape, normal, and multiple scattering characteristics (BSSRDF) of a semi-transparent object can be estimated only from the reflected light of the sine wave pattern light reflected from the semi-transparent object, As described above, it is possible to easily analyze a translucent object without using a plurality of measuring devices.
In addition, according to the present embodiment, rendering is performed, and all the data of the multiple scattering characteristics by the shape, normal line, and spectrum of the translucent object used when generating the CG image of the translucent object can be obtained with a single system. It can be obtained collectively by irradiating only one sequence pattern.
That is, according to the present embodiment, even for a translucent object, its shape, normal information, and scattering characteristics can be simply measured with spectral information. Even when a diffuse reflection object and a translucent object coexist, it is possible to measure the characteristics of both by determining the area of each reflection component. Furthermore, it is possible to reproduce realistically with CG using these.

  It should be noted that the program for realizing the function of the measurement processing unit 5 in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to be translucent. You may perform the analysis process of the scattering of the light in an object. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 Image data acquisition system 2 Imaging part 3, 4 Projector 5 Measurement processing part 6 Translucent object 7 Pattern light irradiation part 100 Control part 101 Light irradiation control part 102 Imaging control part 103 Imaging data analysis part 104 Shape calculation part 105 Normal calculation Unit 106 scattering calculation unit 107 operation unit 108 display unit 109 external interface 110 storage unit

Claims (5)

An image data acquisition system that acquires three-dimensional shape information of a surface of a translucent object, surface normal information, primary scattering characteristics with shadows removed, and multiple scattering characteristics;
A pattern light irradiating unit that irradiates the translucent object with pattern light of two or more different phases;
An imaging unit that spectrally reflects reflected light from the surface of the translucent object irradiated with pattern light and acquires spectral imaging data for each predetermined wavelength;
A phase component , a primary scattering reflection component for determining the normal information and the primary scattering characteristic, and a multiple scattering for determining the multiple scattering characteristic from the spectral imaging data for each wavelength by a predetermined arithmetic expression. An imaging data analysis unit that separates each of the reflection components and generates a phase image, a primary scattered reflection component image, and a multiple scattered reflection component image;
Based on the phase image, a shape calculation unit for obtaining three-dimensional shape information of the surface of the translucent object by a phase shift method,
A normal calculation unit for obtaining the normal information from the primary scattered reflection component images by two or more different projectors;
A scattering calculation unit that obtains a multiple scattering characteristic represented by a predetermined function from the multiple scattered reflection component image. The scattering calculation unit is
The image data acquisition system according to claim 1, wherein the predetermined function indicating a multiple scattering characteristic for each pixel is obtained from the multiple scattered reflection component image. The normal calculation unit is
Finding the shade information of the translucent object from the primary scattered reflection component image obtained from each of the two or more pattern lights and a light ray image representing the direction from two or more projector pixels to the object surface, The image data acquisition system according to claim 1 or 2, wherein a normal line of the translucent object and a first-order scattering characteristic of the image albedo are obtained from the shadow information. The shape calculation unit is
Two or more kinds of pixels of the phase image having different phases for each pattern light and the pixels of the pattern light projected from the projector are matched, and the three-dimensional of the translucent object is obtained by triangulation The image data acquisition system according to any one of claims 1 to 3, wherein the shape is estimated. An image data acquisition method for acquiring three-dimensional shape information on a surface of a translucent object, surface normal information, primary scattering characteristics with shadows removed, and multiple scattering characteristics,
A pattern light irradiation process of irradiating the semi-transparent object with two or more kinds of different phase pattern lights;
An imaging process for spectroscopically reflecting reflected light from the surface of the translucent object irradiated with pattern light and obtaining spectral imaging data for each predetermined wavelength;
A phase component , a primary scattering reflection component for determining the normal information and the primary scattering characteristic, and a multiple scattering for determining the multiple scattering characteristic from the spectral imaging data for each wavelength by a predetermined arithmetic expression. An imaging data analysis process for separating each of the reflection components and generating each of a phase image, a primary scattered reflection component image, and a multiple scattered reflection component image;
Based on the phase image, a shape calculation process for obtaining three-dimensional shape information of the surface of the translucent object by a phase shift method,
Normal calculation process for obtaining the normal information from the primary scattered reflection component image by two or more different projectors, and scattering for obtaining a multiple scattering characteristic represented by a predetermined function from the multiple scattered reflection component image An image data acquisition method comprising: an arithmetic process.