KR101865886B1 - Method and system for estimating surface geometry and reflectance - Google Patents

Method and system for estimating surface geometry and reflectance Download PDF

Info

Publication number
KR101865886B1
KR101865886B1 KR1020160167526A KR20160167526A KR101865886B1 KR 101865886 B1 KR101865886 B1 KR 101865886B1 KR 1020160167526 A KR1020160167526 A KR 1020160167526A KR 20160167526 A KR20160167526 A KR 20160167526A KR 101865886 B1 KR101865886 B1 KR 101865886B1
Authority
KR
South Korea
Prior art keywords
reflectivity
geometry
estimating
function
estimator
Prior art date
Application number
KR1020160167526A
Other languages
Korean (ko)
Inventor
권인소
최경민
Original Assignee
한국과학기술원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한국과학기술원 filed Critical 한국과학기술원
Priority to KR1020160167526A priority Critical patent/KR101865886B1/en
Application granted granted Critical
Publication of KR101865886B1 publication Critical patent/KR101865886B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/48—Thermography; Techniques using wholly visual means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/55—Specular reflectivity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation

Abstract

Disclosed is a system for estimating a geometry and a reflectivity operated by at least one processor, wherein the system including: a geometry estimating unit for estimating an initial geometry of the object by receiving infrared images obtained by photographing an object after varying a light source direction and a viewpoint, and by applying optical stereo to the near infrared images; and a reflectivity estimating unit for repeating a procedure that a reflectivity function of the object is estimated based on the geometry of the object estimated by the geometry estimating unit, and the reflectivity function of the object is inputted to the geometry estimating unit a predetermined number of times, wherein the geometry estimating unit re-estimates the geometry of the object based on the reflectivity function received from the reflectivity estimating unit, and the geometry estimating unit and the reflectivity estimating unit each receive and transmit the estimated results for the predetermined number of times, and simultaneously outputs the final geometry and final reflectivity of the object.

Description

[0001] METHOD AND SYSTEM FOR ESTIMATING SURFACE GEOMETRY AND REFLECTANCE [0002]

The present invention relates to the geometry of an object representation and the estimation of reflectivity.

The Bidirectional reflectance distribution function (BRDF) (simply called the reflectivity or reflectance function) of an object can be obtained by parametric and analytic methods. Parametric methods are mathematical modeling of optical models, and analytic methods are actually experiments on several objects to calculate the model. The parametric method has a characteristic that it is necessary to make a model covering most of the objects well, and the analytic method has a characteristic of selecting several objects and calculating them experimentally based on the object.

Representative existing techniques of analytic methods are described in Merl (KJ Dana, B. Van Ginneken, SK Nayar, and JJ Koenderink, Reflectance and texture of real-world surfaces, ACM Transactions on Graphics (TOG), 1999) and Curet (W. Matusik, H. Pfister, M. Brand, and L. McMillan, A data-driven reflectance model, ACM Transactions on Graphics (TOG), 2003). Merl measures the reflectivity by applying paint of various materials to the smooth spherical object surface. Assume that the surface normal vector is known since it is assumed to be a perfect sphere form. However, it is very difficult to obtain data samples because Merl has limited paint types that can be applied to smooth spherical object surfaces and paint must be applied evenly. Curet acquires samples of various geometries, usually not in sphere form, but usually in daily life, assuming that the surface is plane, without calculating the surface normal vector, and averages the reflectance values obtained from the surface. However, since most objects in the real world look like planar geometries, Curet has limitations.

On the other hand, the method of measuring the reflectance of an object up to now has been mainly studied in the visible light band. However, the method of measuring the reflectance using the visible light image is very inconvenient because the experiment must be performed in a dark room environment where all indoor lighting is turned off.

In recent years, near-infrared imaging has gained considerable popularity with the introduction of three-dimensional depth sensors such as the Microsoft Kinect sensor. Referring to FIG. 1, the near infrared ray image has a smaller pattern of the object surface than a general visible light image, which is a great advantage in the optical computer vision technique. In addition, most indoor lighting emits light in the visible range, so if you use a near-infrared camera, you can take a strong picture regardless of changes in the lighting in the room. Despite the advantages of such a near infrared ray camera, there has not been studied a method of finding a fine three-dimensional surface geometry of an object from a near infrared ray image and measuring the reflectance.

The object of the present invention is to provide a method and system for optically estimating a fine three-dimensional surface geometry and reflectivity of an object using a near infrared ray camera as an analytic method.

There is provided a system for estimating a geometry and a reflectivity of an object to be operated by at least one processor according to an embodiment of the present invention. The system includes a near-infrared (IR) Estimating a reflectivity function of the object from the geometry of the object estimated by the geometry estimator and inputting a reflectivity function of the object to the geometry estimator; Wherein the geometric structure estimating unit re-estimates the geometry of the object based on a reflection function received from the reflectivity estimating unit, and the geometry estimating unit and the reflectivity estimating unit calculate When the results of the estimation for the predetermined number of times are exchanged with each other And it outputs the final geometry of the final reflectance of the object.

The reflectivity estimator may calculate a reflectivity value for each pixel observed for each pixel based on the initial geometry and estimate an initial reflectivity function of the object by fitting the reflectivity value for each pixel according to the angle of the surface normal vector.

The reflectivity estimator may fit the pixel-by-pixel reflectivity value using an angle parameterized with a half-vector of the incident angle and the reflection angle in half angle coordinates.

The geometry estimator may obtain a surface normal vector that minimizes the energy smoothing term associated with the difference between the observed intensity, the geometry, and the intensity of the intensity rendered with the estimated reflectivity and the difference of the neighboring surface normal vectors.

The system includes a database unit for mapping a reflectivity function of the object estimated by the reflectivity estimating unit to a type of a material constituting the object and storing the mapped target near infrared ray image, Dimensional model estimating unit for extracting a reflectivity value of a material of the random object from the reflectance value of the random object and estimating the surface normal vector from the reflectance value of the extracted random object to estimate the geometry of the random object.

The 3D model estimator may estimate a surface normal vector from the reflectivity of the arbitrary object by using a Shape from Shading method, assuming a uniform albedo.

According to another embodiment of the present invention, there is provided a method of estimating the geometry and reflectivity of an object by means of at least one processor, the method comprising the steps of varying a light source direction and a viewpoint to apply an optical stereo to near- Estimating a surface normal vector map, estimating an initial reflectivity function of the object from the surface normal vector map, and estimating a geometric shape of the object from the initial reflectivity function using a Shape from Shading method. Updating the structure and updating the reflectivity function of the object from the updated geometry by repeating a predetermined number of times to obtain the final geometry and final reflectivity of the object.

In estimating the initial reflectivity function, an initial reflectivity function of the object can be estimated by fitting a reflectivity value of each pixel observed for each pixel according to an angle of each surface normal vector.

The step of estimating the initial reflectivity function may fit the per-pixel reflectivity value using an angle parameterized with an incident angle and a half vector of the reflection angle in half angle coordinates.

The obtaining of the final geometry and the final reflectivity may include calculating a reflectivity function of the object according to the direction of the light source and mapping the reflectivity function of the object to the type of the material constituting the object and storing the result in a database.

The method includes the steps of receiving a target near infrared ray image, confirming a material and a light source direction constituting an arbitrary object included in the target near infrared ray image, calculating a reflectance value corresponding to a material constituting the arbitrary object and a light source direction And estimating a geometric structure of the arbitrary object by estimating a surface normal vector from the reflectivity value of the arbitrary object.

According to another embodiment of the present invention, there is provided a system for estimating a geometry and a reflectivity of at least one processor, the system comprising: a database unit for storing an object-based reflectivity function estimated from near infrared rays images obtained by photographing a plurality of sample objects; Infrared ray image from the target near-infrared ray image, estimates a surface normal vector of the target object by referring to a reflectivity value corresponding to the target object in the database unit, And a three-dimensional model estimating unit for obtaining the geometry of the target object. The reflectance function for each object is estimated from the initial geometry of each sample object, and the initial geometry of each sample object is estimated from the near-infrared images of each sample object.

The reflectivity function can be obtained from near-infrared images obtained by photographing each sample object by varying a light source direction and a view angle.

The object-specific reflectivity function can be obtained by repeating a first procedure for estimating the geometry from the estimated current reflectivity function and a second procedure for estimating the reflectivity function from the estimated current geometry.

According to the embodiment of the present invention, it is possible to accurately estimate the fine geometry and the reflectivity of the surface of an object using a near-infrared ray image robust to changes in the room illumination. According to the embodiment of the present invention, the near infrared ray image can be easily and inexpensively photographed even in a dark room environment, which is highly usable.

1 is a view for comparing a visible light image with a near infrared ray image.
2 is a configuration diagram of a geometry and reflectivity estimating system according to an embodiment of the present invention.
3 is an illustration of an installation environment of the photographing apparatus according to an embodiment of the present invention.
4 is an illustration of various types of near infrared rays images taken for database construction according to an embodiment of the present invention.
5 is an illustration of a three-dimensional geometric model extracted from a near-infrared image according to an embodiment of the present invention.
6 is a view for explaining a coordinate system used for estimation of reflectivity according to an embodiment of the present invention.
7 is an illustration of a reflectivity function for an object estimated from a near-infrared image according to an embodiment of the present invention.
8 is a flowchart of a method of constructing a reflectivity database according to an embodiment of the present invention.
9 is a flowchart of a database-based three-dimensional model estimation method according to an embodiment of the present invention.
10 is an example of a database-based three-dimensional model estimation result according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, " " module, " and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

FIG. 2 is a configuration diagram of a geometry and reflectivity estimating system according to an embodiment of the present invention, FIG. 3 is an example of an installation environment of the photographing apparatus according to an embodiment of the present invention, and FIG. FIG. 5 is an illustration of a three-dimensional geometric model extracted from a near-infrared image according to an embodiment of the present invention.

Referring to FIG. 2, the geometry and reflectivity estimation system (simply referred to as a system) 10 includes a photographing unit 100 for photographing sample objects with a near-infrared camera, and a near- A three-dimensional model estimating unit 300 for estimating a three-dimensional model of the target object photographed based on the reflectivity of the sample objects estimated by the database building unit 200; ). Specifically, the database construction unit 200 includes a geometry estimating unit 210, a reflectivity estimating unit 230, and a database unit 250. The system 10 includes at least one processor and at least one memory, and uses a processor and memory to drive a program that implements the operations described in the present invention.

The photographing unit 100 includes a near infrared ray camera 110 for photographing an object and a plurality of infrared LED lights 130 installed around the object. Referring to Figure 3 (a), the illuminations can be set up to provide, for example, twelve different light source orientations. The position of the reflected lights on the chrome ball is shown in FIG. 3 (b). Referring to FIG. 3 (c), the object is photographed at different angles, for example, at nine different angles (viewpoints) in one light source direction. In this case, the photographing unit 100 can capture a total of 108 near-infrared images (= 12 light source changes * 9 different viewpoint changes) for each sample object. The near-infrared image shows much less texture than the visible image (see FIG. 1). It is assumed that the image has a resolution of 1600 pixels by 1200 pixels.

4, the types of the sample objects photographed by the photographing unit 100 include various textile materials such as cotton, polyester, leather, and satin, organic materials such as leaves and wood stumps, plastic materials such as stones, Materials and so on.

The database construction unit 200 estimates the geometry and reflectivity of each sample object on the basis of the near-infrared images of the sample object photographed by the photographing unit 100.

Referring to FIG. 5, the database construction unit 200 estimates a fine three-dimensional geometric structure based on a surface normal vector of each pixel acquired using an optical stereo (photometric stereo). An optical stereo is a technique of digitizing a three-dimensional shape of an object by using information obtained by irradiating illuminations to the object, and it is possible to acquire the surface normal information of the object with one camera while varying the illumination state. Since the specimens used in the present invention are not specular, the surface normal information can be obtained by applying the optical stereo method.

The database construction unit 200 estimates the reflectivity of the object surface using the geometry found as an initial value. The reflectivity estimating unit 230 can increase the accuracy of the geometry and the reflectivity by repeatedly performing the geometry estimation and the reflectivity estimation. Here, the reflectivity is a bidirectional reflectance distribution function (BRDF).

The database construction unit 200 stores the reflectivity of the sample objects in the database.

The three-dimensional model estimating unit 300 estimates a three-dimensional model of the image of the target object based on the reflectivity of the sample objects estimated by the database building unit 200. If the three-dimensional model estimating unit 300 knows the direction vector of the light source and the material constituting the target object, the three-dimensional model estimating unit 300 can extract the reflection characteristic of the target object from the reflectivity of various materials constructed in the database building unit 200. Then, the three-dimensional model estimating unit 300 estimates the shape based on the reflection characteristic of the target object by using a shape from shading method.

A concrete operation method of the database building unit 200 and the three-dimensional model estimating unit 300 will be described below.

FIG. 6 is a diagram illustrating a coordinate system used for estimating the reflectance according to an embodiment of the present invention, and FIG. 7 is an example of a reflectance function for an object estimated from a near-infrared image according to an embodiment of the present invention.

Referring to FIG. 2, the database construction unit 200 includes a geometry estimating unit 210, a reflectivity estimating unit 230, and a database unit 250.

The geometry estimation unit 210 obtains a surface normal vector for each pixel using an optical stereo. At this time, in order to estimate the initial geometry required for the reflectivity estimation, the geometry estimator 210 calculates the surface normal vector of each pixel by applying the optical stereo method, assuming that the object surface has a Lambertian reflectance.

First, the pixel brightness (intensity, I) is calculated from the reflectivity (BRDF)

Figure 112016120924739-pat00001
, The surface normal vector N x , and the intensity L of the light source. Reflectivity
Figure 112016120924739-pat00002
Is a specific substance
Figure 112016120924739-pat00003
Incident angle < RTI ID = 0.0 >
Figure 112016120924739-pat00004
And the outgoing angle
Figure 112016120924739-pat00005
.

Figure 112016120924739-pat00006

Figure 112016120924739-pat00007
Is 1 and Lambersian is assumed, the equation (1) is summarized as the equation (2). In Equation (2)
Figure 112016120924739-pat00008
Is albedo, N is the set of surface normal vectors N x (
Figure 112016120924739-pat00009
), S is the incident angle
Figure 112016120924739-pat00010
.

Figure 112016120924739-pat00011

The geometry estimator 210 solves Equation (2) with respect to N to obtain a surface normal vector for each pixel. The geometry estimator 210 obtains a surface normal vector map based on the pixel-by-pixel surface normal vector, and estimates the three-dimensional geometry of the object from the surface normal vector map. The geometry estimator 210 may obtain a three-dimensional geometric model of the sample objects as shown in FIG.

The reflectivity estimating unit 230 estimates the reflectance of the object surface using the geometry found in the initial value in the geometry estimating unit 210. [

First, the reflectivity estimating unit 230 fits the surface reflectance according to the angles of the surface normal vector. At this time, the reflectivity estimator 230 maps the observed pixel values to parameterized angles using half angle coordinates. That is, the reflectivity estimating unit 230 calculates the reflectivity of the incident light,

Figure 112016120924739-pat00012
And reflection angle
Figure 112016120924739-pat00013
Instead of the standard coordinate system for (b). Angle of incidence in standard coordinate system
Figure 112016120924739-pat00014
And reflection angle
Figure 112016120924739-pat00015
Is expressed by the following equation (3): < EMI ID = 3.0 >
Figure 112016120924739-pat00016
,
Figure 112016120924739-pat00017
) For the function [
Figure 112016120924739-pat00018
]. In the semi-angular coordinate system, h (half angle)
Figure 112016120924739-pat00019
And reflection angle
Figure 112016120924739-pat00020
Is a half vector of < / RTI >

Figure 112016120924739-pat00021

Figure 112016120924739-pat00022
,
Figure 112016120924739-pat00023

The reflectivity estimating unit 230 estimates the reflectivity Normalizes the original near infrared ray image I with the reflectance value I for all pixels i as shown in Equation 4,

Figure 112016120924739-pat00025
.

Figure 112016120924739-pat00026

The reflectivity estimator 230 converts the pixel-by-pixel reflectivity value into a simple low-dimensional parameter model based on the semi-angular coordinate system described in Equation (3). The reflectivity estimating unit 230

Figure 112016120924739-pat00027
For each slice of < RTI ID = 0.0 >
Figure 112016120924739-pat00028
. When the camera is fixed and the object rotates at 9 different viewpoints, 108 x 1200 x 1600
Figure 112016120924739-pat00029
And nine
Figure 112016120924739-pat00030
Can be obtained.
Figure 112016120924739-pat00031
Can be calculated as shown in Equation (5). In Equation 5,
Figure 112016120924739-pat00032
,
Figure 112016120924739-pat00033
,
Figure 112016120924739-pat00034
Can be found with the RANSAC algorithm.

Figure 112016120924739-pat00035

The reflectivity estimating unit 230 and the geometry estimating unit 210 update the reflectivity and the geometry while exchanging the estimated results with each other. The reflectivity estimator 230 estimates the initial reflectivity from the geometry found as the initial value, the geometry estimator 210 obtains the optimal normal vector from the estimated reflectivity, and the reflectivity estimator 230 uses the updated normal vector To obtain the final geometry and the final reflectivity.

The geometry estimator 210 finds a surface normal vector N * that minimizes the energy based on the cost function as shown in Equation (6). In Equation (6), the data term E p and the smoothing term E s are expressed by Equation (7). The data terms E p are modeled to minimize the difference in intensity that is rendered with the geometry and the estimated reflectivity based on the observed pixel intensities (I i ) and Equation (1). The smoothing term E s is modeled to minimize the difference between neighboring normal vectors.

Figure 112016120924739-pat00036

Figure 112016120924739-pat00037

Figure 112016120924739-pat00038

In Equation (7), the reflectivity

Figure 112016120924739-pat00039
Is defined as shown in Equation (5), and M i is a neighboring pixel of the pixel i.

The geometry estimator 210 estimates the updated normal vector N t from the previous normal vector N t - 1 based on the normal vector N * that minimizes the cost function of the current state . The reflectivity estimating unit 230 fits the reflectivity according to the angle of the updated normal vector N t .

Figure 112016120924739-pat00040

The database unit 250 stores a reflectivity function of the sample objects. The reflectivity function is estimated and stored for each light source direction.

Referring to FIG. 7, (a) is a near-infrared ray image of the sample objects, and (b) is a surface normal vector map calculated by the geometry estimating unit 210 from the near-infrared ray image. (c) is a three-dimensional geometry estimated from a surface normal vector map. (d) is albedo. (e)

Figure 112016120924739-pat00041
= 34 degrees.
Figure 112016120924739-pat00042
) < / RTI > (x-axis)
Figure 112016120924739-pat00043
] (y axis). (e) is a red curve, and the observed value is indicated by blue points.

8 is a flowchart of a method of constructing a reflectivity database according to an embodiment of the present invention.

Referring to FIG. 8, the database building unit 200 receives near infrared rays images of a sample object (S110). The database building unit 200 receives the near infrared ray image sets photographed at different angles (different viewpoints) from different light source directions for each sample object.

The database construction unit 200 estimates a surface normal vector map (geometry) for each sample object from near-infrared images using optical stereo (S120).

The database construction unit 200 obtains an initial BRDF by fitting the reflectivity according to the angles of the surface normal vector in the anti-angular coordinate system (S130). The database construction unit 200 acquires the specific

Figure 112016120924739-pat00044
in
Figure 112016120924739-pat00045
Reflectivity
Figure 112016120924739-pat00046
. Reflectivity function
Figure 112016120924739-pat00047
Is estimated in a curve form as shown in FIG. 7 (e).

The database construction unit 200 repeatedly performs the geometric estimation and the reflectivity estimation based on the initial reflectivity function to acquire the final geometry and the final reflectivity function for each sample object (S140). The database construction unit 200 estimates the geometry using a shape estimation method based on optical stereo based shading from the reflectivity function and updates the reflectivity function by fitting the reflectivity according to the angle of the surface normal vector from the estimated geometry Repeat the procedure a certain number of times.

FIG. 9 is a flowchart of a database-based three-dimensional model estimation method according to an embodiment of the present invention, and FIG. 10 is an example of a database-based three-dimensional model estimation result according to an embodiment of the present invention.

Referring to FIG. 9, the 3D model estimating unit 300 receives a near infrared ray image (S210).

The three-dimensional model estimating unit 300 confirms the direction and the direction of the light source constituting the target object included in the near-infrared image (S220).

The three-dimensional model estimating unit 300 obtains a reflectance value corresponding to the material constituting the target object and the light source direction in the reflectance function of various materials constructed in the database building unit 200 (S230). When the materials constituting the target object are various, the three-dimensional model estimating unit 300 obtains reflectance values corresponding to the respective parts.

The three-dimensional model estimating unit 300 finds a surface normal vector that minimizes the energy based on the reflectivity value (S240). The three-dimensional model estimating unit 300 can find the surface normal vector N * that minimizes the energy using Equations (6) and (7).

The three-dimensional model estimating unit 300 generates a three-dimensional model of the target object based on the surface normal vector (S250).

As described above, the 3D model estimating unit 300 is applied to a shape estimation (Shape from Shading) method for estimating a 3D model when an arbitrary object is photographed using a reflectance database for 100 objects constructed can do. Near-infrared images have geometric edges such as wrinkles because they are expressed as uniform texture unlike visible light images. Therefore, it is possible to apply the shape estimation method from the shadow assuming a uniform albedo in the near-infrared image.

Referring to FIG. 10, the target object photographed by the near-infrared camera can be composed of various materials, for example, cotton and polyester blend sweater, denim pants, 100% cotton check shirt, 100% , Polyester, cotton, rayon blend can be photographed.

10, the first row is a visible light image, and the second row is a near infrared ray image. The third column is a normal vector map obtained from the near infrared ray image and the fourth column is a three dimensional model estimated based on the reflectance function of various materials constructed in the database building unit 200.

As described above, according to the embodiment of the present invention, it is possible to precisely estimate the fine geometry and the reflectivity of the surface of an object by using a near-infrared ray image which is robust against changes in the room illumination. According to the embodiment of the present invention, the near infrared ray image can be easily and inexpensively photographed even in a dark room environment, which is highly usable.

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (14)

A geometry and reflectivity estimation system operating by at least one processor,
A geometry estimator for estimating an initial geometry of the object by applying an optical stereo to the near-infrared images, receiving near-infrared images obtained by photographing an object by changing a direction and a direction of a light source, and
And a reflectivity estimator for estimating a reflectivity function of the object from the geometry of the object estimated by the geometry estimator and repeating a procedure of inputting a reflectivity function of the object to the geometry estimator a predetermined number of times,
The geometric structure estimating unit re-estimates the geometry of the object based on the reflection function received from the reflectivity estimating unit,
Wherein the geometric estimator and the reflectivity estimator output the final geometry and the final reflectivity of the object while exchanging the estimated results for the predetermined number of times,
The reflectivity estimator
And calculating an initial reflectivity value of each of the pixels based on the initial geometry, and estimating an initial reflectivity function of the object by fitting the reflectivity value of each pixel according to the angle of the surface normal vector. delete The method of claim 1,
The reflectivity estimator
Fitting the pixel-by-pixel reflectivity value using an angle parameterized with an incident angle and a half-vector of the reflection angle in half angle coordinates. A geometry and reflectivity estimation system operating by at least one processor,
A geometry estimator for estimating an initial geometry of the object by applying an optical stereo to the near-infrared images, receiving near-infrared images obtained by photographing an object by changing a direction and a direction of a light source, and
And a reflectivity estimator for estimating a reflectivity function of the object from the geometry of the object estimated by the geometry estimator and repeating a procedure of inputting a reflectivity function of the object to the geometry estimator a predetermined number of times,
The geometric structure estimating unit re-estimates the geometry of the object based on the reflection function received from the reflectivity estimating unit,
Wherein the geometric estimator and the reflectivity estimator output the final geometry and the final reflectivity of the object while exchanging the estimated results for the predetermined number of times,
The geometry estimator
A surface normal vector that minimizes the energy smoothing term associated with the difference between the observed intensity and the geometry and the intensity difference that is rendered with the estimated reflectivity and the difference of the neighbor surface normal vectors. A geometry and reflectivity estimation system operating by at least one processor,
A geometry estimator for estimating an initial geometry of the object by applying an optical stereo to the near infrared rays image, receiving a near infrared ray image obtained by photographing an object by varying a light source direction and a viewpoint,
A reflectivity estimator for estimating a reflectivity function of the object from the geometry of the object estimated by the geometry estimator and inputting a reflectivity function of the object to the geometry estimator a predetermined number of times,
A database unit for mapping the reflectivity function of the object estimated by the reflectivity estimating unit to the kind of the material constituting the object and storing the mapping function;
And a surface normal vector is estimated from a reflectivity value of the extracted arbitrary object to estimate a surface normal vector of the arbitrary object, Dimensional model estimating unit for estimating a three-
The geometric structure estimating unit re-estimates the geometry of the object based on the reflection function received from the reflectivity estimating unit,
Wherein the geometric estimator and the reflectivity estimator output the final geometry and final reflectivity of the object while exchanging the estimated results for the predetermined number of times. The method of claim 5,
The three-dimensional model estimating unit
And estimating a surface normal vector from the reflectivity of the arbitrary object using a Shape from Shading method, assuming a uniform albedo. A method of estimating an object's geometry and reflectivity by a system operating by at least one processor,
Estimating a surface normal vector map of the object by applying optical stereo to the near infrared rays images obtained by photographing the object by varying the direction and the direction of the light source,
Estimating an initial reflectivity function of the object from the surface normal vector map, and
Updating the geometry of the object from the initial reflectivity function using a Shape from Shading method and updating the reflectivity function of the object from the updated geometry is repeated a predetermined number of times, Obtaining a final geometry and a final reflectivity,
The step of estimating the initial reflectivity function
And estimating an initial reflectivity function of the object by fitting a reflectivity value of each pixel observed for each pixel according to an angle of each surface normal vector. delete 8. The method of claim 7,
The step of estimating the initial reflectivity function
And fitting the reflectance value to the pixel using an angle parameterized with an incident angle and a half vector of the reflection angle in half angle coordinates. A method of estimating an object's geometry and reflectivity by a system operating by at least one processor,
Estimating a surface normal vector map of the object by applying optical stereo to the near infrared rays images obtained by photographing the object by varying the direction and the direction of the light source,
Estimating an initial reflectivity function of the object from the surface normal vector map, and
Updating the geometry of the object from the initial reflectivity function using a Shape from Shading method and updating the reflectivity function of the object from the updated geometry is repeated a predetermined number of times, Obtaining a final geometry and a final reflectivity,
The step of obtaining the final geometry and final reflectivity
Calculating a reflectivity function of the object according to a direction of the light source, mapping the reflectivity function of the object to a type of a material constituting the object, and storing the result in a database. 11. The method of claim 10,
Receiving a target near infrared ray image,
Confirming a direction of a light source and a material constituting an arbitrary object included in the target near infrared ray image,
Extracting a reflectance value corresponding to a light source direction and a substance constituting the arbitrary object in the database; and
Estimating a surface normal vector from a reflectivity value of the arbitrary object and estimating a geometry of the arbitrary object
≪ / RTI > A geometry and reflectivity estimation system operating by at least one processor,
A database unit for storing a reflectivity function for each object estimated from near-infrared images of the plurality of sample objects, and
The method includes the steps of determining a type of a target object included in the target near infrared ray image when the target near infrared ray image is input, estimating a surface normal vector of the target object with reference to a reflectivity value corresponding to the target object, A three-dimensional model estimator for obtaining a geometrical structure of the target object from a surface normal vector,
Lt; / RTI >
The object reflectivity function is estimated from the initial geometry of each sample object,
Wherein the initial geometry of each sample object is estimated from near-infrared images of each sample object. The method of claim 12,
The reflectivity function
The system is obtained from near-infrared images obtained by photographing each sample object by varying a light source direction and a view angle. The method of claim 12,
The object-specific reflectivity function
Wherein a first procedure for estimating a geometry from an estimated current reflectivity function and a second procedure for estimating a reflectivity function from an estimated current geometry are obtained a predetermined number of times.
KR1020160167526A 2016-12-09 2016-12-09 Method and system for estimating surface geometry and reflectance KR101865886B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160167526A KR101865886B1 (en) 2016-12-09 2016-12-09 Method and system for estimating surface geometry and reflectance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160167526A KR101865886B1 (en) 2016-12-09 2016-12-09 Method and system for estimating surface geometry and reflectance

Publications (1)

Publication Number Publication Date
KR101865886B1 true KR101865886B1 (en) 2018-06-08

Family

ID=62599831

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160167526A KR101865886B1 (en) 2016-12-09 2016-12-09 Method and system for estimating surface geometry and reflectance

Country Status (1)

Country Link
KR (1) KR101865886B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020076026A1 (en) * 2018-10-08 2020-04-16 한국과학기술원 Method for acquiring three-dimensional object by using artificial lighting photograph and device thereof
KR20200040194A (en) * 2018-10-08 2020-04-17 한국과학기술원 Acquisition Method for 3D Objects Using Unstructured Flash Photography and Apparatus Therefor
CN117132634A (en) * 2023-10-26 2023-11-28 深圳市华汉伟业科技有限公司 Object morphology estimation method and computer readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030034274A (en) * 2001-09-14 2003-05-09 박보건 Generalized photometric stereo for the hybrid reflectance model using an image sequences
KR20130002760A (en) * 2011-06-29 2013-01-08 국립대학법인 울산과학기술대학교 산학협력단 Surface roughness measurement apparatus and method having intermediate view generator
JP2015232483A (en) * 2014-06-09 2015-12-24 株式会社キーエンス Image inspection device, image inspection method, image inspection program and computer readable recording medium, and apparatus having image inspection program recorded therein
JP2015232476A (en) * 2014-06-09 2015-12-24 株式会社キーエンス Inspection device, inspection method, and program
KR20160098814A (en) * 2015-02-11 2016-08-19 한국과학기술원 Device for obtaining 3d information and method for obtaining 3d information using the same

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030034274A (en) * 2001-09-14 2003-05-09 박보건 Generalized photometric stereo for the hybrid reflectance model using an image sequences
KR20130002760A (en) * 2011-06-29 2013-01-08 국립대학법인 울산과학기술대학교 산학협력단 Surface roughness measurement apparatus and method having intermediate view generator
JP2015232483A (en) * 2014-06-09 2015-12-24 株式会社キーエンス Image inspection device, image inspection method, image inspection program and computer readable recording medium, and apparatus having image inspection program recorded therein
JP2015232476A (en) * 2014-06-09 2015-12-24 株式会社キーエンス Inspection device, inspection method, and program
KR20160098814A (en) * 2015-02-11 2016-08-19 한국과학기술원 Device for obtaining 3d information and method for obtaining 3d information using the same

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020076026A1 (en) * 2018-10-08 2020-04-16 한국과학기술원 Method for acquiring three-dimensional object by using artificial lighting photograph and device thereof
KR20200040194A (en) * 2018-10-08 2020-04-17 한국과학기술원 Acquisition Method for 3D Objects Using Unstructured Flash Photography and Apparatus Therefor
KR102287472B1 (en) 2018-10-08 2021-08-09 한국과학기술원 Acquisition Method for 3D Objects Using Unstructured Flash Photography and Apparatus Therefor
CN117132634A (en) * 2023-10-26 2023-11-28 深圳市华汉伟业科技有限公司 Object morphology estimation method and computer readable storage medium
CN117132634B (en) * 2023-10-26 2024-01-23 深圳市华汉伟业科技有限公司 Object morphology estimation method and computer readable storage medium

Similar Documents

Publication Publication Date Title
US11869139B2 (en) System and method for three-dimensional scanning and for capturing a bidirectional reflectance distribution function
US11182961B2 (en) Method and system for representing a virtual object in a view of a real environment
Alldrin et al. Photometric stereo with non-parametric and spatially-varying reflectance
Oxholm et al. Multiview shape and reflectance from natural illumination
Herbort et al. An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods
EP3051793B1 (en) Imaging apparatus, systems and methods
Brostow et al. Video normals from colored lights
Choe et al. Exploiting shading cues in kinect ir images for geometry refinement
Choe et al. Refining geometry from depth sensors using IR shading images
JP2023511747A (en) Systems and methods for synthesizing data for training statistical models with different imaging modalities, including polarization imaging
Logothetis et al. A cnn based approach for the near-field photometric stereo problem
EP3629303A1 (en) Method and system for representing a virtual object in a view of a real environment
KR101865886B1 (en) Method and system for estimating surface geometry and reflectance
US11275942B2 (en) Method and system for generating training data
Birkbeck et al. Variational shape and reflectance estimation under changing light and viewpoints
Logothetis et al. Near-field photometric stereo in ambient light
Angelopoulou et al. Evaluating the effect of diffuse light on photometric stereo reconstruction
JP5441752B2 (en) Method and apparatus for estimating a 3D pose of a 3D object in an environment
Argyriou et al. Optimal illumination directions for faces and rough surfaces for single and multiple light imaging using class-specific prior knowledge
Quéau et al. A full photometric and geometric model for attached webcam/matte screen devices
JP7463697B2 (en) Gloss acquisition state calculation device, gloss acquisition state calculation method, gloss acquisition state calculation program, terminal, and gloss acquisition state display program
Worthington Reillumination-driven shape from shading
Richter et al. A discriminative approach to perspective shape from shading in uncalibrated illumination
Lenoch Image-based 3D reconstruction of surfaces with highly complex reflectance properties
Khanian et al. Real-Time 3D Shape of Micro-Details

Legal Events

Date Code Title Description
GRNT Written decision to grant