JP2011237369A - Surface condition classification device, surface condition classification method and surface condition classification program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To classify surface condition of an observation object from a camera image.SOLUTION: A surface condition classification device inputs an observed image photographing a surface of an observation object, applies a constant optical model to a luminance value of each pixel of the observed image, estimates a surface condition of the observation object by solving the constant optical model which is applied, and classifies the surface condition of the observation object on the basis of the estimated result.

Description

  The present invention relates to a technique for classifying a surface state of an observation target.

  It is known that the surface of an object observed by a camera image changes in appearance due to various external factors. In particular, due to the influence of humidity, the surface to be observed becomes a wet surface and also a dry surface.

  Currently, it is required to estimate / analyze the characteristics of the surface to be observed by image processing, computer vision, or the like (Non-Patent Document 1). This is important in traffic monitoring because the wet surface on the road is related to tire slip, and in the case of buildings, the difference in appearance between the wet and dry surfaces is corrected for diagnostic imaging. Because it is necessary.

Digital Image Processing Editorial Committee, “Digital Image Processing”, 2nd Edition, Japan, Image Information Education Promotion Committee (CG-ARTS Association), March 1, 2006, p.49-57, 64-68 279-282 "Conjugate gradient method", [online], free encyclopedia "Wikipedia", [searched on May 10, 2010], Internet <URL: http://en.wikipedia.org/wiki/%E5% 85% B1% E5% BD% B9% E5% 8B% BE% E9% 85% 8D% E6% B3% 95> “Steepest descent method”, [online], free encyclopedia “Wikipedia”, [May 10, 2010 search], Internet <URL: http://en.wikipedia.org/wiki/%E6% 9C% 80% E6% 80% A5% E9% 99% 8D% E4% B8% 8B% E6% B3% 95>

  However, conventionally, since only the brightness level of the camera image is used when classifying the surface state of the observation target, there has been a problem that the surface state is often classified incorrectly. Although the surface state can be detected by using a special and expensive device such as a millimeter wave radar, it is currently required to use a general camera image.

  This invention is made | formed in view of the above, and makes it a subject to classify | categorize the surface state of observation object from a camera image.

  The surface state classification apparatus according to claim 1, an input unit that inputs an observation image in which a surface of an observation target is copied, a storage unit that stores the input observation image, and reads the observation image from the storage unit The fitting estimation means for estimating the surface state of the observation object by fitting the luminance value of each pixel of the observation image to a certain optical model and solving the fitted optical model, Classification means for classifying the surface state of the observation target based on the estimation result.

  According to the present invention, an observation image that captures the surface of the observation target is input, and the luminance value of each pixel of the observation image is fitted to a certain optical model, and by solving the fitted constant optical model, Since the surface state of the observation target is estimated and the surface state of the observation target is classified based on the estimation result, the surface state of the observation target can be classified from the camera image.

The surface state classification device according to claim 2 is the surface state classification device according to claim 1, wherein the fitting estimation means calculates the luminance value I (x, y) of each pixel of the observation image. The intensity (K _d cos θ + K _s (cos θ) ⁿ ) of reflected light diffusely reflected and specularly reflected at an arbitrary position on the surface of the object (where K _d , K _s , θ, n are respectively at the arbitrary positions) It is defined as a diffuse reflection coefficient, a specular reflection coefficient, an angle formed by a normal vector of the surface to be observed and the reflected light, and n is a natural number of 1 or more, and from the definition, Σ ^F _{f = 1} Σ _{(x, y ) ∈ Ω} [I (x, y) −K _d cos θ−K _s (cos θ) ⁿ ] The objective function of _f (where F and Ω are the number of observation images and the constant image area of the observation images, respectively) And the luminance value I (x, y) of each pixel of the observed image is The diffuse reflection coefficient value and the specular reflection coefficient value are estimated by substituting into a functional function and solved as a minimization problem, and the classification means estimates the diffuse reflection coefficient estimate value and the specular reflection coefficient estimate. The surface state of the observation target is classified based on the value.

According to the present invention, the luminance value I (x, y) of each pixel of the observation image is converted into the intensity (K _d cos θ + K _s (cos θ) ⁿ ) of the reflected light that is diffusely reflected and specularly reflected at an arbitrary position on the surface of the observation target. (However, K _d , K _s , θ, n are the diffuse reflection coefficient, specular reflection coefficient, angle between the normal vector of the surface to be observed and the reflected light, and n is 1, respectively. From the definition, Σ ^F _{f = 1} Σ _{(x, y) εΩ} [I (x, y) −K _d cos θ−K _s (cos θ) ⁿ ] _f objective function (where, F and Ω are derived as the minimization problem by deriving the number of observation images and the constant image area of the observation image, respectively, and substituting the luminance value I (x, y) of each pixel of the observation image into the objective function. Thus, the diffuse reflection coefficient value and the specular reflection coefficient value are estimated. Since the surface state of the observation target is classified, the surface state of the observation target can be classified from the camera image. In particular, since the objective function is defined as Σ ^F _{f = 1} , the observation state can be classified even from one observation image.

  The surface state classification device according to claim 3 is the surface state classification device according to claim 2, wherein the fitting estimation means applies the constraint condition related to the diffuse reflection coefficient and the specular reflection coefficient to the objective function. It is characterized by doing.

  According to the present invention, since the constraint conditions related to the diffuse reflection coefficient and the specular reflection coefficient are applied to the objective function, it is possible to improve the convergence when solving the objective function as a minimization problem.

  The surface state classification device according to claim 4 is the surface state classification device according to claim 2 or 3, wherein the classification means is configured such that the estimated value of the diffuse reflection coefficient is larger than the estimated value of the specular reflection coefficient. Is characterized in that the surface to be observed is classified as a dry surface, and if it is small, it is classified as a wet surface.

  According to the present invention, when the estimated value of the diffuse reflection coefficient is larger than the estimated value of the specular reflection coefficient, the surface to be observed is classified as a dry surface, and when it is small, the surface is classified as a wet surface. The surface condition of the observation target can be classified as a dry surface or a wet surface.

  The surface state classification method according to claim 5 includes a first step of inputting an observation image obtained by copying a surface to be observed by a computer, and a second step of storing the input observation image in a storage means. The observation image is read from the storage means, the luminance value of each pixel of the observation image is applied to a fixed optical model, and the fixed optical model is solved to obtain the surface of the observation target It has the 3rd step which estimates a state, and the 4th step which classifies the surface state of the observation object based on an estimation result, It is characterized by the above-mentioned.

The surface state classification method according to claim 6 is the surface state classification method according to claim 5, wherein the third step calculates the luminance value I (x, y) of each pixel of the observation image as the observation target. The intensity (K _d cos θ + K _s (cos θ) ⁿ ) of reflected light diffusely reflected and specularly reflected at an arbitrary position on the surface of the surface (where K _d , K _s , θ, n are diffused at the arbitrary positions, respectively) A reflection coefficient, a specular reflection coefficient, an angle formed by a normal vector of the surface to be observed and the reflected light, and n is a natural number of 1 or more, and from the above definition, Σ ^F _{f = 1} Σ _{(x, y) ∈Ω} [I (x, y) −K _d cos θ−K _s (cos θ) ⁿ ] An objective function of _f (where F and Ω are the number of observation images and a constant image area of the observation images, respectively). A luminance value I (x, y) of each pixel of the observation image The diffuse reflection coefficient value and the specular reflection coefficient value are estimated by substituting into a functional function and solving as a minimization problem, and the fourth step includes estimating the diffuse reflection coefficient and the specular reflection coefficient. And classifying the surface state of the observation target based on the estimated value of.

  The surface state classification method according to claim 7 is the surface state classification method according to claim 6, wherein the third step applies a constraint condition related to the diffuse reflection coefficient and the specular reflection coefficient to the objective function. It is characterized by that.

  The surface state classification method according to claim 8 is the surface state classification method according to claim 6 or 7, wherein in the fourth step, the estimated value of the diffuse reflection coefficient is larger than the estimated value of the specular reflection coefficient. When it is large, the surface to be observed is classified as a dry surface, and when it is small, it is classified as a wet surface.

  A surface state classification program according to a ninth aspect causes a computer to execute each step in the surface state classification method according to any one of the fifth to eighth aspects.

  According to the present invention, the surface state of the observation target can be classified from the camera image.

It is a figure which shows the functional block structure of a surface state classification | category apparatus. It is a figure which shows the basic optical model regarding diffuse reflection and specular reflection. It is a figure which shows the classification result of the wet surface and the dry surface. It is a figure which shows the estimation evaluation result of the parameter estimated using two types of artificially produced | generated reflection images.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. However, the present invention can be implemented in many different modes and should not be construed as being limited to the description of the present embodiment.

  FIG. 1 is a diagram showing a functional block configuration of the surface state classification apparatus according to the present embodiment. The surface state classification device 1 includes an image input unit 11, a data storage unit 12, an optical model fitting estimation unit 13, a state classification unit 14, and a display unit 15. First, the functions of these functional units will be described in detail.

  The image input unit 11 has a function of accepting input of one or more observation images obtained by copying the surface of the observation target photographed by a camera (not shown) and inputting the input into the surface state classification device 1. .

  The data storage unit 12 has a function of storing the observation image received / input by the image input unit 11 in a readable manner.

  The optical model fitting estimation unit 13 acquires an observation image from the data storage unit 12 and analyzes the luminance value of each pixel of the observation image in order to analyze the surface state of the observation target (dry state or wet state). It has a function of estimating the surface state of the observation target by solving the fixed optical model.

  The state classification unit 14 has a function of classifying the surface state of the observation target into a dry state or a wet state based on the estimation result by the optical model fitting estimation unit 13.

  The display unit 15 has a function of displaying the classification result of the surface state of the observation target by the state classification unit 14 on the screen.

Next, an optical model used in the optical model fitting estimation unit 13 will be described. In the present embodiment, as shown in FIG. 2, an optical model relating to diffuse reflection D and specular reflection S is used. The angle between the normal vector N at the position P of the incident light L _i and the surface of the observation target 32 from the light source 31 such as sunlight (incident angle) and theta, diffuse reflection coefficient of diffuse reflection D at the position P Assuming that (diffusion reflectance) is K _d and the specular reflection coefficient (specular reflectance) of the specular reflection S is K _s , the luminance value I of the pixel of the observed image captured by the camera 33 connected to the surface state classification device 1 (X, y) can be defined by the expression (1) at the position P. However, n is a natural number of 1 or more, and corresponds to the specular reflection intensity as will be described later.

The observation object 32 photographed by the camera 33 is a set of positions P, and the diffuse reflection coefficient K _d , the specular reflection coefficient K _s , and the incident angle θ are different at each position P. Therefore, in terms of the relationship with the pixel position (x, y) of the observed image, the variables on the right side of Equation (1) are K _d = K _d (x, y), K _s = K _s (x, y), θ = θ (x, y), and has different values depending on the position. Further, according to the law of reflection, an angle (reflection angle) between the normal vector N and the reflected light Lr having the same angle as the incident angle θ may be used instead of the incident angle θ.

In the equation (1), when the diffuse reflection coefficient _Kd is large, the surface of the observation target 32 is in a state of spreading uniformly and the light intensity is smaller than that of the light source 31. On the other hand, when the specular reflection coefficient K _s is large, the surface of the observation target 32 is in a state close to a point such as a spotlight, closer to the light intensity of the light source 31. In particular, the larger the value of n, the stronger the specular reflection.

Next, by solving the equation (1) in which the luminance value of each pixel of the observation image is applied to the optical model related to the diffuse reflection D and the specular reflection S, the diffuse reflection coefficient K _d for the dryness and wetness of the surface to be observed is solved. and described modeling process based on the size of the specular reflection coefficient K _s. Specifically, the four parameters of the diffuse reflection coefficient K _d , the specular reflection coefficient K _s , the incident angle θ (or the reflection angle θ), and the specular reflection intensity n in the expression (1) are input to each pixel of the input observation image. The surface state is modeled by estimating from the luminance value I.

First, an objective function (the following equation (2)) for estimating these four parameters is derived from the equation (1). However, (i, j) is the position of an arbitrary pixel on the image, F is the number of input observation images, and Ω is a constant image area of the observation image. (X, y) indicates the position of the pixel belonging to the certain image region Ω.

Then, the diffuse reflection coefficient K _d , specular reflection coefficient K _s , and incident angle are calculated by substituting the luminance value I (x, y) of each pixel of the observed image into the objective function of Expression (2) and solving as a minimization problem. Each value of θ (or reflection angle θ) and specular reflection intensity n is estimated.

It should be noted that by providing constraint conditions for some of the four parameters, the convergence of the calculation of equation (2) can be improved. Specifically, since the reflection characteristic is based on the balance of two coefficients of the diffuse reflection coefficient K _d and the specular reflection coefficient K _s , the equation (2) is taken into account that K _d + K _s ≈1. ) Is expanded to equation (3).

Moreover, about the value of the specular reflection intensity n, since a sufficient approximate value can be obtained experimentally with about 3 to 5, an upper limit value of the specular reflection intensity n is set. Then, the minimization calculation of Expression (3) is performed. As a necessary condition for the minimization calculation, Equation (4) is given.

Here, such a minimization calculation can be solved using a known calculation method. For example, a least square method can be used. Various numerical methods are known for the least square method, but a conjugate gradient method, a steepest descent method, or the like can be applied (see Non-Patent Documents 2 and 3). In a typical steepest descent method, it is possible to perform the minimization calculation using the following formulas (5) to (8). Here, p is the number of iterations, and h is a convergence adjustment coefficient.

The number of iterations p is set to 20, the convergence adjustment factor h is set to 0.01, the initial values of the four parameters are set to 0 (zero), and each calculation formula is iteratively calculated, whereby the diffuse reflection coefficient K _d , mirror surface The reflection coefficient K _s , the incident angle θ (or the reflection angle θ), and the specular reflection intensity n are calculated.

  Note that since an observation image captured by a camera in an actual environment includes environmental disturbances and noise, it is possible to use a nonlinear least square method based on robust statistics, which is effective.

  Next, the processing flow of the surface state classification apparatus 1 will be described. First, the image input unit 11 inputs one or more observation images obtained by copying the surface of the observation target photographed by the camera (S1).

  Next, the observation image input in S1 is stored in a readable manner by the data storage unit 12 (S2).

Next, the optical model fitting estimation unit 13 defines the luminance value I (x, y) of each pixel of the observation image as the intensity of the reflected light that is diffusely reflected and specularly reflected at an arbitrary position on the surface of the observation target. The objective function of Expression (3) is derived from the definition, and the luminance value I (x, y) of each pixel of the observed image is substituted into the objective function and solved as a minimization problem, whereby four parameters (K _d , K _s , θ, n) are estimated (S3).

Then, the state classification unit 14, among the estimated four parameters were based on the estimated value of the estimated value and the specular reflection coefficient K _s of the diffuse reflection coefficient K _d, the surface state of the observation target is classified (S4) . Specifically, as shown in FIG. 3, if the estimated value of the diffuse reflection coefficient K _d is greater than the estimated value of the specular reflection coefficient K _s (if the K _d> K _s), diffuse reflection is dominant Therefore, the surface to be observed is classified as a dry surface. On the contrary, when the estimated value of the diffuse reflection coefficient K _d is smaller than the estimated value of the specular reflection coefficient K _s (when K _d <K _s ), the specular reflection is dominant, and the surface to be observed is Classified as wet surface.

  Finally, the display unit 15 displays the classification result of S4 on the screen (S5).

  Next, the estimation evaluation results of the parameters estimated by Expression (2) without constraint conditions and Expression (3) with constraint conditions will be described. 4A shows an evaluation result using a diffuse reflection image, and FIG. 4B shows an evaluation result using a specular reflection image.

  One diffuse (F = 1) diffuse reflection image shown on the left in FIG. 4A is artificially generated, and the brightness value of the generated image is substituted into Equation (2) and Equation (3), respectively. Is calculated using formula (3) with constraints rather than using formula (2) without constraints for any parameter, the estimation error is smaller. became. Similarly, in the case of a specular reflection image, the estimation error is smaller when the calculation is performed using Expression (3) having a constraint condition.

  In addition, when a plurality of reflection images are generated (F = 5 or 10) and the same calculation is performed, the estimation error can be further reduced as compared with the case of F = 1.

  Finally, the optical model related to the diffuse reflection D and the specular reflection S is an example of an optical model used by the optical model fitting estimation unit 13, and other optical models may be used as long as the surface state of the observation target can be estimated. Is possible.

  Further, the surface state classification apparatus 1 described in the present embodiment is applied to industrial fields related to multimedia field, coding field, communication field, non-destructive inspection field, monitoring in real environment, image sensing, video production and the like. Is possible.

  Moreover, the surface state classification apparatus 1 demonstrated in this Embodiment is comprised with a computer. That is, the data storage unit 12 is realized by a storage unit such as a memory. The image input unit 11, the optical model fitting estimation unit 13, the state classification unit 14, and the display unit 15 are realized by a calculation unit such as a CPU and executed by a program.

  In addition, the surface state classification device 1 described in the present embodiment is recorded as a program in a readable manner on a recording medium such as an optical storage device or a magnetic storage device, and this recording medium is incorporated into a computer or recorded on a recording medium. Can be downloaded to a computer via an arbitrary communication line, or installed from a recording medium, and the computer is operated by the program, thereby causing each processing operation described above to function as the surface state classification device 1. Of course you can.

  According to the present embodiment, an observation image representing the surface of the observation target is input, and the luminance value of each pixel of the observation image is applied to a fixed optical model, and the fixed optical model is solved. Thus, the surface state of the observation target is estimated and the surface state of the observation target is classified based on the estimation result, so that the surface state of the observation target can be classified from the camera image.

According to the present embodiment, the luminance value I (x, y) of each pixel of the observation image is converted into the intensity (K _d cos θ + K _s (cos θ) of diffuse reflection and specular reflection at an arbitrary position on the surface of the observation target. ^N ) (where K _d , K _s , θ, and n are the diffuse reflection coefficient, specular reflection coefficient, angle between the normal vector of the surface to be observed and the reflected light, n Is a natural number greater than or equal to 1), and Σ ^F _{f = 1} Σ _{(x, y) εΩ} [I (x, y) −K _d cos θ−K _s (cos θ) ⁿ ] _f However, F and Ω are respectively derived from the number of observation images and a constant image region of the observation image), and the luminance value I (x, y) of each pixel of the observation image is substituted into the objective function to minimize the problem. As the diffuse reflection coefficient value and specular reflection coefficient value. Since the surface state of the observation target is classified based on the above, the surface state of the observation target can be classified from the camera image. In particular, since the objective function is defined as Σ ^F _{f = 1} , the observation state can be classified even from one observation image.

  According to the present embodiment, since the constraint conditions related to the diffuse reflection coefficient and the specular reflection coefficient are applied to the objective function, it is possible to improve the convergence when solving the objective function as a minimization problem.

  According to the present embodiment, when the estimated value of the diffuse reflection coefficient is larger than the estimated value of the specular reflection coefficient, the surface to be observed is classified as a dry surface, and when it is small, the surface is classified as a wet surface. Therefore, it is possible to classify the surface state of the observation target according to the dry surface or the wet surface.

DESCRIPTION OF SYMBOLS 1 ... Surface state classification | category apparatus 11 ... Image input part (input means)
12: Data storage unit (storage means)
13: Optical model fit estimation unit (fit fit estimation means)
14 ... State classification part (classification means)
DESCRIPTION OF SYMBOLS 15 ... Display part 31 ... Light source 32 ... Observation object 33 ... Camera S1-S5 ... Step

Claims (9)

An input means for inputting an observation image showing the surface of the observation target;
Storage means for storing the input observation image;
The observation image is read from the storage means, and the luminance value of each pixel of the observation image is applied to a fixed optical model, and the fixed optical model is solved to solve the surface state of the observation target. A fitting estimation means for estimating
Classification means for classifying the surface state of the observation target based on the estimation result;
A surface state classification device comprising: The fitting estimation means includes
The luminance value I (x, y) of each pixel of the observation image is expressed as the intensity (K _d cosθ + K _s (cosθ) ⁿ ) of the reflected light that is diffusely reflected and specularly reflected at an arbitrary position on the surface of the observation target (where, K _d , K _s , θ, and n are the diffuse reflection coefficient, the specular reflection coefficient, the angle between the normal vector of the surface to be observed and the reflected light, and n is 1 or more, respectively, at the arbitrary position. Natural number), and an objective function of Σ ^F _{f = 1} Σ _{(x, y) ∈ Ω} [I (x, y) −K _d cos θ−K _s (cos θ) ⁿ ] _f (where F, Ω is derived as the minimization problem by deriving the number of the observed images and the constant image area of the observed image, respectively, and substituting the luminance value I (x, y) of each pixel of the observed image into the objective function. Solving and estimating the diffuse reflection coefficient value and the specular reflection coefficient value And
The classification means includes
The surface state classification device according to claim 1, wherein the surface state of the observation target is classified based on the estimated value of the diffuse reflection coefficient and the estimated value of the specular reflection coefficient. The fitting estimation means includes
The surface state classification apparatus according to claim 2, wherein a constraint condition related to the diffuse reflection coefficient and the specular reflection coefficient is applied to the objective function. The classification means includes
When the estimated value of the diffuse reflection coefficient is larger than the estimated value of the specular reflection coefficient, the surface to be observed is classified as a dry surface, and when it is smaller, the surface is classified as a wet surface. The surface state classification apparatus according to claim 2 or 3. By computer
A first step of inputting an observation image representing the surface of the observation target;
A second step of storing the input observation image in a storage means;
The observation image is read from the storage means, and the luminance value of each pixel of the observation image is applied to a fixed optical model, and the fixed optical model is solved to solve the surface state of the observation target. A third step of estimating
A fourth step of classifying the surface state of the observation target based on the estimation result;
A surface state classification method characterized by comprising: The third step includes
The luminance value I (x, y) of each pixel of the observation image is expressed as the intensity (K _d cosθ + K _s (cosθ) ⁿ ) of the reflected light that is diffusely reflected and specularly reflected at an arbitrary position on the surface of the observation target (where, K _d , K _s , θ, and n are the diffuse reflection coefficient, the specular reflection coefficient, the angle between the normal vector of the surface to be observed and the reflected light, and n is 1 or more, respectively, at the arbitrary position. Natural number), and an objective function of Σ ^F _{f = 1} Σ _{(x, y) ∈ Ω} [I (x, y) −K _d cos θ−K _s (cos θ) ⁿ ] _f (where F, Ω is derived as the minimization problem by deriving the number of the observed images and the constant image area of the observed image, respectively, and substituting the luminance value I (x, y) of each pixel of the observed image into the objective function. Solving and estimating the diffuse reflection coefficient value and the specular reflection coefficient value And
The fourth step includes
The surface state classification method according to claim 5, wherein the surface state of the observation target is classified based on the estimated value of the diffuse reflection coefficient and the estimated value of the specular reflection coefficient. The third step includes
The surface state classification method according to claim 6, wherein a constraint condition related to the diffuse reflection coefficient and the specular reflection coefficient is applied to the objective function. The fourth step includes
When the estimated value of the diffuse reflection coefficient is larger than the estimated value of the specular reflection coefficient, the surface to be observed is classified as a dry surface, and when it is smaller, the surface is classified as a wet surface. The surface state classification method according to claim 6 or 7.   A surface state classification program that causes a computer to execute each step in the surface state classification method according to any one of claims 5 to 8.