JP4349568B2 - Object recognition apparatus, object recognition method, program, and recording medium - Google Patents

Description

  The present invention eliminates the specular reflection component in a still image or continuously input moving image in a situation where specular reflection or gloss is observed on the object surface, and recognizes the object using the result. The present invention relates to an apparatus and method, a program thereof, and a recording medium.

  In the object recognition technique using an image, when specular reflection or gloss is observed on the surface of a subject, the recognition accuracy is lowered due to the influence. In order to solve this problem, in general, processing using only an area that does not include a specular reflection component or processing using an image of a diffuse reflection component obtained by separating the specular reflection component from an input image is performed. .

  However, in the process using only the region not including the specular reflection component, it is necessary to determine whether or not the specular reflection is included, and a complicated process is required. In addition, when the texture characterizing the subject is concealed by specular reflection, the recognition accuracy is significantly reduced.

  On the other hand, the specular reflection component and the diffuse reflection component are separated, and in the processing using the diffuse reflection component image, the specular reflection component is removed by controlling the polarization of light in the illumination light source and the input unit of the image capturing apparatus. Thus, an image having only a diffuse reflection component can be obtained (see, for example, Non-Patent Document 1). In addition, the specular reflection component and the diffuse reflection component of the subject can be separated from an image taken by illuminating the subject from different directions (see, for example, Non-Patent Document 2). However, in the techniques described in Non-Patent Document 1 and Non-Patent Document 2, in order to obtain a single diffuse reflection component image, a plurality of input images taken under different shooting conditions are required. Is not suitable for a case where a subject is a subject or a moving image is an input image. Furthermore, since a special photographing apparatus is required, the situation in which this technology can be used is also limited.

  Further, from a plurality of images obtained by photographing a subject from multiple directions, corresponding points can be tracked in the image, and the specular reflection component and the diffuse reflection component can be separated using the result (see, for example, Patent Document 1). . However, in the technique described in Patent Document 1, when there are a plurality of illumination light sources, the accuracy of tracking corresponding points is lowered due to the influence of the specular reflection component and gloss, and as a result, the specular reflection component and the diffuse reflection component are separated. Accuracy is reduced.

  Further, by estimating the spectral reflectance of the subject, it is possible to reproduce the color observed under the same illumination even in an illumination environment different from that at the time of photographing (when the illumination light spectrum is different). Currently, development of a high-color re-image system in which this technology is applied to the medical field, an electronic museum (digital archive), and electronic commerce (online shopping, etc.) is in progress. However, image recognition, search, and search are not performed in which the primary purpose is mainly visual identification, and the images stored in the database or the like and the input images are compared with each other. The reason for this is that when specular reflection is observed (including an image), color reproduction is difficult due to the effect, and the texture of the subject surface to be originally observed is hidden by specular reflection. Can be mentioned.

Nobutoshi Kojima, 2 others, “Estimation of Makeup Skin Texture (II) (Quantification of skin irregularities information)”, Journal of the Japan Photography Society, 56 (4), 1993, pp. 264-269 YoichiSato and two others, “Temporal-color space analysis of reflection”, Journal of Optical Society of America, A / vol. 11, November, 1994, pp. 2990-3002. JP 2003-85531 A

  The present invention has been made to solve the above-described problems of the prior art. An illumination light component is used to generate an image (specular reflection removal image) that does not include a specular reflection component, and the obtained specular reflection removal image is obtained. It is an object of the present invention to provide an object recognition apparatus and method, a program thereof, and a recording medium that can stably perform object recognition processing with respect to changes in illumination conditions (the number, type, and arrangement of light sources).

  Further, the present invention provides an object recognition apparatus and method, a program thereof, and an object recognition device capable of collation by simple comparison for each pixel value even when the type of illumination (spectral spectrum) at the time of shooting is different between the input image and the collation image An object is to provide a recording medium.

The present invention includes means for acquiring an illumination light component of an illumination light source used for photographing an object , an input image that is an image of the object photographed by the photographing apparatus, and a spectral radiance of reflected light from the surface of the object. Means for capturing, at least means for accumulating an accumulated image that is an image of an object that has been captured in advance, and an illumination light component at the time of capturing the accumulated image, and a mirror surface from the input image using the acquired illumination light component Means for generating a specular reflection removal image that does not include a reflection component.

Further, the present invention further comprises means for collating the stored images stored in the storage means and the specular reflection removed image. In addition, the image processing apparatus includes means for displaying an input image and a specular reflection removal image of the input image, or an input image, a specular reflection removal image and an accumulation image of the input image, and a specular reflection removal image of the accumulation image.

The specular reflection removal image is generated by dividing the spectral radiance of the input image by the illumination light component to obtain the total reflectance image (calculating the apparent reflectance at each pixel on the object surface using the illumination light), and this total reflection a signal value of each pixel rate image, vector U (λ) = [1,1, ..., 1] (λ is wavelength) performed by projecting the hyperplane to the normal line.

  Note that the input and output characteristics of the device are measured in advance with respect to the illumination light component measurement device and the imaging device used for image capture.

  First, acquisition of illumination light components such as illumination light sources will be described. In general, diffused light from a white plate is used as an illumination light component, and a spectral luminance meter, a color luminance meter, or a digital camera capable of taking an image of three or more bands is used for the measurement. Generally, the three bands refer to the three primary colors “red, green, and blue”. Here, the illumination light component is a spectral value of illumination light, a light primary color, or a signal value of an image of a white plate photographed by a photographing apparatus. When a digital camera is used, the intensity of input light and the output signal value are not generally proportional. Therefore, the characteristics of the digital camera are measured in advance, and the signal value is normalized by brightness with reference to the result. At the time of measurement, only one type of illumination light source is turned on, and the other illuminations are turned off. This is performed for all illumination light sources to obtain a plurality of illumination light components. When the light sources used can be limited to a finite type, such as in an office environment, all of them may be assumed as illumination light sources and data measured in advance may be used. Further, natural light such as sunlight can be used as illumination light. However, when natural light and a plurality of artificial illumination lights are used in combination as an illumination light group, a configuration is required that enables only artificial illumination light to illuminate the subject (object). . As the configuration, for example, a cover that blocks the combination of the subject and artificial illumination light from natural light is used.

  Next, image capturing of a subject will be described. As the photographing device, for example, a digital camera capable of photographing images of three bands or more is used. The output signal value from the imaging device is normalized by brightness based on the device characteristics measured in advance, as in the case of obtaining illumination information. If the image information is lost because the glossiness or specular reflection in the image is too bright and exceeds the dynamic range of the imaging device, the exposure time and aperture value are adjusted to appropriate values or acquired by changing the exposure time. A single image with a wide dynamic range is synthesized. Further, the spectral spectrum or chromaticity value of the light input to the photographing apparatus is estimated from the obtained signal value according to the application.

  Next, the principle of obtaining a specular reflection-removed image that does not include a specular reflection component from the input image using the illumination light component will be described. Here, a case where a spectral spectrum is input will be described.

λは波長、ｋは鏡面反射成分の強度を表わす係数、Ｅ（λ）は照明光の分光分布である。 First, the principle of obtaining a specular reflection-removed image by projecting the signal value of each pixel of the input image onto a space orthogonal to the illumination light component will be described with reference to FIG.
The target object is a non-metallic chromatic diffuse object that can assume two-color regular reflection. For simplicity, consider the case where a subject is illuminated with a single point light source. The spectral radiance (reflected light spectrum) I (λ) of reflected light from one point on the object surface observed when a subject is observed in a certain direction is (1) when the standard dichroic reflection model can be assumed. ) As a linear sum of a diffuse reflection component (first term) and a specular reflection component (second term).

λ is a wavelength, k is a coefficient representing the intensity of a specular reflection component, and E (λ) is a spectral distribution of illumination light.

と表わすことができる。αと（１）式を（２）式に代入すると、

となり、Ｐ（λ）は、拡散反射成分Ｉ_ｄ（λ）をＥ（λ）に直交する空間へ投影したものであることが分かる（図１３）。したがって、Ｐ（λ）は鏡面反射成分の影響を含まないため、この成分に注目することで鏡面反射成分の変化に頑健な画像認識が可能となる。 The specular reflection component has the same spectral distribution as the illumination light. When a component obtained by projecting I (λ) onto a space orthogonal to E (λ) is P (λ),

Can be expressed as Substituting α and equation (1) into equation (2),

Thus, it can be seen that P (λ) is a projection of the diffuse reflection component I _d (λ) onto a space orthogonal to E (λ) (FIG. 13). Therefore, since P (λ) does not include the influence of the specular reflection component, by paying attention to this component, it is possible to perform image recognition that is robust against changes in the specular reflection component.

Ｄ(λ)は各光源単独で被写体を照明した際に観察される拡散反射成分の和、ｋ_ｉは各光源に対応する鏡面反射成分の強度を表わす係数である。 Next, consider a case where a subject is illuminated with L illumination light sources having different spectral distributions. Assuming that the spectral spectrum of the i-th illumination light source is E _i (λ), the spectral radiance (reflected light spectrum) I (λ) of the reflected light from a certain point on the observed object surface is expressed by the following equation. It is.

D (λ) is a sum of diffuse reflection components observed when the subject is illuminated by each light source alone, and k _i is a coefficient representing the intensity of the specular reflection component corresponding to each light source.

従って、全ての照明光源の分光スペクトルに直交する空間Ｇ（λ）を求め、Ｉ（λ）をＧ（λ）へ投影することにより、先の照明光源が１種類の時と同様の処理が可能である。 Here, a space orthogonal to all the spectral spectra E _i (λ) (i = 1, 2,..., N) of each light source is represented by G (λ
), The component P (λ) obtained by projecting the reflected light spectrum I (λ) onto the space G (λ) is expressed by the following equation.

Therefore, by calculating the space G (λ) orthogonal to the spectral spectrum of all illumination light sources and projecting I (λ) onto G (λ), the same processing as when one type of illumination light source is used is possible. It is.

The component P (λ) orthogonal to the illumination light spectrum can also be obtained as follows. A component P _i (λ) orthogonal to the spectral spectrum of the i-th illumination light source of the reflected light spectrum I (λ) is obtained, and P _i (λ) is further converted to a space orthogonal to the spectral spectrum of the (i + 1) -th illumination light source. Project. By repeating this process for i = 0,..., L, a component P (λ) orthogonal to all the illumination light spectra used is obtained as follows.

  Here, the difference in the processing of the above formula (5) or formulas (6) and (7) depends on the usage of the computer resources and the program.

  When the light sources used can be limited to a finite type, such as in an office environment, all of them may be assumed as illumination light sources, and a space orthogonal to all of the assumed spectrum of the illumination light sources may be G (λ).

The above principle holds true for an output signal from a photographing apparatus having linear input / output characteristics. First, a multiband image of a standard white plate (with a number of bands: N ≧ 3, corresponding to an image using a plurality of color filters with strong wavelength selectivity) is taken with each light source lit alone, and i W _i = [W _i (1),..., W _i (N)] obtained by normalizing the signal value of the image under the second light source is used instead of the spectral spectrum E _i (λ) of the illumination spectrum. . Next, a multiband image of the subject is taken. The same photographing device as that used for photographing a standard white plate is used. Consider a case where an object is illuminated with one illumination light source having different spectral spectra. Assuming that one pixel on the image records reflected light from one point on the surface of the subject, the pixel value c (n) of one pixel on the n-th band image is expressed as follows. .

W _i (n) is a normalized pixel value of the white plate under the i-th illumination light source, and D ′ _i (n) and k ′ _i (n) W _i (n) are only the i-th illumination light source. Diffuse reflection component and specular component when the subject is illuminated,
k ′ _i (n) is a coefficient representing the intensity of the specular reflection component. All _{W i} vectors orthogonal to G '= [G' (1 ), ···, G '(N)] sought, c = [c (1) , ···, c (N)] The G By projecting onto ', a component P ′ = [P ′ (1),..., P ′ (N)] that does not include a specular reflection component is obtained.

  By performing the above processing on all the pixels in the image, a specular reflection-removed image not including the specular reflection component is obtained. The obtained image is different from the image of the diffusion component. The pixel value depends on G ′ and becomes a negative value.

  By performing recognition using the component P (λ) or P ′ that is orthogonal to the illumination light spectrum obtained as described above, object recognition that is not affected by specular reflection components and is robust against illumination changes Is possible.

  Furthermore, each pixel value of the obtained component P (λ) or P ′ is normalized by its power (for example, P (λ) / | P (λ) |), and specular reflection independent of illumination conditions An image from which components are removed (illumination invariant image) may be generated, and recognition processing may be performed using this image.

Next, the spectral radiance of the input image is divided by the illumination light component to generate an overall reflectance image (the apparent reflectance at each pixel on the object surface is calculated using the illumination light spectrum), and the overall reflectance image is vectorized The principle of obtaining a specular reflection-removed image by projecting onto a hyperplane having the normal line U (λ) = [1, 1,..., 1] will be described. Here, the case where a spectrum is input is described.

ｘは物体表面の位置座標（ｘ＝［ｘ，ｙ，ｚ］^Ｔ）、λは波長、Ｉ_ｄ（ｘ，ｙ）は拡散反射成分、Ｉ_ｓ（ｘ，λ）は鏡面反射成分、Ｅ（λ）は照明光の分光スペクトル、Ｄ（ｘ）は拡散反射成分の明るさ、ｆ_ｄ（ｘ，λ）は拡散反射率、Ｓ（ｘ）は鏡面反射の強度である。 Consider the case where a subject is illuminated with a single point light source. The spectral radiance (reflected light spectrum) I (x, λ) of the reflected light from one point x on the object surface observed when observing the subject from a certain direction can be assumed when a standard two-color reflection model can be assumed. It is represented by the formula (10). This is basically the same formula as the previous formula (1) (the previous formula (1) is a simplified formula in which the position coordinate x is omitted).

x is the position coordinate of the object surface (x = [x, y, z] ^T ), λ is the wavelength, I _d (x, y) is the diffuse reflection component, I _s (x, λ) is the specular reflection component, E ( λ) is the spectral spectrum of the illumination light, D (x) is the brightness of the diffuse reflection component, f _d (x, λ) is the diffuse reflectance, and S (x) is the intensity of the specular reflection.

とし、Ｆ（ｘ，λ）を、ベクトルＵ（λ）を法線にもつ超平面に投影し、Ｕ（λ）に直交する成分Ｐ（ｘ，λ）を次のように求める。 Here, the total reflectance image F (x, λ)

And F (x, λ) is projected onto a hyperplane having the vector U (λ) as a normal line, and a component P (x, λ) orthogonal to U (λ) is obtained as follows.

これにより、Ｐ（ｘ，λ）は物体の拡散反射率ｆ_ｄ（ｘ，λ）のみに依存することが分かる。また、ｆ_ｄ（ｘ，λ）は、照明光に依存しない物体固有の関数であるため、入力画像と照合用画像とで撮影時の照明の種類が異なる場合でも、Ｐ（ｘ，λ）は分光スペクトル空間内内で同じ方向を指す（Ｐ（ｘ，λ）を正規化したものが等しくなる）。なお、Ｐ（ｘ，λ）の正規化はＰ（ｘ，λ）／｜Ｐ（ｘ，λ）｜で求まる。

This shows that P (x, λ) depends only on the diffuse reflectance f _d (x, λ) of the object. Further, since f _d (x, λ) is an object-specific function that does not depend on illumination light, P (x, λ) can be obtained even when the type of illumination at the time of shooting differs between the input image and the verification image. It points in the same direction in the spectrum space (normalized P (x, λ) is equal). The normalization of P (x, λ) is obtained by P (x, λ) / | P (x, λ) |.

  According to the present invention, it is possible to visualize a texture or the like hidden by specular reflection, and it is possible to suppress a decrease in recognition accuracy due to specular reflection when comparing an input image and an accumulated image (collation image). . Therefore, for example, object recognition is facilitated in an office environment in which a plurality of illumination light sources exist, a room with a window, or an illumination environment different from that when an accumulated image is captured. Further, the apparent reflectance on the object surface (each pixel) is calculated using the illumination light spectrum, and the obtained result is expressed as a vector U (λ) = [1, 1,..., 1] (values of all components are The same image is projected onto a hyperplane with the normal as the normal line, and the image from which the specular reflection component has been removed is used. Collation by simple comparison for each value is possible.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of an object recognition system according to an embodiment of the present invention. The object recognition apparatus 1 is constituted by a computer such as a personal computer, for example, and controls exposure time, image capture, object recognition, and other processes of the digital CCD camera 2 that is a photographing apparatus. The captured image is stored inside the object recognition apparatus 1. The object recognition apparatus 1 stores an image previously captured for verification, an illumination light component at the time of capturing, a specular reflection removed image, and the like. The photographing result of the subject 3, the specular reflection removal image, the result of the object recognition processing, and the like are displayed on the result display monitor 4. The illumination light source group 5 is composed of a plurality of light sources having different spectral spectra, and can be controlled to be turned on and off independently by an illumination control device (not shown). Note that the object recognition device 1 may also serve as a lighting control device. An image of the white plate 6 is taken by the digital CCD camera 2 and used for calculating the illumination light component of each light source.

  FIG. 2 is an overall configuration diagram showing an embodiment of the object recognition apparatus 1 according to the present invention. The object recognition apparatus 1 captures an input image captured by the imaging apparatus, an imaging apparatus characteristic measuring unit 11 that measures input and output characteristics of the imaging apparatus, an illumination light information acquisition unit 12 that acquires an illumination light component of the illumination light source, and the like. Image capturing unit 13, specular reflection removing unit 14 that generates a specular reflection removal image that does not include a specular reflection component from an input image, an image that has been previously captured for verification, an illumination light component at the time of photographing, and further, specular reflection removal An image storage unit (collation image database) 15 that stores images and the like, an image analysis unit (image recognition unit) 16 that performs recognition processing by collating the specular reflection-removed image with a collation image, an object recognition result, or an input The image and its specular reflection removal image, and further, an image display control unit 17 that controls display of the stored image and part or all of the specular reflection removal image. .

  FIG. 3 shows an overall process flow diagram of the object recognition apparatus 1. The imaging device characteristic measurement unit 11 measures input and output characteristics of the digital CCD camera 2 that is an imaging device in advance (S1001). The imaging device characteristic data of the measurement result is sent to the illumination light information acquisition unit 12 and the image capture unit 13.

  The illumination light information acquisition unit 12 captures an image of the white plate 6 photographed by the digital CCD camera 2, converts the signal value by referring to the photographing device characteristic data (γ correction processing), and standardized signal value The illumination light component is acquired from the data and stored (S1002). This illumination light component is sent to the specular reflection removing unit 14.

  The image capturing unit 13 captures an image of the subject 3 photographed by the digital CCD camera 2, converts signal values with reference to photographing device characteristic data (γ correction processing), and outputs a standardized input image (S1003). Here, in the input image captured by the digital CCD camera 2, brightness and color information relating to the specular reflection component is recorded.

The specular reflection removing unit 14 divides the spectral radiance of the input image by the illumination light component to generate an overall reflectance image, and has the vector [1, 1,..., 1] as a normal line. By projecting onto the hyperplane, a specular reflection removal image is obtained ( S1004 ).

  The image analysis unit 16 uses the specular reflection removal image obtained by the specular reflection removal unit 14, and collates the specular reflection removal image with a collation image stored in advance in the image storage unit 15 to recognize the image. Processing is performed, and the recognition result of the object in the image is output (S1005). Here, as the verification image, an image (accumulated image) that has been captured in advance and stored in the image storage unit 15 or a specular reflection-removed image thereof is used.

  The image display control unit 17 displays the object recognition result obtained by the image analysis unit 16 on the result display monitor 2 (S1006). At this time, the image display control unit 17 may display an input image, a specular reflection removed image, and the like together with the object recognition result.

  Although the necessary arrows are omitted in the configuration diagram of FIG. 2, the specular reflection removing unit 14 also inputs the illumination light component data of the accumulated image from the image accumulation unit 15, and the illumination light information acquisition unit 12 It is also possible to obtain a specular reflection-removed image by obtaining a projection space common to the input image and the accumulated image using the illumination light component data. In this case, the calculated projection space data is sent to the image storage unit 15. When the specular reflection removal image of the stored image is used as the collation image, the image storage unit 15 generates a specular reflection removal image of the stored image using, for example, the projection space data calculated by the specular reflection removal unit 14. The result is sent to the image analysis unit 16 as a verification image.

  FIG. 4 shows another processing flow diagram of the object recognition apparatus 1. 4, S2001 to S2004 are the same as S1001 to S1004 in FIG. In FIG. 4, if the specular reflection removal image of the input image is obtained in the specular reflection removal unit 14 (S2005), the image recognition processing in the image analysis unit 16 is not performed, and the image display control unit 17 The specular reflection-removed image, the image of the image storage unit 15 and the like are displayed on the result display monitor 4, and the recognition is left to human visual discrimination (S2005). In this case, the image analysis unit (image recognition unit) 16 can be omitted, and the configuration of the object recognition device is simplified.

  Note that the operation in the case where the image for collation, the illumination light component at the time of photographing, the specular reflection removal image, etc. is stored in advance in the image storage unit 15 is basically described in the processing flow diagram of FIG. Since it is the same as that, description is abbreviate | omitted.

Next, the configuration and operation of each part of the object recognition apparatus 1 shown in FIG. 2 will be described.
The imaging device characteristic measurement unit 11 measures input and output characteristics of the digital CCD camera 2 in advance by, for example, the method shown in FIG. Note that the automatic correction functions such as white balance, gain, and γ value of the target digital CCD camera 2 are turned off. First, a white color with a known brightness is displayed on a display device such as a CRT, and this is photographed by the digital CCD camera 2. Next, the brightness of the displayed white is changed, and an image is taken again. This operation is repeated several times, and the relationship between the brightness of the input light to the photographing apparatus (digital CCD camera) and the output signal value is measured. Based on the measurement result, a conversion formula (polynomial approximation or conversion table) is obtained as imaging device characteristic data for estimating the brightness of the input light from the output signal value from the imaging device (see FIG. 12).

  FIG. 5 shows a configuration example of the illumination light information acquisition unit 12. The illumination light information acquisition unit 12 includes an illumination light capture unit 121, a normalization processing unit 122, a normalization processing unit 123, and an illumination light component accumulation unit 124. An image (illumination light) of the white plate 6 photographed by the digital CCD camera 2 is captured by the illumination light information capturing unit 121, and the standardization processing unit 122 captures the photographing device characteristic data from the photographing device characteristic measuring unit 11. The signal value is converted with reference (γ correction processing) to obtain a standardized signal value. Further, the normalization processing unit 124 performs normalization processing so that the power becomes 1, and the result is stored in the illumination light component storage unit 125 as an illumination light component. The illumination control device 7 controls illumination of the illumination light source group 5 so that only one illumination light source is lit when the white plate 6 is imaged. This processing is performed on all illumination light sources, and the obtained data of a plurality of illumination light components is accumulated in the illumination light component accumulation unit 125 and sent to the specular reflection removal unit 14. Note that the illumination light information acquisition unit 12 may measure in advance illumination light components of a limited number of commonly used illumination light sources and store them in the illumination light component accumulation unit 124.

  FIG. 6 shows a configuration example of the image capturing unit 13. The image capturing unit 13 includes a normalization processing unit 131. An input image of the subject 3 photographed by the digital CCD camera 2 is converted into a standard by converting a signal value referring to the photographing device characteristic data from the photographing device characteristic measuring unit 11 (γ correction processing) in the standardization processing unit 131. The converted image is sent to the specular reflection removing unit 14. The standardized image includes brightness and color information regarding the glossy and specular reflection component.

  FIG. 7 shows a first configuration example of the specular reflection removing unit 14. This is a configuration example in the case where a specular reflection removed image is obtained by projecting an input image onto a space orthogonal to the illumination light component. The specular reflection removal unit 14 includes a projection space calculation unit 141 and a projection calculation processing unit 142. The projection space calculation unit 141 calculates a space (projection space data) orthogonal to all the illumination light components obtained by the illumination light information acquisition unit 12. Based on the result, the projection calculation processing unit 142 projects the signal value of each pixel of the normalized input image obtained by the image capturing unit 13 onto the projection space, and removes the specular reflection component. By performing this process for all the pixels, a specular reflection removed image is obtained.

  As described above, in the specular reflection removing unit 14, the illumination light component data of the accumulated image from the image storage unit 15 and the illumination light component data at the time of photographing a human power image taken by the illumination light information acquiring unit 12 are used. The projection space calculation unit 141 calculates the projection space data common to the input image and the stored image, and the projection calculation processing unit 142 calculates each of the normalized input images obtained by the image capturing unit 13. The signal value of the pixel may be projected onto the projection space, and the specular reflection component may be removed. The projection sound data common to the input image and the accumulated image calculated by the projection space calculating unit 141 is also sent to the image storing unit 15.

  Although omitted in FIG. 7, a normalization processing unit is provided on the output side of the projection calculation processing unit 142 so that each pixel value of the specular reflection-removed image obtained by the projection calculation processing unit 142 is normalized by its power. It may be. Thereby, a specular reflection removal image (illumination invariant image) independent of the illumination condition is obtained.

FIG. 8 shows a second configuration example of the specular reflection removing unit 14. This generates a total reflectance image by dividing the spectral radiance of the input image by the illumination light component, and this total reflectance image has a vector U (λ) = [1, 1,..., 1] as a normal line. It is a structural example of this invention in the case of obtaining a specular reflection removal image by projecting on a hyperplane. The specular reflection removal unit 14 includes a normal vector setting unit 143, a total reflectance image generation unit 144, and a projection calculation processing unit 145. The normal vector setting unit 143 holds a vector U (λ) = [1, 1,..., 1] in advance. The dimension (number of elements) N of the vector corresponds to the color component of the image to be acquired. For example, when the color component is 3, N = 3 and U (λ) = [1, 1, 1]. The total reflectance image generation unit 144 generates a total reflectance image from the illumination light component data group obtained by the illumination light information acquisition unit 12 and the input image obtained by the image capturing unit 13. That is, the apparent reflectance at each pixel of the input image is calculated using the illumination light spectrum. In the projection calculation processing unit 145, the vector [1, 1,..., 1] is input from the normal vector setting unit 143, and the total reflectance image generation unit 144 obtains the hyperplane having the vector as a normal line. The signal value (pixel value) of each pixel of the total reflectance image is projected to remove the specular reflection component. By performing this process for all pixels, a specular reflection-removed image from which the specular reflection component has been removed can be obtained from the input image.

  Although omitted in FIG. 8, a normalization processing unit is provided on the output side of the projection calculation processing unit 145 so that each pixel value of the specular reflection removal image obtained by the projection calculation processing unit 142 is normalized by its power. It may be. Thereby, a specular reflection removal image (illumination invariant image) independent of the illumination condition is obtained.

  FIG. 9 shows a first configuration example of the image storage unit (collation image database) 15. The main image storage unit 15 includes an illumination light component storage unit 151, a normalized image storage unit 152, and a specular reflection removed image storage unit 153. The standardized image storage unit 152 and the specular reflection removal image storage unit 153 store the standards calculated by the image capturing unit 13 and the specular reflection removal unit 14 for all images previously captured by the digital CCD camera 2 for verification. Each of the converted image and the specular reflection removed image is accumulated. The illumination light component storage unit 151 stores the illumination light component data of the illumination light source from the illumination light information acquisition unit 12 when the verification image is captured. The standardized image of the standardized image storage unit 152 and / or the specular reflection removal image of the specular reflection removal image storage unit 153 are sent to the image analysis unit 16 as a verification image.

  FIG. 10 shows a second configuration example of the image storage unit 15. The main image storage unit 15 includes an illumination light component storage unit 151, a normalized image storage unit 152, a projection space calculation unit 154, and a projection calculation processing unit 155. The illumination light component storage unit 151 and the normalized image storage unit 152 are the same as those in FIG. In FIG. 10, the specular reflection removal image storage unit 153 is eliminated, and the specular reflection removal image as the verification image is generated by the image storage unit 15 itself. Necessary projection space data is calculated in the same manner by the image storage unit 15 itself, or the data sent from the specular reflection removal unit 14 is used. That is, the projection space calculation unit 154 calculates projection space data from the illumination light component in the illumination light component storage unit 151 corresponding to each standardized image in the standardized image storage unit 152, and the projection calculation processing unit 155 Then, a specular reflection removed image of each standardized image (accumulated image) of the standardized image storage unit 152 is calculated, and this is used as a verification image. Alternatively, the illumination light component data in the illumination light component accumulating unit 151 is sent to the specular reflection removing unit 14, and the projection calculation processing unit 154 uses the obtained input image and the projection space data common to the accumulated image to obtain a normalized image. A specular reflection removal image of each normalized image (accumulated image) in the accumulation unit 152 is calculated, and this is used as a collation image.

  In general, any or all of the matching images described with reference to FIGS. 9 and 10 can be used as matching images.

  FIG. 11 shows a configuration example of the image analysis unit 16. The image analysis unit 16 basically includes an image matching unit 161. The image collating unit 161 collates the specular reflection removed image from the specular reflection removing unit 14 with the collating image from the image storage unit 15 to obtain an object recognition result. Note that the image analysis unit 16 analyzes the specular reflection removal image from the specular reflection removal unit 14 and performs the recognition result without performing the collation processing with the image for collation from the image storage unit 15 as necessary. You may make it obtain.

  The image display control unit 17 displays the recognition result of the object from the image analysis unit 15 on the result display monitor 2. A configuration diagram of the display control unit 17 is omitted. As described above, in the display control unit 17, the input image from the image capturing unit 13 and the specular reflection removal unit 14 and the specular reflection removal image thereof, the accumulation image from the image storage unit 15 and the specular reflection removal thereof are included. It is also possible to display only the image (image for comparison) or the input image and the specular reflection-removed image on the result display monitor 4 so that the person can visually discriminate.

  Although one embodiment of the present invention has been described above, some or all of the processing functions of each part in the object recognition apparatus shown in FIG. 2 are configured by a computer program, and the program is executed using the computer. Needless to say, the invention can be realized, or the processing procedure shown in FIGS. 3 and 4 can be configured by a computer program and the program can be executed using the computer. In addition, a computer-readable recording medium such as an FD, MO, ROM, memory card, CD, or the like is stored in a computer-readable recording medium. In addition, the program can be recorded and stored on a DVD, a removable disk, etc., and the program can be distributed through a network such as the Internet.

  The object recognition technique of the present invention can be applied to object search and object recognition by a robot, object shape estimation, character recognition, image search on a database, texture analysis with an endoscopic image, and the like.

1 is a schematic configuration diagram of an object recognition system according to an embodiment of the present invention. It is a figure which shows the example of whole structure of the object recognition apparatus in this invention. FIG. 3 is a diagram showing an overall processing flow (part 1) of FIG. 2. It is a figure which shows the whole processing flow (the 2) of FIG. It is a figure which shows the detailed structure of an illumination light information acquisition part. It is a figure which shows the detailed structure of an image acquisition part. It is a figure which shows the detailed structure (the 1) of a specular reflection removal part. It is a figure which shows the detailed structure (the 2) of a specular reflection removal part. It is a figure which shows the detailed structure (the 1) of a storage part, such as an image. It is a figure which shows the detailed structure (the 2) of storage parts, such as an image. It is a figure which shows the detailed structure of an image analysis part. It is a figure explaining the measurement of the input-output characteristic of an imaging device. It is a figure explaining the principle of a specular reflection component removal.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object recognition apparatus 2 Digital CCD camera 3 Subject 4 Monitor for result display 5 Illumination light source group 6 White board 7 Illumination control apparatus 11 Imaging apparatus characteristic measurement part 12 Illumination light information acquisition part 13 Image capture part 14 Specular reflection removal part 15 Image Equal accumulation unit 16 Image analysis unit 17 Image display control unit S1001, S2001 Imaging device characteristic measurement step S1002, S2002 Illumination light component measurement step S1003, S2003 Image capture step S1004, S2004 Specular reflection component removal step S1005 Image analysis step S1006 Recognition result, etc. Display Step S2005 Display Step of Input Image, Specular Reflection Removal Image, etc.

Claims (14)

An object recognition apparatus that recognizes an object by collating a specular reflection-removed image obtained by removing a specular reflection component from an input image, which is an image of the object, and an accumulated image that is an image of the object photographed in advance. And
Illumination light information acquisition means for acquiring an illumination light component of an illumination light source at the time of capturing the input image ;
An image capturing means for capturing an input image captured by the imaging device and a spectral radiance of reflected light from the surface of the object ;
Wherein the stored image is an image of the object which is previously shot and an illumination light component at the time of photographing of the stored images, and storage means for at least storing,
A total reflectance image is generated by dividing the spectral radiance of the reflected light from the surface of the object by the illumination light component acquired by the illumination light information acquisition means , and the signal value of the total reflectance image is expressed as a vector U ( Specular reflection removal that generates a specular reflection removal image from which the specular reflection component is removed from the input image by projecting onto the hyperplane with λ) = [1,1,..., 1] (λ is the wavelength) as a normal line Means,
An object recognition apparatus comprising: 2. The object recognition apparatus according to claim 1, further comprising image analysis means for collating the specular reflection-removed image with the accumulated image accumulated in the accumulation means. 3. The object recognition apparatus according to claim 1, wherein the input image and the specular reflection removal image of the input image, or the input image, the specular reflection removal image of the input image and the accumulated image are generated in advance and object recognition apparatus characterized by further having an image display control means for displaying a specular reflection removal image of the stored image which is stored along with the stored images.   The object recognition apparatus according to any one of claims 1 to 3, further comprising a photographing device characteristic measuring unit that measures input and output characteristics of the photographing device in advance, wherein the image capturing unit includes the photographing device characteristic measurement. An object recognition apparatus characterized in that a standardized image is obtained based on input / output characteristics of a photographing apparatus as a measurement result of a means.   4. The object recognition apparatus according to claim 1, wherein the specular reflection removing unit removes the illumination light components related to all illumination light sources from the input image, so that the specular reflection component does not include the specular reflection component. 5. An object recognition apparatus that generates a removed image.   The object recognition apparatus according to claim 1, further comprising an illumination light component accumulating unit that accumulates illumination light components of finite types of commonly used illumination light sources in advance. An object recognition apparatus characterized in that the reflection removal means obtains a specular reflection removal image using the accumulated illumination light component. An object recognition method for recognizing an object by collating a specular reflection-removed image obtained by removing a specular reflection component from an input image that is an image of the object and an accumulated image that is an image of the object that has been captured in advance. And
Illumination light information acquisition step of acquiring an illumination light component of an illumination light source at the time of capturing the input image;
  An image capturing step for capturing an input image captured by the imaging device and a spectral radiance of reflected light from the surface of the object;
An accumulation control step of accumulating at least an accumulation image that is an image of the object that has been imaged in advance and an illumination light component at the time of imaging of the accumulation image;
The spectral radiance of the reflected light from the surface of the object is divided by the illumination light component acquired in the illumination light information acquisition step to generate a total reflectance image, and the signal value of the total reflectance image is expressed as a vector U ( Specular reflection removal that generates a specular reflection removal image from which the specular reflection component is removed from the input image by projecting onto the hyperplane with λ) = [1,1,..., 1] (λ is the wavelength) as a normal line Steps,
An object recognition method characterized by comprising: The object recognition method according to claim 7.
An object recognition method, further comprising an image analysis step of collating the specular reflection-removed image with the accumulated image accumulated in the accumulation means. 9. The object recognition method according to claim 7, wherein the input image and the specular reflection removal image of the input image, or the input image, the specular reflection removal image of the input image, and the accumulated image are generated in advance. An object recognition method, further comprising an image display control step of displaying on the display means a specular reflection-removed image of the accumulated image accumulated together with the accumulated image. 10. The object recognition method according to claim 7, further comprising a photographing device characteristic measuring step for measuring input and output characteristics of the photographing device in advance, wherein the photographing device characteristic measurement is performed in the image capturing step. An object recognition method characterized by obtaining a standardized image based on input / output characteristics of a photographing apparatus, which is a measurement result of a step. 10. The object recognition method according to claim 7, wherein in the specular reflection removing step, the specular reflection not including the specular reflection component is performed by removing the illumination light components related to all illumination light sources from the input image. An object recognition method comprising generating a removed image. 10. The object recognition method according to claim 7, further comprising: an illumination light component accumulation control step in which illumination light components of a finite type of illumination light source that is generally used are accumulated in an accumulation unit in advance. An object recognition method further comprising: obtaining a specular reflection removal image using the accumulated illumination light component in the specular reflection removal step. The program for functioning a computer as an object recognition apparatus of any one of Claims 1 thru | or 6. A computer-readable recording medium on which a program for causing a computer to function as each unit of the object recognition device according to any one of claims 1 to 6 is recorded.

Images