JP6671653B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for estimating reflected light and fluorescence from an image of an object to be measured, and a program to which the image processing method is applied.

  When an object is irradiated with light, an observer looking at the object not only sees reflected light generated by directly reflecting incident light on the surface of the object, but also, depending on the properties of the object, Material) may absorb light and emit fluorescence. Fluorescence is a phenomenon in which when an object is irradiated with light that excites the fluorescence, the object emits light having a wavelength different from that of the irradiated light. The reflected light reflects light of the same wavelength as the light incident on the object, whereas in the case of fluorescent light, it emits light of a wavelength longer than the wavelength of light (absorption) that is incident on and absorbed by the object. I do.

FIG. 12 shows an example of characteristics of reflected light in the visible light region for three objects (lettuce, tomato, and butter). The reflectivity shown on the vertical axis in FIG. 12 is a relative value measured individually for each object, and does not indicate the magnitude relationship of the reflected light from the three objects.
For example, lettuce a has an increased reflectance in the green region. Therefore, the lettuce looks green to the observer. The reflectance of tomato b increases in the red region. Therefore, tomatoes appear red to the observer.
The reflected light of each color (wavelength) directly reflects the light irradiated on each object. That is, irradiating the lettuce with light in the green wavelength band causes the lettuce to appear green, and irradiating the tomato with light in the red wavelength band causes the tomato to appear red.
On the other hand, in the case of the above-described fluorescence, light on the long wavelength side, which is light having a wavelength different from the light incident on each object, is emitted. It is known that the fluorescent component is emitted from various objects, but the wavelength and distribution characteristics change depending on the object.

  Heretofore, in order to accurately detect the reflected light and the fluorescent light while distinguishing them, an extremely complicated and highly accurate analyzer has been required. For example, the wavelength band of the ultraviolet light and the visible light is divided into a plurality of narrow bands, and light of each band is irradiated on the object to be measured. Then, the wavelength of the light emitted from the object to be measured is measured by a measuring device such as an optical spectrum analyzer. When the light measured by the measuring instrument is only the component having the same wavelength as the irradiated light, it is determined that the reflected light is only. Further, when the light measured by the measuring instrument has a longer wavelength range than the irradiated light, the light in the longer wavelength range is determined to be fluorescent.

In this way, the emission and measurement of light by ultraviolet light and visible light divided into narrow bands are performed in the entire wavelength range of visible light, so that the reflected light and fluorescence of the measured object are separated and detected. can do.
By accurately detecting the reflected light and the fluorescence of the object to be measured, it is possible to find out, for example, the place of production and the type of plants such as agricultural products. For example, it is known that the wavelength distribution of the fluorescent component of mango varies depending on the place of production. Specifically, mango from Okinawa in Japan, mango from Miyazaki in Japan, and mango from Taiwan can be accurately determined from the wavelength distribution of the fluorescent component. It is also said that the amount of buckwheat flour contained in the buckwheat can be determined from the wavelength distribution of the fluorescent component.

Patent Literature 1 describes a technique for calculating a feature amount of fluorescence from images of a plurality of different wavelength bands in order to specify a fluorescence wavelength band emitted by a specimen.
Patent Literature 2 discloses a programmable light source having a wavelength characteristic that repeats intensity at predetermined wavelength intervals, irradiates a subject with two illumination lights in which the intensity of the programmable light source is switched, and performs shooting for each illumination light. A technique for detecting fluorescence from a difference between images captured twice is described.

JP 2013-114233 A International Publication WO2015 / 080275

  As described in Patent Literature 1, in order to perform imaging for each of a plurality of bands having a narrow wavelength band and to detect fluorescence, an extremely large number of times of imaging is required, and the imaging operation is troublesome. There was a problem. In particular, when measuring the entire target object or the entire target scene instead of measuring the fluorescence component at one point of the target object, there is a problem that it takes much more time for the measurement, which is an extremely difficult task. Was. Further, there is a problem that a measuring instrument such as an optical spectrum analyzer is very expensive and the handling thereof is complicated, and an analysis system using the measuring instrument becomes expensive.

On the other hand, in the technique described in Patent Literature 2, imaging is performed twice by changing the conditions of the light source, and fluorescence is detected from the difference between the images captured twice. Thus, there is an advantage that fluorescence can be easily detected without using a special measuring device such as an optical spectrum analyzer.
However, in the technique described in Patent Document 2, it is necessary to perform two shootings on one measurement target object by changing the conditions of the light source, and there is a problem that a moving target object cannot be measured. . Further, the programmable light source used in the technique described in Patent Document 2 has a problem that an extremely expensive special light source is required.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and a program that can accurately and easily estimate fluorescence even when a measured object is a moving object.

An image processing apparatus according to an aspect of the invention includes a light source that emits illumination light having characteristics in which a frequency component higher than a frequency component of the spectral distribution of fluorescence is included in a plurality of wavelength bands, and a measurement target irradiated with the illumination light by the light source. A spectral camera that obtains a spectral component by spectrally capturing an object in a plurality of wavelength bands to obtain a spectral component, and a spectral component for each of a plurality of wavelength bands of image data of one frame obtained by the spectral camera are used to determine an object to be measured. Estimate the reflection and fluorescence of the measured object by calculation using the principal component analysis unit that approximates the spectral reflectance and fluorescence with multiple bases, and the calculation using the characteristics of the illumination light and the multiple bases obtained by the principal component analysis unit And a fluorescence estimating unit.

Further, the image processing method of the present invention uses a spectroscopic camera to measure a plurality of measurement targets irradiated with illumination light having characteristics in which a frequency component higher than the frequency component of the spectral distribution of the fluorescent component is included in a plurality of wavelength bands. A plurality of spectral reflectances and a plurality of fluorescent components of an object to be measured are obtained from a photographing process of spectrally photographing in a wavelength band of a plurality of wavelength bands and spectral components of one frame of image data obtained in the photographing process for each of a plurality of wavelength bands. And an estimation step of estimating the reflection and fluorescence of the measured object by an operation using the characteristics of the illumination light and a plurality of bases obtained by the principal component analysis unit.

  Further, the program of the present invention is used by being mounted on a computer in order to execute each step of the image processing method by the computer.

  According to the present invention, a reflected light component to be observed is obtained by photographing an object to be measured illuminated with illumination light having a characteristic in which a frequency component higher than a frequency component of the fluorescence spectral distribution is included in a plurality of wavelength bands. Is expressed as the product of the base and the high-frequency illumination light, and is different from the base of the fluorescent component, so that it is possible to perform an operation to separate the reflected light component and the fluorescent component contained in the captured image, and to accurately estimate the fluorescence. become.

FIG. 1 is a configuration diagram illustrating a system according to an embodiment of the present invention. FIG. 3 is an explanatory diagram showing a general relationship among illumination light, spectral reflectance, absorption component, and fluorescence emission. It is explanatory drawing which shows the example of the reflection and fluorescence change when there is no high frequency component in illumination light (FIG. 3A) and when there is a high frequency component (FIG. 3B). 5 is a flowchart illustrating a flow of a process according to an embodiment of the present invention. FIG. 5B is a characteristic diagram showing an example of the spectral reflectance of various objects according to the embodiment of the present invention (FIG. 5A), and an example of the top five bases of the spectral reflectance of an object (FIG. 5B). FIG. 6 is a diagram illustrating an example of a linear sum of n bases according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an example of ideal light source characteristics when estimating fluorescence by applying an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a characteristic of a programmable light source when estimating fluorescence by applying an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of characteristics of an HID lamp when fluorescence is estimated by applying an embodiment of the present invention. FIG. 4 is a characteristic diagram illustrating an example of a light source that cannot be applied to fluorescence estimation for comparison with the present invention. FIG. 11A is a diagram of an example in which reflection and fluorescence components are estimated in an embodiment of the present invention. FIG. 11A shows reflection and fluorescence components of a target object observed in green, and FIG. 11B shows a target object observed in red. 2 shows the reflection and the fluorescence component of the light. It is a figure showing an example of a reflection characteristic.

  Hereinafter, an embodiment of the present invention (hereinafter, referred to as “present example”) will be described with reference to FIGS. 1 to 11.

[1. System configuration example]
FIG. 1 is a diagram illustrating the entire system of the image processing apparatus according to the present embodiment. This system detects reflected light from a subject 90, which is an object to be measured, and estimates reflection and fluorescence emitted from the subject 90 by an operation using the detected reflected light component.

  The subject 90 is photographed by the camera 40 in a state where illumination light from the light source 20 is emitted. In the case of this example, the illumination light emitted by the light source 20 has a characteristic in which a frequency component higher than the frequency component of the spectral distribution of the fluorescent component to be detected (estimated) is included in a plurality of wavelength bands. One of the light sources satisfying such a condition is, for example, a high-intensity discharge lamp (hereinafter, referred to as an “HID lamp”). HID lamps are used for headlights of automobiles and the like. One example of using an HID lamp is to use a light source of another type as long as the light source emits illumination light having a characteristic in which a frequency component higher than the frequency component of the fluorescent component is included in a plurality of wavelength bands. May be. A specific example of the characteristics of the illumination light will be described later.

Then, with the light source 20 outputting the illumination light, the camera 40 captures an image of the subject 90 in a capturing step.
As the camera 40, a spectral camera that divides a range including a band of visible light into a predetermined number of bands (for example, 30 bands) and obtains data of each pixel for each of the divided bands is used. Also, the camera 40 is capable of shooting a moving image by continuously shooting at a predetermined frame rate. However, when the subject 90 is stationary, the camera 40 shoots a still image. Forty may be used.

Image data obtained by photographing with the camera 40 is sent to the image processing unit 30, and the image processing unit 30 performs an arithmetic process of estimating a reflection component and a fluorescence component for each pixel in the image.
The image processing unit 30 includes a principal component analysis unit 31 and a fluorescence estimation unit 32 as a processing unit for estimating reflection and fluorescence from a camera image.
The principal component analysis unit 31 performs a principal component analysis process of the spectral reflectance and the fluorescence emission, and converts the spectral reflectance and the spectral distribution of the fluorescence emission into a specific number of basis linear models. The principal component analysis unit 31 obtains a specific number of base linear models by frequency-analyzing the spectral reflectance by, for example, Fourier transform. However, it is one example that the principal component analysis unit 31 performs the frequency analysis in real time, and instead of performing the frequency analysis, the database has data corresponding to the spectral reflectance and the linear model of the basis, and refers to the database. Alternatively, a base linear model may be obtained from the spectral reflectance. Here, processing for obtaining a base linear model using a database is applied. Further, a linear model of the basis of the spectral reflectance is obtained from the database of the spectral reflectance of the color, and a linear model of the basis of the fluorescence is also obtained from the database showing the spectral distribution of the fluorescence emission.
Then, the fluorescence estimation unit 32 estimates the reflection component and the fluorescence component based on the linear basis of the spectral reflectance, the basis linearity of the fluorescence (the coefficient is unknown), and the characteristics of the illumination light. The process of estimating the reflection component and the fluorescence component is a process of estimating the coefficient of the basis line of the reflection and the fluorescence, and will be described later in detail.

Then, the image processing unit 30 obtains image data obtained by extracting the reflection and each component of the reflection and the fluorescence component estimated by the fluorescence estimation unit 32 from the image data obtained by photographing by the camera 40. Each image data representing only the reflection component and the fluorescence component obtained by the image processing unit 30 is supplied to the image display unit 50, and the image of the reflection and the fluorescence component is displayed on the image display unit 50.
The fluorescence image data obtained by the image processing unit 30 is supplied to the characteristic analysis unit 60. The characteristic analysis unit 60 analyzes the distribution state of the fluorescent characteristics of the subject 90 from the fluorescent image data. Note that the characteristic analysis unit 70 may analyze the distribution state of the reflected light component and the distribution state of the light absorption component. The processing in the image processing unit 30 is executed under the control of the control unit 10.

[2. Explanation of the principle of estimating fluorescence]
Next, the principle of estimating the fluorescence by the image processing apparatus of the present embodiment will be described with reference to FIGS.
FIG. 2 is a diagram illustrating an example of a fluorescent component E contained in light obtained by observing an object when the object is illuminated with the illumination light L. In FIG. 2, the horizontal axis is wavelength and the vertical axis is intensity.
It is known that the color of an object surface observed by irradiating illumination light is the product of the spectral distribution of the illumination light and the spectral reflectance of the object surface. The spectral reflectance indicates the ratio of reflection of incident light to each wavelength.

この式は蛍光がない場合であるが、実際には多くの物質は、図２に示すように蛍光成分Ｅ（λ）を含んでいる。つまり、観察光の分光分布Ｐ（λ）には、分光分布ｌ（λ）と分光反射率Ｒ（λ）に、蛍光成分Ｅ（λ）を加算する必要がある。但し、蛍光成分Ｅ（λ）には、照明光と吸光度との積で示される係数が乗算される。この係数は蛍光発光の強度を決定するものであり、物体の表面での各派長の光の吸収度合いを示す吸光度ａ（λ）と分光分布ｌ（λ）の積分により求められる。

吸収成分ａ（λ）は、蛍光成分ｅ（λ）とは異なる波長域，短波長側に現われる。
最終的に反射成分と蛍光成分が観察される場合の式は、

ここで、ｐ₁(λ)は観察光p(λ)に含まれる反射成分，ｐ_２(λ)は観察光p(λ)に含まれる蛍光成分を表している。
各波長での観察光p(λi)は、以下の行列式で表すことができる。

本発明で必要な処理は、観察光ｐ(λ)から分光反射率r(λ)と蛍光e(λ)を推定することであるが、観測値の数が未知数よりも少ないため、推定は困難となる。すなわち、ｎ個の観測値（p(λ1), p(λ2), …, p(λn)）に対して、未知数が2n (（r(λ1), r(λ2), …, r(λn)およびe(λ1), e(λ2), …, e(λn)）となってしまうため、推定は不可能である。推定する未知数の数をｎ以下に減らすため，本発明では分光反射率r(λ)と蛍光発光成分e(λ)を、基底表現を用いて表す。 Here, when the object to be measured does not emit fluorescence at all, the product of the spectral distribution of the illumination light and the spectral reflectance of the object surface is the spectral value of each pixel of the captured image. For example, when the spectral distribution of the illumination light is l (λ) and the spectral reflectance of the object is r (λ), the spectral distribution P ₁ (λ) of the reflected light captured as the captured image is obtained by the following equation.

This equation is for no fluorescence, but in practice many substances include a fluorescent component E (λ) as shown in FIG. That is, it is necessary to add the fluorescence component E (λ) to the spectral distribution l (λ) and the spectral reflectance R (λ) to the spectral distribution P (λ) of the observation light. However, the fluorescence component E (λ) is multiplied by a coefficient represented by the product of the illumination light and the absorbance. This coefficient determines the intensity of the fluorescent light emission, and is obtained by integrating the absorbance a (λ) indicating the degree of absorption of each major light on the surface of the object and the spectral distribution l (λ).

The absorption component a (λ) appears on a shorter wavelength side than the fluorescence component e (λ).
The equation when the reflection component and the fluorescence component are finally observed is:

Here, p ₁ (λ) represents a reflection component contained in the observation light p (λ), and p ₂ (λ) represents a fluorescence component contained in the observation light p (λ).
The observation light p (λi) at each wavelength can be expressed by the following determinant.

The processing required in the present invention is to estimate the spectral reflectance r (λ) and the fluorescence e (λ) from the observation light p (λ), but the estimation is difficult because the number of observations is smaller than the unknown. Becomes That is, for n observations (p (λ1), p (λ2),..., P (λn)), the unknown is 2n ((r (λ1), r (λ2),..., R (λn) And e (λ1), e (λ2),..., E (λn)), so that the estimation is impossible. (λ) and the fluorescence emission component e (λ) are represented using a base expression.

3A and 3B show the characteristic l (λ) r (λ) of the reflected light when the object is irradiated with the illumination light (left side in each figure) and the characteristic we (λ) of the fluorescent component (right side in each figure). Are shown side by side. In each characteristic diagram, the horizontal axis represents wavelength, and the vertical axis represents intensity.
FIG. 3A shows a case where the illumination light applied to the object does not have the condition described with reference to FIG. 1, that is, does not have a frequency component higher than the high frequency component included in the fluorescent component (a smooth component such as a halogen bulb or sunlight). FIG. 5 shows reflected light in the case of illumination light having a spectral distribution) and a fluorescent component appearing in a specific wavelength range.

  On the other hand, FIG. 3B shows the reflected light when the illumination light applied to the object has a higher frequency component than the high frequency component included in the fluorescent component, and the fluorescent component appearing in a specific wavelength range. Here, Ea indicates the spectral distribution of the illumination light and the fluorescence intensity calculated based on the absorbance of the fluorescent substance described with reference to FIG. 3A, and Eb indicates the spectral distribution and the fluorescent substance under the illumination light of FIG. 3B. 3A and 3B, it can be seen that the form of the fluorescent component does not change even if the illumination light contains a high-frequency component. The illumination light contains high frequency components. Although details of an example of the light source will be described later, for example, there is a light source having characteristics as shown in FIG. 9A as an example. Therefore, the reflected light that reflects the illumination light also includes a sharp component. For example, reflected components include sharp components Pa, Pb, and Pc as shown in FIG. 3B.

  According to the present invention, as shown in FIG. 3B, when the illumination light includes a high-frequency component, the reflected light component includes the sharp component of the illumination light, and the fluorescence component includes a high-frequency spectral component of the illumination light. Utilizing that the form of the fluorescence emission wavelength does not change even in the case of having a distribution, a process of estimating the fluorescence component by calculation is performed.

FIG. 4 shows a procedure performed by the image processing apparatus according to the present embodiment until an image of a subject is captured and a fluorescent component is estimated.
First, based on an instruction from the control unit 10, the camera 40 captures one frame of a subject illuminated with the illumination light L from the light source 20, and the image processing unit 30 captures one frame of a spectral image (step S1). S11).

The principal component analysis unit 31 of the image processing unit 30 performs principal component analysis of the spectral reflectance using the database, and obtains k1 (k1 is an integer, for example, 8) base linear models. Here, the image processing unit 30 has a database in which the correspondence between the spectral reflectance and the linear model of the base is stored. Using the database, an image obtained by approximating the corresponding linear model of the base of k1 from the spectral reflectance. Estimate.
In addition, as for the fluorescent component, an approximation by k2 (k2 is an integer, for example, 12) base linear models is obtained (step S12). For this fluorescent component, a k2 basis linear model is obtained using the database of fluorescent emission. However, at this time, since the reflection component and the fluorescence component have not been estimated, the base coefficients of the reflection component and the fluorescence component are unknown, and a linear model of the base whose coefficient is unknown is obtained. Although the base numbers k1 and k2 are different numbers, they may be the same number.
It is known that when the spectral reflectance is represented by a k1 basis linear model, most components of the spectral reflectance can be represented by, for example, a limited number of top eight linear basis. It is also known that most of the fluorescent components can be expressed by about 12 linear bases similarly.
Further, the fluorescence estimating unit 32 of the image processing unit 30 sets the basis number of the spectral reflectance to k1, the basis number of the fluorescence component to k2, and calculates the estimated value of the fluorescence component by calculating a determinant using the characteristics of the illumination light L. Is calculated (step S13). The determinant used for estimating the reflection component and the fluorescence component will be described later.

  FIG. 5 shows an example in which the spectral reflectance is represented by k1 base linear models. FIG. 5A shows the spectral reflectance of various materials. In FIG. 5A, the horizontal axis represents the wavelength, and the vertical axis represents the intensity of the reflectance. FIG. 5B shows a linear model using the top k1 (here, k1 is 5) bases of the spectral reflectance of one specific substance. In FIG. 5B, the horizontal axis represents the wavelength, and the vertical axis represents the function value.

FIG. 6 shows the spectral reflectances decomposed by a linear model of n bases.
That is, characteristics of the spectral reflectance shown at the left end of Figure 6, decompose coefficients alpha _1, coefficient alpha _2, the coefficient alpha _3, · · ·, a linear model of the base that corresponds with the (k1) of different coefficients of coefficient alpha _k1 be able to. That is, one spectral reflectance is decomposed into k1 different frequency components.

全ての波長における分光反射率は行列式として記述できる。

The approximation of the spectral reflectance by the linear model described in FIG. 6 can be expressed by the following equation. Here, an example of a linear model using k1 (k1 dimension) bases is shown. α _j is the coefficient of the j-th base, and b _j (λ) is the form of the j-th reflectance base.

The spectral reflectance at all wavelengths can be described as a determinant.

照明光l(λ)が高い周波数分布を持つ場合（蛍光成分に含まれる高周波成分よりも高い周波数成分）、光源l(λ)と基底bj(λ)の積も、光源２０が出力する照明光が持つ高い周波数成分を示すことになる。図７〜９に基底と照明光の積の分光分布の例を示す。
同様に、蛍光成分の線形モデルによる近似は以下の式で表すことができる。β_ｊはｊ番目の基底の係数であり、c_ｊ（λ）は、ｊ番目の蛍光成分の基底の形である。

ある照明光l（λ）のもとで観察される蛍光成分p2(λ)は、蛍光発光eを基底ｃjで表現して記述することができる。

wは物体の表面での各派長の光の吸収度合いを示す係数であり、吸光度a（λ）と分光分布ｌ（λ）の積分により求められる。
蛍光成分の場合、観察される分光は吸収成分a（λ）と分光分布l（λ）の積分により計算される係数wをe(λ)かけるだけであり、観察される蛍光成分の周波数は蛍光基底の最高周波数は変化しない。このことは，照明光が高周波な分布を持つ場合にも発光波長の形態が変化しないことを意味している。 The reflection component p1 (λ) observed under a certain illumination light l (λ) can be described by expressing the spectral reflection r by a basis bj.

When the illumination light l (λ) has a high frequency distribution (a frequency component higher than the high-frequency component included in the fluorescent light component), the product of the light source l (λ) and the base bj (λ) also indicates the illumination light output from the light source 20. Will show the high frequency components of. 7 to 9 show examples of the spectral distribution of the product of the base and the illumination light.
Similarly, the approximation of the fluorescent component by the linear model can be expressed by the following equation. β _j is the coefficient of the j-th base, and c _j (λ) is the base form of the j-th fluorescent component.

The fluorescence component p2 (λ) observed under a certain illumination light l (λ) can be described by expressing the fluorescence emission e by the basis cj.

w is a coefficient indicating the degree of absorption of each major light on the surface of the object, and is obtained by integrating the absorbance a (λ) and the spectral distribution l (λ).
In the case of the fluorescent component, the observed spectrum is simply multiplied by e (λ) by the coefficient w calculated by integrating the absorption component a (λ) and the spectral distribution l (λ), and the frequency of the observed fluorescent component is The highest frequency of the base does not change. This means that the form of the emission wavelength does not change even when the illumination light has a high frequency distribution.

蛍光成分の線形係数βをw倍した係数β’を定義すると，次式を得る。

この式にもとづき、分光反射を表す線形係数αi(i=1,..k1)と蛍光成分を表すβ’i
i(i=1,..k2)を推定する。
反射特性と線形係数αと蛍光特性を表すβ’を推定できれば基底の線形和により獲反射特性および特性を計算することができる。ここで，未知数はk1+k2個であり、k1+k2が観察数以下、すなわちk1+k2<=nの場合、この式を解くことができる。 Then, the fluorescence estimating unit 32 of the image processing unit 30 performs a process of estimating the fluorescent component using the spectral reflectance expressed by the n linear models and the characteristics of the illumination light l. Will be Using the basis expression, the observation light including the reflection component and the fluorescence component can be described by the following determinant.

If a coefficient β ′ obtained by multiplying the linear coefficient β of the fluorescent component by w is defined, the following equation is obtained.

Based on this equation, a linear coefficient αi (i = 1, .. k1) representing the spectral reflection and β′i representing the fluorescence component
Estimate i (i = 1, .. k2).
If the reflection characteristic, the linear coefficient α, and β ′ representing the fluorescence characteristic can be estimated, the capture reflection characteristic and the characteristic can be calculated by the linear sum of the bases. Here, the number of unknowns is k1 + k2, and when k1 + k2 is equal to or less than the number of observations, that is, k1 + k2 <= n, this equation can be solved.

  In the above determinant, the basis bj of the spectral reflectance and the basis cj of the fluorescent component have similar spectral characteristics, so that the brightness observed under illumination light having a smooth spectral distribution such as a halogen bulb or sunlight. It is difficult to stably obtain both coefficients α and β using p (λ). That is, it is extremely difficult to separate the reflected light and the fluorescence from the determinant described above. Therefore, in the present invention, in consideration of the spectral distribution of the illumination light output from the light source 20, the product l (λ) bj (λ) of the basis in the above equation is different from the basis cj (λ) of the fluorescent component. By providing different characteristics, the linear coefficients αj and β′j are stably obtained.

  As described above, when the illumination light has a high frequency, the observed reflection component (the product of the spectral reflectance r and the light source 1) has a higher frequency component than the fluorescence component e. Similarly, also in the basis expression approximating the reflection component, the product of the basis bj (λ) approximating the spectral reflectance and the illumination light l (λ) has a high frequency. For this reason, the spectral characteristics obtained by the integration of the basis of the spectral reflectance and the illumination light (l (λ) bj (λ)) have a different spectral distribution from the basis cj (λ) that approximates the fluorescent component. The determinant described above can be solved by the unit 32, the spectral reflectance r (λ) can be restored by the linear sum of the estimated linear coefficient α and the basis b (λ), and the estimated linear coefficient β and the basis c ( The spectral distribution we (λ) obtained by multiplying the fluorescence component and the fluorescence component e (λ) by w can be restored by the linear sum of λ). However, in order to accurately estimate the fluorescence component e by solving the determinant, it is necessary that a plurality of high-frequency components (sharp components) included in the illumination light are dispersed in a certain wide band.

  By obtaining the reflection component R and the fluorescence component E in this way, the image processing unit 30 can convert the spectral image obtained by capturing the object to be measured into an image of the reflection component R and the fluorescence component E. The image of the reflection component R and the fluorescence component E obtained by the image processing unit 30 is displayed by the image display unit 50. Further, the characteristic analysis unit 60 can analyze the distribution characteristics of the fluorescent component E.

[3. Example of light source]
7 to 9 show examples of illumination light satisfying the above-described conditions. 7A to 9A show the wavelength characteristics of the illumination light in the band of 420 nm to 700 nm, B shows the frequency of the spectral distribution of the illumination light, and C shows all the k1 bases superimposed. The horizontal axis of A in FIGS. 7 to 9 indicates the wavelength and the vertical axis indicates the level, and the horizontal axis of B in FIGS. 7 to 9 indicates the frequency of the spectral distribution by wavelength (nm). When the level (vertical axis) is high, it indicates that the wavelength component (high-frequency component) is largely contained in the illumination light. The horizontal axis of C in FIGS. 7 to 9 indicates the frequency component of the product of the basis of the spectral reflectance and the illumination light.

FIG. 7A shows an example in which, as wavelength characteristics of illumination light, ideal illumination light which is dispersed in many very sharp bands is calculated by simulation. In the case of the wavelength characteristic as shown in FIG. 7B, a long wavelength (i.e. high frequency components) than 10nm in the high-frequency component f ₁ is seen to contain many. Also, when looking at the distribution of the product of the base and the illumination light shown in FIG. 7C, it can be seen that there are many components with a long wavelength exceeding 10 nm in each base.
By preparing a light source that outputs illumination light having ideal characteristics as shown in FIG. 7, it is possible to favorably estimate the fluorescence. However, at present, it is difficult to obtain a light source having such ideal characteristics.

FIG. 8A shows an example of illumination light using a programmable light source as a light source, the intensity of which changes at a relatively short constant wavelength interval. In the case of this wavelength characteristic as shown in FIG. 8B, the high-frequency component f ₂ are concentrated in a specific location (about 20 nm). Also, when looking at the product of the base and the illumination light shown in FIG. 7C, it can be seen that there are many components of long wavelengths exceeding 10 nm in each base.
By preparing a programmable light source and outputting illumination light as shown in FIG. 8, it is possible to satisfactorily estimate the fluorescence. However, such programmable light sources are expensive and are not readily available light sources.

FIG. 9A shows an example of illumination light in which a HID lamp is used as a light source, and sharp and strong portions, which are high-frequency components, are dispersed in a plurality of bands. In the case of this wavelength characteristic as shown in FIG. 9B, high frequency components f _3, are present relatively large in a wavelength range longer than 10 nm. It can also be seen from the basis expression shown in FIG. 9C that there are many components with long wavelengths exceeding 10 nm in each basis.
By preparing an HID lamp and outputting illumination light as shown in FIG. 9, it is possible to satisfactorily estimate the fluorescence. Since HID lamps are widely used light sources, they can be obtained relatively easily.
Therefore, by using an HID lamp as a light source, it is possible to relatively easily obtain an image processing apparatus capable of favorably estimating a fluorescent component.

FIG. 10 shows, as a reference (comparative example), characteristics of a light source that cannot be used in the present invention.
FIG. 10 shows the characteristics of a fluorescent lamp (left end of FIG. 10), the characteristics of a color LED (Light Emitting Diode) lamp (center of FIG. 10), and the characteristics of a white LED (right side of FIG. 10). 7A to 9, FIG. 10A shows the wavelength characteristics of the illumination light in the band of 420 nm to 700 nm, FIG. 10B shows the high-frequency components of the illumination light, and FIG. 10C shows all n bases. And the product of the distribution of the illumination light.

For fluorescent lamps, as shown as a high frequency component f ₄ in FIG. 10B, fewer long wavelength components than 10 nm, in the product of the illumination and the base shown in FIG. 10C, less longer components than 10 nm, to estimate the fluorescence component Not suitable for
For the case of a color LED is also shown as a high frequency component _{f 5} in FIG. 10B, fewer long wavelength components than 10 nm, in the product of the illumination and the base shown in FIG. 10C, less longer components than 10 nm, the fluorescence component Not suitable for estimation.
For the case of the white LED also, as shown as a high frequency component _{f 6} in FIG. 10C, almost no longer wavelength components than 10 nm, in the product of the illumination and the base shown in FIG. 10C, almost no longer components than 10 nm, fluorescence Not suitable for estimating components.
When the light sources shown in FIGS. 10A and 10B are used, an appropriate estimation result cannot be obtained even if an attempt is made to calculate the fluorescence component by the calculation based on the determinant described above.

[4. Example of estimating fluorescence]
FIG. 11 shows an example of estimating the fluorescence component by the image processing apparatus of this example.
FIG. 11A shows an example of green reflected light (left) and green fluorescent component (right) of a substance.
Three reflection light R _{G1, R} G2 shown on the left side of FIG. _11A, R _G3 represents accurate reflection component, respectively, the reflected light component when using a programmable light source, the reflected light component when using HID lamps . Further, three fluorescent components _{E G1,} E G2 shown on the right side in FIG. _{11A, E G3,} respectively exact fluorescent components, a fluorescent component when using a programmable light source, and a fluorescence component when using HID lamps Show. The accurate reflected light component R _G1 and fluorescent component E _G1 are measured using a conventionally known high-precision fluorescence measuring device. The three fluorescent components _EG1 , _EG2 , _EG3 do not differ greatly. Therefore, according to the image processing apparatus of the present example, it can be understood that the fluorescence component can be satisfactorily estimated without using an expensive fluorescence measurement apparatus having a complicated configuration as in the related art.

FIG. 11B shows an example of a red reflected light (left) and a red fluorescent component (right) of a substance.
The three reflected lights R _R1 , R _R2 , and R _R3 shown on the left side of FIG. 11B indicate an accurate reflected light component, a reflected light component when using a programmable light source, and a reflected light component when using an HID lamp, respectively. . The three fluorescent components E _R1 , E _R2 , and E _R3 shown on the right side of FIG. 11B are an accurate fluorescent component, a fluorescent component when a programmable light source is used, and a fluorescent component when a HID lamp is used. Show.
As for the red component, there is no great difference between the three fluorescent components E _R1 , E _R2 , and E _R3 . Therefore, according to the image processing apparatus of the present example, it can be seen that the fluorescence component can be satisfactorily estimated. When this technique is applied to a spectral image, a reflection component and a fluorescence component can be estimated for each pixel of the spectral image.

  As described above, according to the image processing apparatus of the present example, it is possible to accurately estimate the fluorescence emitted from the object to be measured from one frame image captured by the camera 40 that obtains a spectral image. In addition, since the fluorescence can be estimated by the calculation of the determinant expressing the spectral reflectance in the basis, the fluorescence can be estimated by a relatively simple calculation, and the fluorescence can be estimated in real time while photographing with the camera 40. Therefore, it is possible to estimate (measure) the fluorescence in real time while photographing a moving object such as a living thing, which has an effect that the fluorescence measurement of various objects can be performed very easily. Moreover, a general light source such as an HID lamp can be used as the light source 20, and a system capable of measuring (estimating) fluorescence can be assembled at a low cost.

[5. Modification]
As a light source applicable to the image processing apparatus of this example, among the light sources already on the market, an HID lamp exists as described above, but any light source that satisfies the requirements for the illumination light described so far can be used. , Other types of light sources may be used. For example, even a light source using an organic EL (Electro Luminescence) panel or a light source using a light emitting diode (LED) may output illumination light in which sharp high-frequency components are present at a plurality of locations. If it can be created, it can be applied.
The above-described wavelength characteristics of the HID lamp are merely examples, and other wavelength characteristics may be used.

  Further, in the image processing apparatus shown in FIG. 1, the image processing unit and the image analysis unit may be configured by a dedicated circuit. For example, the image processing unit and the image analysis unit described in the flowchart of FIG. An image processing apparatus may be realized by creating a program (software) and mounting the program on a computer device. The program in this case may be recorded on a recording medium such as an optical disk or a semiconductor memory, for example.

  Further, in the system configuration shown in FIG. 1, the system includes a camera that captures an image and an image processing device that analyzes the captured image in real time. For example, an image captured by a spectroscopic camera is stored in a memory or the like. The recorded image may be analyzed at a later date using an image processing device (or a computer device in which a program functioning as an image processing device is installed) to detect (estimate) the fluorescence. Alternatively, the data of the captured spectral image is sent to an image processing device (or a computer device) that performs an arithmetic process for estimating the fluorescence prepared at another location using a communication line, and the image processing device transmits the fluorescence data. An estimation result may be obtained.

  Also, an example of the determinant for estimating the fluorescence described above is shown, and the process of estimating the fluorescence using a high-frequency component (a component higher than the fluorescence) included in the light source is performed based on the same principle. Other calculation methods may be applied as long as the calculation is performed.

  DESCRIPTION OF SYMBOLS 10 ... Control part, 20 ... Light source, 30 ... Image processing part, 31 ... Principal component analysis part, 32 ... Fluorescence estimation part, 40 ... Camera, 50 ... Image display part, 60 ... Characteristic analysis part, 90 ... Subject (measurement object) Object

Claims (7)

A light source that emits illumination light having characteristics in which a frequency component higher than the frequency component of the fluorescence is included in a plurality of wavelength bands,
An object to be measured irradiated with illumination light by the light source, a spectral camera that obtains a spectral component by spectrally photographing a plurality of wavelength bands,
From the spectral components for each of a plurality of wavelength bands of one frame of image data obtained by the spectral camera, a principal component analysis unit that approximates the spectral reflectance of the measured object with a plurality of bases,
An image processing apparatus comprising: a fluorescence estimating unit configured to estimate fluorescence of the object to be measured by calculation using characteristics of the illumination light and a plurality of bases obtained by the principal component analysis unit. The image processing apparatus according to claim 1, wherein the high frequency component included in the illumination light includes a frequency component having a wavelength longer than at least 10 nm. The image processing device according to claim 1, wherein the light source that emits illumination light having characteristics in which a frequency component higher than the frequency component of the fluorescence is included in a plurality of wavelength bands is a high-intensity discharge lamp. A photographing step of photographing an object under measurement irradiated with illumination light having characteristics in which a frequency component higher than the frequency component of the fluorescent component is included in a plurality of wavelength bands, by spectrally capturing the object in a plurality of wavelength bands with a spectral camera,
A principal component analysis step of approximating the spectral reflectance of the object to be measured by a plurality of bases from spectral components for each of a plurality of wavelength bands of one frame of image data obtained in the imaging step;
An image processing method, comprising: a fluorescence estimation step of estimating fluorescence of the object to be measured by calculation using characteristics of the illumination light and a plurality of bases obtained in the principal component analysis step. The image processing method according to claim 4, wherein the high-frequency component included in the illumination light includes a frequency component having a wavelength longer than at least 10 nm. As a light source for emitting the illumination light, an image processing method according to claim 4 or 5 using high intensity discharge lamps that emit light of the illumination light emission intensity that contains high-frequency components of the heterogeneous. A photographing procedure of spectrally photographing an object under measurement irradiated with illumination light having characteristics in which a frequency component higher than the frequency component of the fluorescent component is included in a plurality of wavelength bands,
A principal component analysis step of approximating the spectral reflectance of the object to be measured by a plurality of bases from spectral components for each of a plurality of wavelength bands of one frame of image data obtained by the imaging procedure;
In a calculation using the characteristics of the illumination light and a plurality of bases obtained in the principal component analysis procedure, a fluorescence estimation procedure for estimating the fluorescence of the measured object,
A program that is implemented on a computer and executed.

Images