JPWO2019159565A1 - Information management devices, terminals, information management systems and programs - Google Patents

Abstract

An information management device that manages information on an object, which is classified into one or more categories and stores basic data related to the spectrum of the object, and information on the spectrum measurement data and the category of the object. Provided is an information management device including a data acquisition unit to be acquired and a base update unit that updates the base data stored in the base storage unit based on the data acquired by the data acquisition unit.

Description

The present invention relates to information management devices, terminals, information management systems and programs.

Conventionally, a technique for estimating the spectrum of a subject's spectral spectrum using a basis function has been known (see, for example, Patent Document 1).
Patent Document 1 Japanese Unexamined Patent Publication No. 2010-12280

It is preferable to use more appropriate basis functions in order to improve the accuracy of spectrum estimation.

In order to solve the above problems, in the first aspect of the present invention, an information management device for managing information on an object is provided. The information management device is classified into one or more categories, and has a basic storage unit that stores basic data related to the spectrum of the object, a data acquisition unit that acquires spectrum measurement data of the object and information related to the category, and data acquisition. It is provided with a base update unit that updates the base data stored in the base storage unit based on the data acquired by the unit.

In the second aspect of the present invention, a terminal that can be connected to an information management device is provided. The terminal generates a spectral image of the object based on the image acquisition unit that acquires the captured image of the object, the terminal side data acquisition unit that acquires the base data from the information management device, and the captured image and the base data. An image generation unit is provided.

In the third aspect of the present invention, a terminal that can be connected to an information management device is provided. The terminal acquires information about the spectrum of the object from the image acquisition unit that acquires the captured image of the object, the data transmission unit on the terminal side that transmits the captured image to the information management device, and the information management device. It is equipped with a terminal-side data acquisition unit.

A fourth aspect of the present invention provides an information management system including an information management device of any aspect and a terminal of any aspect.

In the fifth aspect of the present invention, a program for operating a computer as an information management device or a terminal is provided.

The outline of the above invention does not list all the necessary features of the present invention. Sub-combinations of these feature groups can also be inventions.

It is a block diagram which shows an example of the information management system 250 which concerns on one Embodiment of this invention. It is a block diagram which shows an example of an image generation apparatus 207. It is a figure which shows an example of the captured image 12. It is a figure which shows an example of the basis function 22. It is a figure which shows an example of the spectrum coefficient 42. It is a figure which shows an example of the characteristic of a virtual spectroscopic filter 32. It is a figure which shows an example of the spectroscopic function 36. It is a flowchart which shows the operation example of the image generation apparatus 207. It is a figure which shows another example of image generation apparatus 207. It is a figure which shows an example of the spectrum image 806. It is a figure explaining the ray space data which shows the ray space of illumination to an object 300 when spectrum measurement data was measured. It is a figure which shows another example of the information management apparatus 100. An example of a computer 1200 in which a plurality of aspects of the present invention can be embodied in whole or in part is shown.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.

FIG. 1 is a block diagram showing an example of an information management system 250 according to an embodiment of the present invention. The information management system 250 includes a terminal 200 and an information management device 100. For example, the terminal 200 is a computer managed by a user or the like. For example, the information management device 100 is a server connected to one or more terminals 200 via a network 252. The network 252 may be the Internet or a local network.

The information management device 100 manages information on the object. The object is, for example, an object whose image is captured by a user or the like. The information of the object includes an image of the object, a spectrum of the object, and the like. The object is a fruit such as an apple or a mandarin orange, and the information of the object is an image and a spectrum obtained by capturing the image of the object. The object may be a crop other than fruit, meat, processed food, or a living body or its tissue.

The information management device 100 includes a management device side data transmission unit 102, a management device side data acquisition unit 104, a base update unit 106, and a base storage unit 108. The management device side data transmission unit 102 transmits data to each terminal 200 via the network 252. The management device side data acquisition unit 104 acquires data from each terminal 200 via the network 252.

The basal storage unit 108 stores basal data regarding the spectrum of the object. The basis data includes a plurality of basis functions described later. The basal storage unit 108 classifies the basal data into one or a plurality of categories and stores them. The basis functions included in the basis data are calculated by acquiring a plurality of spectral data for each object 300 belonging to the category (see FIG. 3 etc.) and statistically processing these data by principal component analysis or the like. .. The category corresponds to the category of the object. The spectrum can be estimated more accurately by estimating the spectrum of the object using the basis data of the corresponding category. As the object category, fruit types such as apples and mandarins can be used. Further, the category of the object 300 may be divided into a plurality of items (harvest time, cultivation method, component, etc.) from different viewpoints, or a hierarchical structure (variety, production area, producer, etc.). The basis acquisition unit 20, which will be described later, may acquire basis data according to the category specified by the user.

The data acquisition unit 104 on the management device side acquires the spectrum measurement data of the object and the information regarding the category of the object. The spectral measurement data is, for example, spectral data such as spectral reflectance obtained by measuring the surface of an object. The spectrum measurement data may include spectrum data including the band of the basis function to be calculated. The management device side data acquisition unit 104 acquires information from a plurality of terminals 200.

The base update unit 106 updates the base data stored in the base storage unit 108 based on the spectrum measurement data and the category information acquired by the management device side data acquisition unit 104. For example, the basis update unit 106 stores the spectrum measurement data used for calculating the basis data currently stored by the basis storage unit 108. The basis update unit 106 may add the newly acquired spectrum measurement data to the stored spectrum measurement data to generate new basis data. The basis update unit 106 generates a basis function that can minimize the error between the linear sum of the basis functions included in the basis data and the respective spectrum measurement data. The basis update unit 106 may decompose the collected spectrum measurement data into a plurality of basis functions by a method such as principal component analysis. The basis update unit 106 may change the number of basis functions included in the basis data.

The basis update unit 106 may update the basis data every time the spectrum measurement data is acquired, or may accumulate the spectrum measurement data acquired during a certain period and update the basis data every time. .. When adding new spectrum measurement data, the basis update unit 106 may discard a part of the existing spectrum measurement data.

The basis update unit 106 may classify the spectrum measurement data according to the time when the data was measured. For example, the basis update unit 106 classifies the spectrum measurement data on a monthly basis. In this case, the basal storage unit 108 may store the basal data at each time when the spectrum measurement data is measured. This makes it possible to manage appropriate basal data for an object whose state and color change depending on the time, such as agricultural products.

The basis update unit 106 may classify the spectrum measurement data by parameters other than the timing. For example, it can be classified by parameters such as location (latitude, longitude, etc.), temperature, humidity, etc. where the spectrum measurement data was measured. These environmental parameters are preferably attached to the spectral measurement data. The basis update unit 106 may classify the spectrum measurement data according to the ray space data described later.

The terminal 200-1 includes a terminal-side data transmission unit 202, a terminal-side data acquisition unit 204, and a spectrum acquisition unit 206. The terminal-side data acquisition unit 204 acquires data from the information management device 100 via the network 252. The terminal-side data transmission unit 202 transmits data to the information management device 100 via the network 252.

The spectrum acquisition unit 206 acquires spectrum measurement data of the object. The spectrum measurement data can be measured by, for example, a spectroscopic measuring instrument or the like. The spectroscopic measuring instrument may be provided on an inspection line for agricultural products and the like.

The terminal-side data transmission unit 202 transmits the spectrum measurement data acquired by the spectrum acquisition unit 206 to the information management device 100 with category information of the object. The terminal-side data transmission unit 202 may attach the light ray space data indicating the light ray space of illumination to the object at the time of measuring the spectrum measurement data to the spectrum measurement data and transmit it.

In this way, by aggregating the spectrum measurement data and the category information sent from the plurality of terminals 200 via the network 252 into the information management device 100, a more accurate basis function can be generated. The management device side data transmission unit 102 may transmit information based on the base data stored in the base storage unit 108 to the terminal 200. As a result, the terminal 200 can accurately estimate the spectrum of the object by using a highly accurate basis function.

In the example of FIG. 1, the information management device 100 transmits to the terminal 200-2 to the basis function. The terminal 200-2 includes an image generation device 207. The same terminal 200 may be provided with the spectrum acquisition unit 206 and the image generation device 207.

The terminal-side data transmission unit 202 of the terminal 200-2 specifies the category to which the object belongs and requests the information management device 100 for the base data. The terminal-side data transmission unit 202 may further specify parameters such as the time associated with the base data. In response to the request, the management device side data transmission unit 102 extracts the basal data of the designated category from the basal data stored in the basal storage unit 108 and transmits it to the terminal 200-2. The image generation device 207 generates a spectroscopic image of the object based on the basis data acquired by the terminal side data acquisition unit 204.

FIG. 2 is a block diagram showing an example of the image generator 207. The structure of the image generator 207 is not limited to this example. The image generation device 207 of this example includes an image acquisition unit 10, a basis acquisition unit 20, a function setting unit 30, a spectrum coefficient calculation unit 40, and an image generation unit 50.

The image acquisition unit 10 acquires captured image data obtained by capturing at least a part of an object. The captured image may be a two-dimensional image. The image pickup device that has captured the captured image data has sensitivity in a predetermined wavelength band. The image pickup apparatus may be sensitive to light having a wavelength in the visible band, and may be sensitive to light having a wavelength outside the visible band. The image pickup apparatus may acquire a still image of an object, or may acquire a moving image.

The image generation device 207 generates a spectroscopic image when an arbitrary spectral characteristic is applied to an object based on the captured image data acquired from the image pickup device. By adjusting the spectral characteristics, for example, it is possible to generate a spectral image when the characteristics of the illumination light irradiating the object or the characteristics of the spectral filter of the imaging device are changed.

The captured image data includes the value of each pixel at a plurality of wavelengths. As an example, the captured image data includes the value of each pixel at the wavelength corresponding to the three colors of red, blue, and green. As another example, the captured image data may include the value of each pixel at three wavelengths in the infrared band. In the present specification, the spectrum shows the distribution of the intensity for each wavelength of light, and the wavelength resolution thereof is assumed to be higher than a predetermined threshold value. For example, when the wavelength resolution (wavelength interval between images) of multiple captured image data captured at different wavelengths (or wavelength bands) is set as a threshold, the spectrum is larger than the wavelength resolution (multiple wavelength intervals) of the captured image data. It shall have high resolution. In this example, the captured image data contains values at three wavelengths, so the spectrum contains values at at least four or more wavelengths. The wavelength resolution of the spectrum may be 10 nm or less, for example 1 nm.

The basis acquisition unit 20 acquires basis data regarding the spectrum of the object. In this example, the base data received by the terminal side data acquisition unit 204 from the information management device 100 is input to the base acquisition unit 20. Basis data includes data of multiple basis functions. An arbitrary spectrum is expressed by multiplying each basis function by a predetermined weighting coefficient and adding them. The greater the number of basis functions contained in the basis data, the smaller the error for any spectrum.

The basis acquisition unit 20 acquires basis data according to the category of the object. The category of the object corresponding to the captured image data may be determined from the category information attached to the captured image data acquired from the imaging device, and the base acquisition unit 20 may determine based on the captured image data. For example, the basal acquisition unit 20 can determine the category of the object by comparing the image of the object included in the captured image data with the preset template image.

The function setting unit 30 sets a spectroscopic function for generating a spectroscopic image through a virtual spectroscopic filter having arbitrary spectral characteristics by using the basal data acquired from the basal acquisition unit 20. The spectral characteristics may be set by the user of the image generator 207 or the like. The function setting unit 30 may calculate the spectral function by multiplying each basis function indicated by the basis data by the spectral characteristics.

The spectrum coefficient calculation unit 40 calculates from the captured image a spectral coefficient indicating a weighting coefficient of each basis function included in the basis data for each pixel indicating an object in the captured image. As described above, an arbitrary spectrum is represented by the linear sum of a plurality of basis functions. The spectral coefficient is a combination of weighting coefficients of a plurality of basis functions that can best approximate the value of each wavelength (or color) contained in the captured image data. In this example, the spectral coefficient is determined for each pixel of the captured image. In another example, the spectral coefficient may be determined for each pixel block containing a plurality of pixels.

The image generation unit 50 generates a spectral image of the object 300 based on the spectral function set by the function setting unit 30 and the spectral coefficient calculated by the spectral coefficient calculation unit 40. The spectroscopic image is an image corresponding to the spectroscopic characteristics set by the user or the like. With such a configuration, a spectroscopic image corresponding to an arbitrary spectral characteristic can be generated with a small amount of calculation.

FIG. 3 is a diagram showing an example of the captured image 12. The image acquisition unit 10 of this example acquires three captured images 12-1, 12-2, and 12-3 corresponding to three wavelengths (or colors). In this example, the captured image 12 and the wavelength (or color) have a one-to-one correspondence. Each captured image 12 has a plurality of pixels arranged in two dimensions. The captured image data showing the captured image 12 is data indicating the intensity of light of each wavelength (or color) received by the imaging device for each pixel.

FIG. 4 is a diagram showing an example of a basis function 22 included in the basis data. In this example, three basis functions 22-1, 22-2, and 22-3 are shown, but the number of basis functions 22 included in the basis data is not limited to three. The basis data may include more basis functions 22. In the examples of FIGS. 3 and 4, the numbers of the basis functions 22 and the captured images 12 are the same, but the numbers of the basis functions 22 and the captured images 12 do not have to be the same.

As shown in FIG. 4, the shapes of the basis functions 22 are different. For example, each basis function 22 has a different wavelength at which a maximum value or a minimum value is obtained. Various spectra can be expressed by multiplying such a basis function 22 by an appropriate weighting coefficient and adding them. However, depending on the shape of the basis function 22, it may or may not be accurately expressed. For example, in the example of FIG. 4, since the shapes of the basis functions 22 are similar in the long wavelength band 23, the reproduction accuracy may not be high for the spectrum that changes in the long wavelength. As described above, the basis acquisition unit 20 can acquire the basis data according to the category of the object 300. For example, a plurality of basis functions 22 are prepared for the category of "apples produced in Aomori Prefecture", and from there, basis data including at least the same number of basis functions 22 as the number of wavelengths captured in the captured image 12 is acquired. To do. Thereby, the accuracy of the spectrum estimation of the object 300 can be improved.

FIG. 5 is a diagram showing an example of the spectral coefficient 42. The spectrum coefficient calculation unit 40 calculates the spectrum coefficient 42, which has a one-to-one correspondence with the basis function 22. In this example, each spectral coefficient 42 has bits 44 that correspond one-to-one with the pixels of the captured image 12. In another example, each spectral coefficient 42 may have bits 44 for each pixel block containing the plurality of pixels of the captured image 12. FIG. 5 shows a spectral coefficient image in which each bit 44 is arranged two-dimensionally. The region 48 in FIG. 5 is an enlarged region of a part of the region 46 in the spectral coefficient image.

The value of each bit 44 in each spectral coefficient 42 is a weighting coefficient of the basis function 22 corresponding to the spectral coefficient 42 for the pixel corresponding to the bit 44. In FIG. 5, the value of each bit 44 is shown by the shade of hatching.

The spectral coefficient calculation unit 40 may store the calculated combination of the spectral coefficients 42. The spectrum coefficient calculation unit 40 may store the spectrum coefficient 42 and the basis data in association with each other. From the spectrum coefficient 42 and the basal data, the spectrum in each pixel of the object 300 can be estimated.

FIG. 6 is a diagram showing an example of the characteristics of the virtual spectroscopic filter 32. As described above, the virtual spectroscopic filter 32 can be set to any characteristic by the user or the like. Since the spectral characteristics of a normal spectroscopic filter are defined, it is necessary to replace the spectroscopic filter in order to change the spectral characteristics. However, since the virtual spectroscopic filter 32 of this example is a virtual spectroscopic filter used for calculation. , The spectral characteristics can be set freely. A plurality of virtual spectroscopic filters 32 can be set. For example, the function setting unit 30 can set a virtual spectroscopic filter 32 corresponding to each wavelength (or color) in each pixel of the spectroscopic image. More specifically, when each pixel of the spectroscopic image is represented by a red, blue, and green pixel value, the function setting unit 30 has a virtual corresponding to each wavelength (or color) of red, blue, and green. The spectroscopic filter 32 may be set.

The function setting unit 30 may specify the characteristics of the virtual spectroscopic filter 32 itself from the user or the like. The function setting unit 30 may specify the type of the virtual imaging device and the like from the user and the like. The function setting unit 30 may use the characteristics of the virtual spectroscopic filter 32 corresponding to the information specified by the user or the like. The characteristics of a plurality of types of virtual spectroscopic filters 32 may be registered in advance in the function setting unit 30.

FIG. 7 is a diagram showing an example of the spectroscopic function 36. The function setting unit 30 sets the spectral function 36 by multiplying each basis function 22 included in the basis data by the spectral characteristics of the virtual spectral filter 32. That is, the function setting unit 30 multiplies the value of the basis function 22 and the value of the spectral characteristics of the virtual spectroscopic filter 32 at each wavelength. The function setting unit 30 sets a spectroscopic function 36 for each virtual spectroscopic filter 32 for each basis function 22.

In the example of FIG. 7, the spectroscopic function 36-1-1 corresponding to the basis function 22-1 and the virtual spectroscopic filter 32-1 is shown. The function setting unit 30 also calculates the spectroscopic functions 36-1-2 and 36-1-3 corresponding to the basis functions 22-1 and the other virtual spectroscopic filters 32-2 and 32-3. Further, the function setting unit 30 similarly sets the spectroscopic function 36 multiplied by each virtual spectroscopic filter 32 for the other basis functions 22.

The function setting unit 30 may store the spectroscopic function 36. The function setting unit 30 may store the spectroscopic function 36, the information for identifying the object 300, and the information for identifying the virtual spectroscopic filter 32 in association with each other.

The image generation unit 50 generates a spectral image of the object 300 based on the spectroscopic function 36 and the spectral coefficient 42. The image generation unit 50 multiplies the combination of the spectroscopic function 36 and the spectral coefficient 42 corresponding to the same basis function 22, and then adds the multiplication results.

The image generation unit 50 of this example multiplies the spectral coefficient 42-1 and the plurality of spectral functions 36, the spectral coefficient 42-2 and the plurality of spectral functions 36, and the spectral coefficient 42-3 and the plurality of spectral functions 36, respectively. The image generation unit 50 multiplies the spectroscopic function 36 by the spectral coefficient 42 for each bit 44 of the spectral coefficient 42. The image generation unit 50 adds the multiplication result of the same spectroscopic function 36 and each spectral coefficient 42 for the same bit 44. As a result, the pixel value of each wavelength (or color) of each bit 44 can be calculated. The image generation unit 50 uses the pixel value of each bit 44 obtained by addition as the pixel value of each pixel of the spectroscopic image.

FIG. 8 is a flowchart showing an operation example of the image generation device 207. The basis acquisition unit 20 acquires basis data including u basis functions 22 (S702). u is an integer of 2 or more. u is preferably an integer equal to or greater than the number of wavelengths at which the captured image 12 is captured. The function setting unit 30 sets the spectroscopic functions 36-n-s for each of the basis functions 22-n and the virtual spectroscopic filter 32-s (S704). n is an integer of 1 or more and u or less, and s is an integer of 1 or more.

The image acquisition unit 10 acquires the captured image 12 (S706). The spectrum coefficient calculation unit 40 calculates the spectrum coefficient 42 (S708). The processing of S702 to S708 is not limited to the order shown in FIG. For example, the processing of S706 may precede S702.

ただし、ｓ（λ）は対象物３００の分光反射率のスペクトル、ｃ_ｍ（λ）は撮像装置の感度特性のスペクトル、ｐ（λ）は照明光のスペクトルである。なおｃ_ｍ（λ）およびｐ（λ）は既知とする。ｍは、撮像画像１２の数に対応する。例えば、赤、青、緑のように３つの波長成分についてそれぞれ撮像画像１２を取得した場合、ｍは１、２、３のように３つの整数で表される。ｃ_ｍ（λ）は、撮像装置における各波長成分の感度特性（例えばフィルタの特性）に対応する。As an example, the spectral coefficient 42 can be calculated as follows. Pixel value I _m of the pixels in the captured image 12 can be modeled by the following equation.

However, s (λ) is the spectrum of the spectral reflectance of the object 300, _cm (λ) is the spectrum of the sensitivity characteristics of the image pickup device, and p (λ) is the spectrum of the illumination light. Note that _cm (λ) and p (λ) are known. m corresponds to the number of captured images 12. For example, when the captured image 12 is acquired for each of the three wavelength components such as red, blue, and green, m is represented by three integers such as 1, 2, and 3. _cm (λ) corresponds to the sensitivity characteristic (for example, the characteristic of the filter) of each wavelength component in the image pickup apparatus.

ただし、σ（ｎ）はスペクトル係数であり、ｂ_ｎ（λ）はスペクトル係数４２−ｎに対応する基底関数であり、ｕは用いる基底関数の数である。それぞれの基底関数ｂ_ｎ（λ）は既知とする。Further, the spectrum s (λ) of the object 300 is represented by the following equation.

However, σ (n) is a spectrum coefficient, b _n (λ) is a basis function corresponding to the spectrum coefficient 42−n, and u is the number of basis functions to be used. It is assumed that each basis function b _n (λ) is known.

数３により、それぞれの画素値Ｉ_ｍを、既知成分ｂ_ｎ（λ）ｃ_ｍ（λ）ｐ（λ）と、未知のスペクトル係数σ（ｎ）で表すことができる。Substituting the equation 2 into the equation 1 gives the following equation.

The number 3, each of the pixel values _{I m,} the known components _{b n (λ) c m (} λ) p (λ), can be expressed by the unknown spectral coefficients σ (n).

Next, the known component b _n (λ) c _m (λ) p (λ) in Equation 3 is replaced with a predetermined coefficient. More specifically, for each combination of m and n, the known components b _n (λ) _cm (λ) p (λ) are replaced with predetermined coefficients. As an example, consider the case where the number of basis functions is 3 and the number of captured images 12 is 3. That is, u = 3, m = 1, 2, and 3.

ｍ＝２、３の場合も、既知成分を係数Ｂ_ｎ、Ｃ_ｎを用いて下式のように置き換える。

In the case of m = 1, the known component is replaced with the coefficient _An as shown in the following equation.

Also in the case of m = 2 and 3, the known components are replaced by the coefficients B _n and C _n as shown in the following equation.

画素値Ｉ_ｍは撮像画像１２から既知であるので、数６の連立方程式を解くことで、スペクトル係数σ（１）、σ（２）、σ（３）を算出できる。なお、ノイズの影響や、スペクトルを推定するうえでの多項式の上位項のみを用いることの影響で、算出したスペクトル係数を用いた対象物のスペクトルが負の値を含むことがある。そこで、対象物のスペクトルが必ず正となるように数６の連立方程式を解いてもよい。具体的には、制約つきの最小二乗法を用いることが考えられる。The following equation can be obtained from the numbers 3, 4, and 5.

Since the pixel value _Im is known from the captured image 12, the spectral coefficients σ (1), σ (2), and σ (3) can be calculated by solving the simultaneous equations of Equation 6. In addition, the spectrum of the object using the calculated spectrum coefficient may include a negative value due to the influence of noise and the influence of using only the upper term of the polynomial in estimating the spectrum. Therefore, the simultaneous equations of Equation 6 may be solved so that the spectrum of the object is always positive. Specifically, it is conceivable to use the constrained least squares method.

ただし、ｒ_ｔ（ｎ）は分光関数３６−ｎ−ｔである。The image generation unit 50 generates a spectroscopic image (S710). The pixel value R _{t for} each wavelength component in each pixel calculated by the image generation unit 50 is shown below. t is an integer of 1 or more and s or less, and corresponds to each wavelength component.

_{However, r} t (n) is the spectral function 36-n-t.

That, sigma a (n) and r _{t (n)} by adding the result of multiplying u times, it calculates the pixel values R _t for each pixel. Therefore, the pixel value R _t can be calculated with a small amount of calculation.

FIG. 9 is a diagram showing another example of the image generator 207. The image generator 207 of this example does not calculate the spectroscopic function 36. The image generation device 207 of this example includes an image acquisition unit 10, a spectrum coefficient calculation unit 40, a spectrum image generation unit 802, and an image generation unit 804. The image acquisition unit 10 and the spectrum coefficient calculation unit 40 are the same as the image acquisition unit 10 and the spectrum coefficient calculation unit 40 shown in FIG.

The spectrum image generation unit 802 multiplies the spectrum coefficient 42 by the corresponding basis function 22. As a result, a spectral image having a predetermined wavelength resolution is generated. The image generation unit 804 generates a spectroscopic image by multiplying the spectral image by the spectral characteristics of the virtual spectroscopic filter 32.

FIG. 10 is a diagram showing an example of the spectrum image 806. The spectrum image generation unit 802 generates a spectrum image 806 corresponding to each wavelength. For example, when the band of interest is 600 nm to 1000 nm and the wavelength resolution is 1 nm, the spectrum image generation unit 802 generates about 400 spectrum images 806.

Each pixel of each spectral image 806 indicates a pixel value at that wavelength. Each spectral image 806 can be calculated by multiplying the corresponding spectral coefficient 42 and the basis function 22 and adding the respective multiplication results for each wavelength.

ただし、ｆ（λ）は波長λにおけるスペクトル画像８０６の画素値であり、ｖ_ｔ（λ）は分光特性ｔの波長λにおける値である。つまり、本例の画像生成装置１００においては、各画素値Ｒ_ｔを算出するのに、ｆ（λ）とｖ_ｔ（λ）を４００回程度（波長分解能が１ｎｍで、注目帯域が６００ｎｍから１０００ｎｍの場合）乗算した結果を加算する。The image generation unit 804 generates a spectroscopic image based on the spectral characteristics of the set virtual spectroscopic filter 32 and the spectral image 806. Specifically, the image generation unit 804 calculates each pixel value R _t of the spectroscopic image based on the following equation.

However, f (λ) is the pixel value of the spectral image 806 at the wavelength λ, and v _t (λ) is the value of the spectral characteristic t at the wavelength λ. That is, in the image generation device 100 of this example, f (λ) and v _t (λ) are performed about 400 times (wavelength resolution is 1 nm, and the attention band is 600 nm to 1000 nm) in order to calculate each pixel value R _t. In the case of) Add the result of multiplication.

As described above, according to the image generator 207 shown in FIG. 2, it is only necessary to multiply u times (the number of basis functions 22) and add once to calculate each pixel value R _t . Therefore, a spectroscopic image for an arbitrary virtual spectroscopic filter 32 can be generated with a very small amount of calculation. Further, by storing the spectral coefficient 42 and acquiring the basis data from the information management device 100, a spectroscopic image having arbitrary virtual spectral characteristics can be easily created.

Further, since the spectroscopic image of the object 300 is generated from the captured image 12 of u times (the number of basis functions) or less, a short imaging time is required. Therefore, the time for illuminating the object 300 can be shortened, and the damage to the object 300 can be reduced. In addition, a spectroscopic image of the moving image can be easily generated from the captured image 12 of the moving image.

In addition, a spectroscopic image can be generated with the wavelength resolution of the basis data. Therefore, a spectroscopic image having a high wavelength resolution can be easily generated. Further, even if the wavelength resolution is increased, the amount of data used for the calculation does not increase so much.

FIG. 11 is a diagram for explaining the ray space data indicating the ray space of illumination to the object 300 when the spectrum measurement data is measured. In the example of FIG. 11, the illumination unit 410 that irradiates the object 300 with illumination light is shown. The ray space data is data indicating the position, type and number of light sources 412 that irradiate the object 300 with illumination light, and at least one of the intensity, spectrum, irradiation angle and polarization of the irradiation light.

When the illumination conditions shown in the ray space data change, the spectrum measurement data may change. For example, a spectroscopic measuring device measures surface reflected light, surface scattered light, and internally scattered light in the measurement region of the object 300. The internally scattered light is a component emitted by scattering the incident light inside the object 300. When the illumination conditions such as the incident angle of light change, the surface reflected light, the surface scattered light, and the internally scattered light incident on the spectroscopic measuring device may change. Therefore, if the basis function is updated from the spectrum measurement data without considering the illumination conditions, the accuracy of the spectrum estimation may not be improved.

The basis update unit 106 may calculate the basis function based on the spectrum measurement data and the ray space data. For example, the basis update unit 106 corrects the spectrum measurement data based on the difference between the ray space data attached to the spectrum measurement data and a predetermined reference condition, and then calculates the basis function. The effect of the difference from the reference value of each parameter of the ray space data on the spectrum measurement data may be measured in advance. A correction table or a correction function for calculating the correction amount of the spectrum measurement data according to the difference from the reference value may be set in advance in the base update unit 106 for each parameter of the ray space data. In another example, the basis update unit 106 may update the basis functions classified according to the ray space data with the corresponding spectral measurement data.

The position of the light source 412 is a three-dimensional relative position (x, y, z) with respect to the object 300. The position of the light source 412 may be relative to the measurement region of the object 300. When there are a plurality of light sources 412, the ray space data may include data indicating the positions of the respective light sources 412.

The type of light source 412 may be information about means of producing light. As an example, the type of the light source 412 is information indicating which light source the light source 412 is, such as a halogen lamp, a xenon lamp, or an LED. The illumination unit 410 may have a plurality of types of light sources.

The number of light sources 412 refers to the number of light sources 412 that irradiate the object 300 with light at the time of measuring the spectrum data. The wavelength of the light emitted by the light source 412 may be the wavelength of the principal component having the highest intensity in the light, or may be a spectrum over a predetermined wavelength range.

The irradiation angle of the light emitted by the light source 412 may include the angle θa of the optical axis 440 of the light. The optical axis 440 is the central axis of the light emitted by the light source 412. The angle θa may be an angle with respect to the surface 302 of the object 300, or may be an angle with respect to a predetermined reference plane. If the surface 302 of the object 300 has an inclination, the spectroscopic measuring device may measure the angle of inclination in the measurement region. The ray space data may include an inclination angle in the measurement region of the surface 302 of the object 300.

When the illumination unit 410 is a condensing system illumination, the irradiation angle of the light emitted by the light source 412 may include a condensing angle Φ at which the light is condensed on the surface 302. When the illumination unit 410 has a condensing lens 438, the condensing angle Φ is determined from the characteristics of the condensing lens 438. The irradiation angle of the light emitted by the light source 412 may include a divergence angle φ in which the light is diverged from the light source 412.

The polarization state of the light emitted by the light source 412 includes the presence or absence of polarization of the light, the direction of polarization, and the like. The ray space data may include at least the irradiation angle of the light, including the position, type, number of the light source 412, the spectrum of the light emitted by the light source 412, the intensity, the irradiation angle, and part or all of the polarization state. You may.

By updating the basis function based on such ray space data, the basis function can be updated in consideration of the influence of the illumination condition on the spectrum measurement data. Therefore, the basis function can be updated accurately.

FIG. 12 is a diagram showing another example of the information management device 100. The information management device 100 of this example further includes a spectrum estimation unit 208 in addition to the configuration shown in FIG. In FIG. 12, the image generation device 207 is also shown, and the structure other than the image generation device 207 of the terminal 200 is omitted.

The image generation device 207 of this example attaches the data of the captured image 12 acquired by the image acquisition unit 10 and requests the information management device 100 for the base data. The management device side data acquisition unit 104 acquires the data of the captured image 12. The spectrum estimation unit 208 estimates the spectrum of the object 300 included in the captured image 12 based on the captured image data and the basal data acquired from the basal storage unit 108. Specifically, the spectrum estimation unit 208 calculates the spectrum coefficient 42 of the object. The spectrum estimation unit 208 may perform the same or the same processing as the spectrum coefficient calculation unit 40 shown in FIG. The management device side data transmission unit 102 transmits information about the spectrum generated by the spectrum estimation unit 208 (that is, the spectrum coefficient 42) to the terminal 200 in addition to the base data.

The image generation unit 50 of the image generation device 207 generates a spectroscopic image based on the spectroscopic function 36 set by the function setting unit 30 and the spectral coefficient 42 acquired from the information management device 100. In this case, the load of data processing on the terminal 200 can be further reduced.

FIG. 13 shows an example of a computer 1200 in which a plurality of aspects of the present invention can be embodied in whole or in part. The program installed on the computer 1200 causes the computer 1200 to function as an operation associated with the device according to an embodiment of the present invention or as one or more "parts" of the device, or the operation or the one or more "parts". A unit can be run and / or a computer 1200 can be run a process according to an embodiment of the invention or a stage of the process. Such a program may be executed by the CPU 1212 to cause the computer 1200 to perform a specific operation associated with some or all of the blocks of the flowchart and block diagram described herein. The function may be implemented by FPGA. As an example, the program causes the computer 1200 to function as one of the information management device 100, the terminal 200, and the image generator 207.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, a graphic controller 1216, and a display device 1218, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a hard disk drive 1224, a DVD-ROM drive 1226, and an IC card drive, which are connected to the host controller 1210 via the input / output controller 1220. The computer also includes legacy I / O units such as the ROM 1230 and keyboard 1242, which are connected to the I / O controller 1220 via the I / O chip 1240.

The CPU 1212 operates according to the programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in the graphic controller 1216 itself, and displays the image data on the display device 1218.

The communication interface 1222 communicates with other electronic devices via a network. The hard disk drive 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD-ROM drive 1226 reads the program or data from the DVD-ROM 1201 and provides the program or data to the hard disk drive 1224 via the RAM 1214. The IC card drive reads the program and data from the IC card and / or writes the program and data to the IC card.

The ROM 1230 internally stores a boot program or the like executed by the computer 1200 at the time of activation, and / or a program depending on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

The program is provided by a computer-readable storage medium such as a DVD-ROM 1201 or an IC card. The program is read from a computer-readable storage medium, installed on a hard disk drive 1224, RAM 1214, or ROM 1230, which is also an example of a computer-readable storage medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and provides a link between the program and the various types of hardware resources described above. The device or method may be configured to implement the operation or processing of information according to the use of the computer 1200.

For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and performs communication processing on the communication interface 1222 based on the processing described in the communication program. You may order. Under the control of the CPU 1212, the communication interface 1222 reads and reads transmission data stored in a transmission buffer area provided in a recording medium such as a RAM 1214, a hard disk drive 1224, a DVD-ROM 1201, or an IC card. The data is transmitted to the network, or the received data received from the network is written to the reception buffer area or the like provided on the recording medium.

Further, the CPU 1212 allows the RAM 1214 to read all or necessary parts of a file or database stored in an external recording medium such as a hard disk drive 1224, a DVD-ROM drive 1226 (DVD-ROM1201), or an IC card. Various types of processing may be performed on the data on the RAM 1214. The CPU 1212 may then write back the processed data to an external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on recording media for information processing. The CPU 1212 describes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, and information retrieval described in various parts of the present disclosure with respect to the data read from the RAM 1214, and is specified by the instruction sequence of the program. Various types of processing may be performed, including / replacement, etc., and the results are written back to the RAM 1214. Further, the CPU 1212 may search for information in a file, a database, or the like in the recording medium. For example, when a plurality of entries each having an attribute value of the first attribute associated with the attribute value of the second attribute are stored in the recording medium, the CPU 1212 is the first of the plurality of entries. The attribute value of the attribute of is searched for the entry that matches the specified condition, the attribute value of the second attribute stored in the entry is read, and the first attribute satisfying the predetermined condition is selected. You may get the attribute value of the associated second attribute.

The program or software module described above may be stored on a computer 1200 or in a computer-readable storage medium near the computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a dedicated communication network or a server system connected to the Internet can be used as a computer-readable storage medium, whereby the program can be sent to the computer 1200 via the network. provide.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the claims that the form with such modifications or improvements may also be included in the technical scope of the invention.

The order of execution of operations, procedures, steps, steps, etc. in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is particularly "before" and "prior to". It should be noted that it can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. is not.

10 ... Image acquisition unit, 12 ... Captured image, 22 ... Base function, 23 ... Band, 20 ... Base acquisition unit, 30 ... Function setting unit, 32 ... Virtual spectroscopy Filter, 36 ... Spectral function, 40 ... Spectral coefficient calculation unit, 42 ... Spectral coefficient, 44 ... Bit, 46 ... Region, 48 ... Region, 50 ... Image generation unit , 100 ... Information management device, 102 ... Management device side data transmission unit, 104 ... Management device side data acquisition unit, 106 ... Base update unit, 108 ... Base storage unit, 200 ...・ Terminal, 202 ・ ・ ・ terminal side data transmission unit, 204 ・ ・ ・ terminal side data acquisition unit, 206 ・ ・ ・ spectrum acquisition unit, 207 ・ ・ ・ image generator, 208 ・ ・ ・ spectrum estimation unit, 250 ・ ・Information management system, 252 ... network, 300 ... object, 302 ... surface, 410 ... lighting unit, 412 ... light source, 438 ... lens, 440 ... optical axis, 802 ... Spectral image generation unit, 804 ... Image generation unit, 806 ... Spectral image, 1200 ... Computer, 1201 ... DVD-ROM, 1210 ... Host controller, 1212 ... CPU , 1214 ... RAM, 1216 ... graphic controller, 1218 ... display device, 1220 ... input / output controller, 1222 ... communication interface, 1224 ... hard disk drive, 1226 ... DVD-ROM Drive, 1230 ... ROM, 1240 ... I / O chip, 1242 ... Keyboard

Claims (12)

An information management device that manages information about objects.
A basal storage unit that is classified into one or more categories and stores basal data regarding the spectrum of the object.
A data acquisition unit that acquires spectrum measurement data of the object and information related to the category, and
An information management device including a base update unit that updates the base data stored in the base storage unit based on the data acquired by the data acquisition unit. The data acquisition unit acquires ray space data indicating the ray space of illumination when the spectrum measurement data is generated, and obtains the ray space data.
The information management device according to claim 1, wherein the base updating unit updates the base data using the ray space data. The information management device according to claim 1 or 2, wherein the data acquisition unit acquires the spectrum measurement data and information related to the category from a plurality of terminals via a network. The information management device according to claim 3, further comprising a data transmission unit that transmits the basal data stored by the basal storage unit to the terminal in response to a request from any of the plurality of terminals. .. The information management device according to claim 4, wherein the data transmission unit transmits the basic data corresponding to the category to which the object specified by the terminal belongs to the terminal. The data acquisition unit acquires an image captured by capturing the object from the terminal.
A spectrum estimation unit that estimates the spectrum of the object based on the captured image and the basal data is provided.
The information management device according to claim 4 or 5, wherein the data transmission unit transmits information on the estimated spectrum to the terminal. A terminal that can be connected to the information management device according to claim 4 or 5.
An image acquisition unit that acquires an image of the object,
A terminal-side data acquisition unit that acquires the base data from the information management device,
A terminal including an image generation unit that generates a spectroscopic image of the object based on the captured image and the basal data. A terminal that can be connected to the information management device according to claim 6.
An image acquisition unit that acquires an image of the object,
A terminal-side data transmission unit that transmits the captured image to the information management device,
A terminal-side data acquisition unit that acquires information about the spectrum of the object from the information management device, and
A terminal equipped with. The information management device according to claim 4 or 5,
An information management system including the terminal according to claim 7. The information management device according to claim 6 and
An information management system including the terminal according to claim 8. A program for causing a computer to function as the information management device according to any one of claims 1 to 6. A program for operating a computer as a terminal according to claim 7 or 8.

Images