JP2014089075A - Spectral reflectance measuring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a spectral reflectance measuring system capable of: improving accuracy in spectral reflectance measurement even in the case of having an error factor such as variations in spectral transmittance or positional deviation; and contributing to cost reduction due to the relaxing of manufacturing accuracy.SOLUTION: A plurality of kinds of light having known spectral energy distributions are made to be incident to store spectral sensitivity characteristic data for each area on a sensor in a memory 30. Based on the stored spectral sensitivity characteristic data, a spectral transmittance measurement of each pixel is calculated by a pixel spectral characteristic calculation unit 34. Based on the calculated spectral transmittance, a spectral reflectance of an object is calculated by an object spectral reflectance calculation unit 36.

Description

  The present invention relates to a spectral reflectance measurement system that measures the spectral reflectance of an object.

As an apparatus for measuring spectral reflectance, a spectroscope using a prism, a diffraction grating (grating), a liquid crystal tunable filter, or the like is known.
By measuring the spectral reflectance of the object, accurate color reproduction that does not depend on illumination or a display device can be performed in the visible light region.
In the ultraviolet and infrared regions, it is possible to detect a difference in components even with an object whose difference is not obvious when viewed with the human eye.

Conventional spectrometers have the problem that they can measure the spectrum of a point or line segment in real time (instantaneously), but cannot measure the surface at once.
When measuring the surface, it was measured by scanning using a spectrometer for line measurement.
In this method, when measuring the spectrum of a cell or the like, the subject itself moves, so that there is a problem that the spectrum corresponding to the subject cannot be measured accurately.

In Patent Document 1, for the purpose of measuring a two-dimensional spectral reflectance characteristic, a plurality of channels having different wavelength regions are switched using a wavelength tunable filter, and a spectrum of a photographing subject is estimated from a photographed multiband image. A method is disclosed.
While switching channels with a wavelength tunable filter, a multiband image composed of a plurality of original images obtained by photographing the same subject by the photographing means is acquired, and the spectrum of the subject is estimated from the multiband image.
The number of channels of the wavelength tunable filter is 10 or more.

However, since the method described in Patent Document 1 obtains band images one by one while switching channels with a wavelength tunable filter, spectrum images of a plurality of channels cannot be measured in real time (instantaneously) at a time.
That is, the operation of obtaining each band image by switching the channel of the wavelength tunable filter is required 10 times or more, and it is inevitable that time is required.
For this reason, since the subject itself moves, the problem that the spectrum corresponding to the subject cannot be measured accurately has not been solved.

For such a spectroscope, there is a light field camera (plenoptic camera) as means for measuring the spectral reflectance characteristics of points and line segments in real time (instantaneously).
In plenoptic cameras, it is known that a two-dimensional spectral reflectance of a test object can be measured in one shot by placing a filter with spatially varying spectral transmittance near the aperture of the main lens. .

  Assuming a spectral reflectance measurement device that uses a plenoptic camera configuration, under ideal conditions, the spectral transmittance of the sensor is determined based on the spectral transmittance of the filter and the spectral sensitivity characteristics of the sensor. Spectral reflectance can be measured.

However, in actual plenoptic cameras, the spectral characteristics of the filter may vary due to various factors such as variations in the spectral transmittance of the filter, manufacturing errors such as displacement of the filter installation position, and deviation of the incident angle of the reflected light from the object. It has been found that the method of determining the spectral transmittance of the sensor from only the transmittance and the spectral sensitivity characteristic of the sensor includes many errors.
For this reason, even if the spectral reflectance measuring device using the configuration of the plenoptic camera is embodied, it cannot be said that the spectral reflectance measurement accuracy is sufficient.
In order to increase the measurement accuracy, it is necessary to reduce manufacturing errors as much as possible, but in such cases, an increase in cost is inevitable.

  The present invention was devised in view of such a current situation, and can improve the accuracy of spectral reflectance measurement even if there is an error factor such as dispersion or misalignment of the spectral transmittance of the filter. It is an object of the present invention to provide a spectral reflectance measurement system that can contribute to a reduction in cost due to relaxation.

  In order to achieve the above object, the present invention eliminates the influence of the error factor by making the light of a known wavelength incident after manufacturing, obtaining spectral sensitivity characteristic data, and performing calibration based on this. It was decided to.

  Specifically, the present invention includes an optical system, a sensor that converts optical information collected by the optical system into electronic information, a filter that is disposed near a stop of the optical system, and that has a plurality of spectral characteristics. A lens array disposed between the optical system and the sensor and arranged with a plurality of lenses substantially parallel to a two-dimensional plane direction of the sensor; and spectral sensitivity characteristic data for each region on the sensor. A spectral reflectance measurement system having storage means and subject spectral reflectance calculation means for calculating the spectral reflectance of the subject based on spectral sensitivity characteristic data stored in the storage means.

  According to the present invention, even if there is an error factor such as a manufacturing error, it is possible to improve the accuracy of the spectral reflectance measurement, and it is possible to realize a spectral reflectance measuring system that can contribute to a reduction in cost due to a reduction in manufacturing accuracy. .

1 is a schematic configuration diagram of a spectral reflectance measurement system according to an embodiment of the present invention. It is a control block diagram. It is a schematic diagram which shows the example of arrangement | positioning of the light source at the time of calibration. It is a schematic diagram which shows the structure in the case of measuring the spectral reflectance of an object. It is a flowchart which shows the measurement procedure of a spectral reflectance. It is a schematic diagram for demonstrating the principle of the spectral reflectance measuring system of this invention. It is a figure which shows an example of the relationship between the position of the filter and the transmittance | permeability which a spectral transmittance changes continuously. It is a figure which shows an example of the picked-up image by a plenoptic camera structure.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
Before describing the specific configuration of the spectral reflectance measurement system according to the present embodiment, the principle of the present invention will be described with reference to FIG.
Here, for easy understanding, the main lens 24 as an optical system is shown as a single lens, and the aperture position S of the main lens 24 is the center of the single lens.
At the center of the main lens 24, a filter 26 whose spectral transmittance continuously varies spatially is disposed. Here, a state in which the spectral transmittance continuously changes spatially is displayed with color shading.
In practice, no filter is located in the lens.

The directionality of the continuity of the spectral transmittance of the filter 26 is not limited as long as it is within one plane.
For example, the surface that is orthogonal to the optical axis of the main lens 24 may have continuity in the vertical direction in FIG. 1, the direction orthogonal thereto, or the direction intersecting obliquely.

For example, the relationship between the position of the filter where the spectral transmittance continuously changes and the transmittance is shown in FIG. 7 (extracted from the website of Nikon Corporation's linear variable filter (LVF)).
Such a filter can be produced by depositing a thin film in a wedge shape on a transparent substrate (optical glass or the like).
That is, by changing the thickness of the thin film steplessly, the spectral transmittance also changes continuously.

A microlens array (hereinafter referred to as “MLA”) 3 composed of a plurality of microlenses (small lenses) is disposed near the condensing position of the main lens 24.
A light receiving element array as a sensor is arranged on the image surface 6. The light receiving element array includes a plurality of light receiving elements. In the following, reference numeral 6 is denoted as an image sensor.
The diameter of the micro lens of the MLA 3 and each light receiving element constituting the image sensor 6 are in a relation of approximately 30: 1 to 10: 1.
The MLA 3 is a lens array in which a plurality of small lenses are arranged substantially parallel to the two-dimensional plane direction of the image sensor 6.

The image sensor 6 is an image sensor represented by a CCD or the like, and is a monochrome sensor in which a color filter for each pixel is not mounted.
The image sensor 6 is a sensor that converts optical information collected by the main lens 24 into electronic information.
Of the light emitted from the object 1, the light beam incident on the opening of the main lens 24 is the target of spectral reflectance measurement.
The light beam incident on the main lens 24 is a collection of innumerable light beams, and each light beam passes through different positions of the stop of the main lens 24.

Since the filter 26 is disposed at the stop position of the main lens 24, each light beam passes through a filter having a different spectral transmittance.
The light rays that have passed through the filter 26 once form an image in the vicinity of the MLA 3, and then reach different positions of the sensor by the MLA 3.
That is, since the position of the sensor surface corresponds to the filter position through which the light beam has passed, the spectral reflectance at a certain point of the object can be measured simultaneously.

  Although FIG. 6 shows the case of only one point on the optical axis, the same is true for off-axis, and two-dimensional spectral reflectance can be measured simultaneously.

The filter 26 is disposed near the stop of the main lens 24.
Here, “near the aperture” means a portion that includes the aperture position and through which rays of various angles of view can pass.

The image photographed with the configuration of FIG. 6 is a series of small circles as shown in FIG. The reason for forming a circle is that the aperture shape of the single lens is a circle.
Each small circle is referred to herein as a “macro pixel”. When all the macro pixels are collected, it becomes one image.
Each macro pixel is formed directly under each small lens (micro lens) constituting the MLA 3. The diameter of the macro pixel and the diameter of the microlens are almost the same.

As shown in FIG. 6, the light beam that has passed through the lower portion of the filter 26 reaches the upper portion of the macro pixel, and the light beam that has passed through the upper portion of the filter 26 reaches the lower portion.
When the filter 26 is arranged so that the lower part of the filter 26 has a short wavelength and the upper part has a spectral transmittance of a long wavelength, the short wavelength and the lower part of the light reach the upper part of the macro pixel.
Spectral reflectance can be obtained by calculating the average value for each macro pixel row and calculating in consideration of the spectral intensity of illumination, the spectral transmittance of the lens, the spectral transmittance of the filter, and the spectral sensitivity of the light receiving element. it can.

FIG. 1 shows a configuration of a spectral reflectance measurement system according to the present embodiment.
The spectral reflectance measurement system 10 has an imaging unit 12 for acquiring spectral sensitivity characteristic data, and is electrically connected to the imaging unit 12, and processes the acquired spectral sensitivity characteristic data to process the subject (object 1). And a processing unit 16 for measuring the spectral reflectance.
The imaging unit 12 includes a lens module 18 and a camera unit 20.
The lens module 18 includes a lens barrel 22, a main lens 24 provided in the lens barrel 22, a filter 26, and a lens 28.

The camera unit 20 has an MLA 3 and an image sensor 6 therein.
The MLA 3 has a configuration in which a plurality of microlenses are arranged in a direction orthogonal to the optical axis of the main lens 24.

As shown in FIG. 2, the processing unit 16 includes a memory 30 as a storage unit and a subject spectral reflectance calculation unit 32.
The subject spectral reflectance calculation means 32 includes a pixel spectral characteristic calculator 34, a subject spectral reflectance calculator 36, and a controller 38.
The memory 30 is a non-volatile memory that stores (stores) data such as spectral sensitivity characteristics for each region on the image sensor 6.

The pixel spectral characteristic calculator 34 calculates the spectral transmittance of each pixel from the spectral characteristics of each pixel stored in the memory 30.
The subject spectral reflectance calculator 36 calculates the spectral reflectance of the subject using the spectral transmittance of each pixel calculated by the pixel spectral characteristic calculator 34.
The control unit 38 controls the entire processing unit 16.

In the present embodiment, the subject spectral reflectance calculation unit 32 is electrically connected to the imaging unit 12 to construct the entire system. However, a part of the subject spectral reflectance calculation unit 32, for example, the memory 30 is included in the imaging unit. It is good also as a structure with which 12 is provided integrally. The electrical connection can be wired or wireless.
Further, the entire subject spectral reflectance calculating unit 32 may be incorporated in the imaging unit 12.

The procedure for measuring the two-dimensional spectral reflectance will be specifically described below.
First, illumination having a plurality of known spectral energy distributions is incident on the imaging unit 12 after manufacture.
As shown in FIG. 3, a surface light source 50 having a sufficiently large area as compared with the diaphragm surface of the imaging unit 12, that is, the diaphragm surface of the main lens 24, is disposed on the front surface of the imaging unit 12.
FIG. 3 shows an arrangement example of the surface light source 50 at the time of calibration.

A plurality of types (six types here) of surface light sources 50 that emit incident light are prepared, and the spectral energy distributions of the six types of surface light sources are all known.
By measuring the camera response (= brightness value) of each pixel once for each surface light source, six camera responses are obtained for each pixel.
Table 1 shows an example of sensor sensitivity values transmitted through the MLA 3 and received by the small area 6a of the image sensor 6. The small area 6a is set to 3 × 3 pixels (pixel unit) for simplification. In Table 1, λ means wavelength.

The part divided from the X coordinate and the Y coordinate in Table 1 indicates a region on the sensor, and the camera response (= luminance value) of each pixel is synonymous with the spectral sensitivity characteristic data.
That is, the region is classified for each region where incident light passes through the filter.

Data of the number of pixels × the number of wavelengths (3 × 3 × 6 = 72 in the example of Table 1) is stored in the memory 30 together with the spectral energy distributions of the six types of surface light sources.
In this embodiment, six types are used. However, the more types of surface light sources, the more accurately the spectral transmittance of each pixel can be calculated.
Surface light sources 1, 2, 3, 4, 5, and 6 distinguished by wavelength numbers are surface light sources having strong energy only at 400 nm, 450 nm, 500 nm, 550 nm, 600 nm, and 650 nm, respectively, and the intensity of each surface light source Used the same.

Next, a method for measuring the spectral reflectance of the object 1 will be described.
FIG. 4 shows an installation example when obtaining the spectral reflectance of the object 1. In FIG. 4, the code | symbol 52 has shown the light source for illumination.

[Calculation method of spectral reflectance]
The calculation method of the spectral reflectance is modeled as follows (Reference: Introduction to Spectral Image Processing, Chapter 4, University of Tokyo Press).
G (x, y) = S ^ tEr (x, y) + n (x, y) ... Spectral reflectance estimation model formula
x, y: pixel position
G: mx1 column vector representing the camera response r: lx1 column vector representing the spectral reflectance of the object
S = [s1, s2, ... sm]: matrix of lxm, i-th column si is spectral sensitivity characteristic of i-th band
E: lxl diagonal matrix, the diagonal component of the matrix is the spectral energy distribution of the illumination
N: Noise Note that ^ t is a transposed matrix, camera

[Calculation method of spectral transmittance of each pixel]
The spectral transmittance of each pixel can be calculated by changing the above model as follows.
G (x, y) = S ^ tEr (x, y) + n (x, y) ... Pixel spectral transmittance estimation model formula
x, y: pixel position
G: mx1 column vector representing the camera response (m is the number of surface light sources)
r: mxl matrix representing the spectral reflectance of the surface light source (l is the number of bands)
S: Spectral response for each band of pixel (x, y) (column vector of lx1)
E: Square matrix (because lighting is not used from Fig. 5)
N: Noise Note that ^ t is a transposed matrix, camera

In the above spectral transmittance calculation method, “each band” corresponds to “each region passing through a filter”.
Actually, a method of calculating the spectral transmittance of the pixel (x, y) = (0, 0) will be described as an example.
The spectral sensitivity characteristic of the pixel (0, 0) to be calculated is S (0, 0) = [s1, s2, s3, s4, s5, s6].
s1, s2, s3, s4, s5, s6 are
The spectral sensitivity characteristics are wavelengths 400 nm, 450 nm, 500 nm, 550 nm, 600 nm, and 650 nm, respectively.

Based on the model, the camera response obtained by the pixel (0, 0) is set as a column vector of mx1.
From Table 1,
G = [245,233,100,60,20,13] ^ t can be set. In addition, the spectral reflectance of the object (= surface light source) when these camera responses are obtained is that each surface light source has a normalized light amount and has a peak only at a specific wavelength. = E (square matrix).

By assuming that N = 0 and solving the above model equation, S (0,0) = [245,233,100,60,20,13] ^ t can be obtained (if N ≠ 0, (Refer to Introduction to spectral image processing, p65, University of Tokyo Press).
S (0,0) represents that the spectral transmittance at the vertex (0,0) is 245 at 400 nm, 233 at 240 nm, 13 at 650 nm.
By performing this calculation for all the pixels, the spectral transmittance of each pixel can be obtained.

S: A spectral sensitivity characteristic (column vector of lx1) for each band of the pixel (x, y) is selected and set from spectral sensitivity characteristic data stored in the memory 30 as shown in Table 1.
Executing the spectral transmittance calculation method in this way means calibration of the imaging unit 12 after manufacture.
Thereby, the error of the imaging unit 12 that occurs during manufacturing can be corrected.
That is, according to the present invention, illumination having a plurality of known spectral energy distributions is made incident, and the spectral transmittance of the combined sensor and filter, that is, the camera is measured. It is defined and used.

In this embodiment, the spectral transmittance of each pixel is calculated. However, when calculating the spectral sensitivity of all the pixels, the amount of calculation becomes a problem, or the size of the area in which the output data is stored is calculated. Becomes a problem.
In order to avoid this, it is important to sample pixels for calculating the spectral reflectance.

As a sampling method, there are a method of simply performing thinning, and a method of determining a corresponding pixel region on the filter and the sensor and registering and using the spectral sensitivity for each pixel region as shown in FIG.
Further, there is a method of reducing the number of data to be registered in the memory by correcting pixels between the sampled data by linear interpolation or the like.
In the next [Method for calculating spectral reflectance of subject], calculation is performed after the sampled spectral reflectance is restored.

[Calculation method of spectral reflectance of subject]
A method for obtaining the spectral reflectance of the object 1 will be described.
As can be seen from FIG. 4, the ambient light reflected by the object 1 is arranged so as to capture only the small area 6 a of the image sensor 6.
Therefore, the spectral reflectance estimation model formula can be applied with m = 9.

Specifically, S =
[S (0,0), S (0,1), S (0,2), S (1,0), S (1,1), S (1,2), S (2,0), S (2,1), S (2,2)].
Further, assuming that the illumination has the same energy distribution at each wavelength, let E be a square matrix.

If the sensor sensitivity value of pixel (x, y) is V (x, y),
G = [V (0,0), V (0,1), V (0,2), V (1,0), V (1,1), V (1,2), V (2,0 ), V (2,1), V (2,2)].
By solving for r using these, the spectral reflectance of the object 1 can be obtained.

The procedure for measuring the spectral reflectance of the object 1 will be described based on the flowchart of FIG.
First, as shown in FIG. 4, the object 1 is installed at a position away from the imaging unit 12 by a specified distance (S1).
Next, shooting is performed by pressing a shooting button (not shown) of the imaging unit 12 (S2).
The captured image is acquired by the image sensor 6 and stored in the memory 30 (S3).

Next, the control unit 38 that has received the signal indicating the end of imaging instructs the pixel spectral characteristic calculation unit 34 to measure the spectral transmittance of each pixel (S4).
In the pixel spectral characteristic calculation unit 34, the calculation is performed based on the above-described [Method of calculating spectral transmittance of each pixel].
The calculated spectral transmittance of each pixel is stored in the memory 30 (S5).

Next, the control unit 38 instructs the subject spectral reflectance calculation unit 36 to measure the spectral reflectance of the object 1 (S6).
The subject spectral reflectance calculator 36 calculates the spectral reflectance of the object 1 using the “photographed image and the spectral transmittance of each pixel” stored in the memory 30 as input.
The calculation method follows the above [Calculation Method of Spectral Reflectance of Subject].

  As is apparent from the above description, according to the present invention, by adding the memory 30 and the subject spectral reflectance calculation means 32, an error factor such as a manufacturing error can be obtained even in the configuration of an existing plenoptic camera. Can be eliminated, and a highly accurate spectral reflectance measurement system can be obtained.

DESCRIPTION OF SYMBOLS 1 Object 3 Micro lens array as a lens array 6 Image sensor as a sensor 12 Imaging part 24 Main lens as an optical system 26 Filter 32 Subject spectral reflectance calculation means

JP 2001-99710 A

Claims (5)

Optical system,
A sensor that converts optical information collected by the optical system into electronic information;
A filter disposed near the stop of the optical system and having a plurality of spectral characteristics;
A lens array that is arranged between the optical system and the sensor and in which a plurality of lenses are arranged substantially parallel to a two-dimensional plane direction of the sensor;
Storage means for storing spectral sensitivity characteristic data for each area on the sensor;
Subject spectral reflectance calculating means for calculating the spectral reflectance of the subject based on spectral sensitivity characteristic data stored in the storage means;
Spectral reflectance measurement system having The spectral reflectance measurement system according to claim 1,
The spectral reflectance measurement system according to claim 1, wherein the spectral sensitivity characteristic data is obtained by making a plurality of types of light having known spectral energy distributions incident. The spectral reflectance measurement system according to claim 2,
The light having a known spectral energy distribution is emitted from a surface light source having a larger area than the diaphragm surface of the optical system. In the spectral reflectance measuring system according to any one of claims 1 to 3,
The spectral reflectance measurement system according to claim 1, wherein the region is a pixel unit. In the spectral reflectance measuring system according to any one of claims 1 to 3,
The said area | region is classified for every area | region where incident light passes the said filter, The spectral reflectance measuring system characterized by the above-mentioned.

Images