JP7331439B2 - IMAGING DEVICE, SPECTRAL FILTER, AND IMAGING METHOD - Google Patents

Description

The present disclosure relates to imaging devices, spectral filters, and imaging methods.

2. Description of the Related Art Conventionally, techniques related to imaging devices such as CCDs (Charge-Coupled Devices) and CMOSs (Complementary Metal-Oxide-Semiconductors) used in imaging devices are known.

For example, Patent Document 1 relates to a solid-state imaging device represented by a light receiving device such as a CCD and a CMOS. A solid-state imaging device having a color filter of one color selected from a plurality of colors for each pixel is disclosed.

Japanese Patent No. 5017812

A spectral filter included in such a conventional imaging device is manufactured so that a single dye out of a plurality of dyes included in the spectral filter is formed in a region having the same arrangement as one or more pixels. . Therefore, a spectral filter has a fine structure equivalent to that of a pixel. At this time, if the types of dyes increase, the number of processes for forming the dyes on the spectral filter increases according to the number of types of dyes. As a result, manufacturing costs have increased with the complication of the manufacturing process.

An object of the present disclosure is to provide an imaging device, a spectral filter, and an imaging method that can reduce manufacturing costs even if the number of types of dyes increases.

An imaging device according to some embodiments is an imaging device that acquires image data based on an imaging signal, comprising: an imaging device that outputs the imaging signal; a control unit that generates the image data, wherein the imaging element includes a spectral filter containing at least three types of dyes having different light transmission wavelengths, and a plurality of pixels arranged in a grid, and an image sensor that receives the light transmitted through the spectral filter and converts it into the imaging signal as an electrical signal, wherein the pigment is formed, among the plurality of regions on the same plane of the spectral filter. , one said region and another said region overlap each other. According to such an imaging device, the manufacturing cost can be reduced even if the types of dyes are increased. More specifically, unlike a conventional spectral filter that complicates the manufacturing process and causes high cost, the spectral filter according to one embodiment has a plurality of regions in which dyes are formed. and overlap with other regions. Therefore, the manufacturing method of the spectral filter is simplified and the manufacturing cost is reduced.

An imaging device according to one embodiment includes a storage unit that stores transmittance data in which wavelength dependence of transmittance when the light passes through the spectral filter is associated with each pixel of the image sensor, The control unit calculates the spectral spectrum of the light incident on the spectral filter based on the transmittance data acquired from the storage unit and the imaging signal output from the image sensor, and calculates the calculated spectral spectrum of the light. The image data may be generated based on the spectrum. Accordingly, even when the imaging apparatus according to the embodiment uses the spectral filter manufactured by the simplified manufacturing method, the control unit can accurately calculate the optical spectrum of the light.

A spectral filter according to some embodiments is a spectral filter used in an imaging element provided in an imaging device that acquires image data based on an imaging signal, and contains at least three types of dyes having different light transmission wavelengths. and wherein, of a plurality of areas on the same plane of the spectral filter in which the dye is formed, one of the areas overlaps with another of the areas. With such a spectral filter, the manufacturing cost can be reduced even if the types of dyes are increased. More specifically, unlike a conventional spectral filter that complicates the manufacturing process and causes high cost, the spectral filter according to one embodiment has a plurality of regions in which dyes are formed. and overlap with other regions. Therefore, the manufacturing method of the spectral filter is simplified and the manufacturing cost is reduced.

An imaging method according to some embodiments includes a spectral filter including at least three types of dyes having different light transmission wavelengths, and a plurality of pixels arranged in a grid pattern, wherein light transmitted through the spectral filter and an image sensor that receives an image and converts it into an imaging signal as an electric signal, and an imaging method for acquiring image data based on the imaging signal output from an imaging device, wherein the imaging signal is received from the image sensor obtaining the wavelength dependence of the transmittance when the light is transmitted through the spectral filter to the transmittance data associated with each pixel of the image sensor and the imaging signal obtained from the image sensor; calculating the spectral spectrum of the light incident on the spectral filter based on the above; and generating the image data based on the calculated spectral spectrum, wherein the pigment is formed, wherein the Among the plurality of regions on the same plane of the spectral filter, one said region and another said region overlap each other. According to such an imaging method, the manufacturing cost can be reduced even if the types of dyes are increased. More specifically, unlike a conventional spectral filter that complicates the manufacturing process and causes high cost, the spectral filter according to one embodiment has a plurality of regions in which dyes are formed. and overlap with other regions. Therefore, the manufacturing method of the spectral filter is simplified and the manufacturing cost is reduced. Furthermore, even when a spectral filter manufactured by a simplified manufacturing method is used, the optical spectrum of light can be calculated with high accuracy in the imaging method according to the embodiment.

An imaging method according to an embodiment may include pre-storing the transmittance data before obtaining the imaging signal. Thereby, the transmittance data is used as a calibration value of the image sensor. By pre-storing the transmittance data by the calibration method in this manner, the imaging method according to the embodiment can calculate the light spectrum with higher accuracy.

Advantageous Effects of Invention According to the present disclosure, it is possible to provide an imaging device, a spectral filter, and an imaging method that can reduce manufacturing costs even if the number of types of dyes increases.

1 is a block diagram showing an example of the configuration of an imaging device according to one embodiment; FIG. FIG. 2 is a schematic diagram showing a single spectral filter in FIG. 1; FIG. 2 is a schematic diagram showing an example of the configuration of the imaging element in FIG. 1; FIG. 4 is a graph showing a first example of transmittance when light passes through the spectral filter of FIG. 3; FIG. 4 is a graph showing a second example of transmittance when light passes through the spectral filter of FIG. 3; FIG. 4 is a graph showing a third example of transmittance when light passes through the spectral filter of FIG. 3; 2 is a flow chart showing an example of an imaging method using the imaging apparatus of FIG. 1; It is a schematic diagram which shows the structure of the conventional image pick-up element. FIG. 7 is a graph showing a first example of transmittance when light passes through the spectral filter of FIG. 6; FIG. 7 is a graph showing a second example of transmittance when light passes through the spectral filter of FIG. 6; FIG. 7 is a graph showing a third example of transmittance when light passes through the spectral filter of FIG. 6;

First, the background and problems of the prior art will be mainly described.

FIG. 6 is a schematic diagram showing the configuration of a conventional imaging device. A conventional imaging device has a spectral filter and an image sensor. FIG. 6 is a schematic diagram showing an image sensor in which only a part of an image sensor such as CCD and CMOS is cut out. Similarly, FIG. 6 is a schematic diagram showing a spectral filter in a state where only a part of the spectral filter is cut out corresponding to the image sensor.

For example, the image sensor shown in FIG. 6 has a plurality of grid-like pixels arranged in three columns and three rows. The spectral filter has the same size and shape as the pixels of the image sensor, and has a plurality of regions with the same arrangement as one pixel. Each region of the spectral filter is formed with only a single pigment out of the plurality of pigments contained in the spectral filter. The spectral filter is arranged in front of the image sensor so as to be parallel to the image sensor along, for example, the incident direction of light. Alternatively, the spectral filter is formed integrally with the image sensor.

The light of the object to be measured that has entered the imaging device passes through the spectral filter and enters each pixel of the image sensor. Light is converted into electrical signals at each pixel of the image sensor. An imaging apparatus having the imaging element disperses the light of the object to be measured based on the electrical signal output from the imaging element to generate image data.

7A is a graph showing a first example of transmittance when light passes through the spectral filter of FIG. 6. FIG. The graph of FIG. 7A shows, for example, the transmittance of the spectral filter of FIG. 6 when light passes through the pigment formed in the region corresponding to the pixels in the first vertical and first horizontal columns on the image sensor. Shows wavelength dependence. 7B is a graph showing a second example of transmittance when light passes through the spectral filter of FIG. 6. FIG. The graph in FIG. 7B shows, for example, the transmittance of the spectral filter in FIG. 6 when light passes through the pigment formed in the regions corresponding to the pixels in the first vertical and second horizontal columns on the image sensor. Shows wavelength dependence. 7C is a graph showing a third example of transmittance when light passes through the spectral filter of FIG. 6. FIG. The graph of FIG. 7C shows, for example, the transmittance of the spectral filter of FIG. 6 when light passes through the pigment formed in the region corresponding to the pixels in the first vertical row and the third horizontal row on the image sensor. Shows wavelength dependence.

As shown in FIGS. 7A to 7C, the transmittances of light passing through the dyes formed in the corresponding regions on the spectral filter have different wavelength characteristics. For example, the pigments formed in each region have light transmission wavelengths different from each other.

As described above, in the conventional imaging device shown in FIG. 6, for example, in the spectral filter, dyes are formed in regions having the same arrangement as the pixels on the image sensor. Therefore, in manufacturing a spectral filter, it is necessary to realize a fine structure in the spectral filter as well as the pixel on the image sensor. At this time, if the types of dyes increase, the number of processes for forming the dyes on the spectral filter increases according to the number of types of dyes. As a result, manufacturing costs have increased with the complication of the manufacturing process.

An object of the present disclosure is to provide an imaging device, a spectral filter, and an imaging method that can reduce the manufacturing cost even if the number of types of dyes increases by simplifying the manufacturing method of the spectral filter. An embodiment of the present disclosure will be mainly described below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing an example of the configuration of an imaging device 1 according to one embodiment. An example of the configuration of an imaging device 1 according to an embodiment will be mainly described with reference to FIG.

The imaging device 1 is, for example, a hyperspectral camera, a multispectral camera, or the like. The imaging device 1 has an imaging element 10 , a control section 20 , a storage section 30 , an input section 40 and a communication section 50 . The imaging device 1 acquires image data based on the imaging signal.

The imaging device 10 is, for example, a CCD, a CMOS, or the like. The imaging device 10 outputs an imaging signal as an electrical signal to the control unit 20 based on the light L incident upon imaging of an image. The imaging device 10 has a spectral filter 11 and an image sensor 12 .

The spectral filter 11 is any filter element containing at least three dyes having different light transmission wavelengths. The image sensor 12 includes a plurality of pixels arranged in a grid pattern on the same plane, receives the light L that has passed through the spectral filter 11, and converts it into an image pickup signal as an electrical signal. For example, the image sensor 12 has, for each pixel included in the image sensor 12, a photoelectric conversion element that converts the light L into an electrical signal having a signal intensity corresponding to the intensity of the light L. FIG.

Control unit 20 includes one or more processors. For example, the control unit 20 includes a processor that enables processing related to the imaging device 1 . The control unit 20 is connected to each component constituting the imaging device 1 and controls and manages the entire imaging device 1 including each component. For example, the control unit 20 controls the imaging device 10 to acquire an imaging signal, and generates image data based on the imaging signal output from the imaging device 10 .

The storage unit 30 is an arbitrary storage such as a HDD (Hard Disk Drive), SSD (Solid State Drive), EEPROM (Electrically Erasable Programmable Read-Only Memory), ROM (Read-Only Memory), and RAM (Random Access Memory). It stores information necessary to implement the operation of the imaging device 1, including the device. The storage unit 30 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 30 is not limited to being built in the imaging device 1, and may be an external storage device connected via a digital input/output port such as a USB. The storage unit 30 stores, for example, transmittance data in which the wavelength dependence of the transmittance when the light L is transmitted through the spectral filter 11 is associated with each pixel of the image sensor 12 . The storage unit 30 stores image data generated by the control unit 20, for example.

The input unit 40 includes any input interface that receives an input operation by the user of the imaging device 1 . The input unit 40 receives an input operation by the user of the imaging device 1 and acquires input information by the user. The input unit 40 outputs the acquired input information to the control unit 20 . For example, the user may use the input unit 40 to input arbitrary information required for transmittance data stored in the storage unit 30 .

The communication unit 50 includes any communication interface compatible with any communication protocol based on wire or wireless. For example, the communication unit 50 may transmit image data generated by the control unit 20 to any external device. For example, the communication unit 50 may receive any information necessary for realizing the operation of the imaging device 1 from any external device. More specifically, the communication unit 50 may receive any information necessary for the transmittance data stored in the storage unit 30 or the transmittance data itself from any external device.

FIG. 2 is a schematic diagram showing the single spectral filter 11 in FIG. FIG. 2 is a schematic diagram showing the spectral filter 11 with only a part of the spectral filter 11 cut out. The configuration of the spectral filter 11 according to one embodiment will be mainly described with reference to FIG.

The spectral filter 11 includes, for example, five types of dyes C1, C2, C3, C4, and C5 having different light transmission wavelengths on the same plane. The spectral filter 11 is manufactured by coating or printing dyes C1, C2, C3, C4, and C5 on arbitrary substrates including, for example, glass substrates and organic film substrates. The dyes C1, C2, C3, C4, and C5 may be formed in the form of particles or thin films on the substrate surface during the coating or printing process. The spectral filter 11 is not limited to the above configuration, and may be manufactured by directly coating or printing the pigments C1, C2, C3, C4, and C5 on the pixel surfaces of the image sensor 12 .

Unlike a conventional spectral filter in which pigments are formed in completely divided regions corresponding to pixel columns of an image sensor, the spectral filter 11 according to one embodiment has pigments C1, C2, C3, C4, and C5. are formed on the same plane of the spectral filter 11, one region and the other region overlap each other. For example, one region labeled C2 in FIG. 2 overlaps another region labeled C5. For example, as shown in FIG. 2, a portion of one region and a portion of another region may overlap each other. Without being limited to this, for example, one region may be included in another region, and the entire one region and part of the other region may overlap each other.

Unlike conventional spectral filters in which pigments are formed in regions regularly arranged in correspondence with pixel columns of the image sensor, the spectral filter 11 according to one embodiment does not depend on the pixel columns of the image sensor 12. First, dyes C1, C2, C3, C4, and C5 are formed in a plurality of arbitrarily arranged regions. For example, in the spectral filter 11 according to one embodiment, dyes C1, C2, C3, C4, and C5 are formed in a plurality of randomly arranged regions. Therefore, in the spectral filter 11 according to one embodiment, there may be regions where none of the dyes C1, C2, C3, C4, and C5 are formed.

Unlike conventional spectral filters in which dyes are formed in regions having sizes corresponding to pixels of the image sensor, the spectral filter 11 according to one embodiment can be any size without depending on the size of the pixels of the image sensor 12 . The pigments C1, C2, C3, C4, and C5 are respectively formed in a plurality of regions having a size of . Unlike conventional spectral filters in which dyes are formed in regions having shapes corresponding to the pixels of the image sensor, the spectral filter 11 according to one embodiment has an arbitrary shape regardless of the shape of the pixels of the image sensor 12. dyes C1, C2, C3, C4, and C5 are respectively formed in a plurality of regions of .

FIG. 3 is a schematic diagram showing an example of the configuration of the imaging element 10 of FIG. 1. As shown in FIG. FIG. 3 is a schematic diagram showing the spectral filter 11 in a state in which only a part of the spectral filter 11 is cut out, corresponding to FIG. Similarly, FIG. 3 is a schematic diagram showing the image sensor 12 in a state in which only a part of the image sensor 12 is cut out corresponding to the spectral filter 11. As shown in FIG. The configuration of the imaging device 10 according to one embodiment will be mainly described with reference to FIG. 3 .

For example, the image sensor 12 shown in FIG. 3 has nine grid-like pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 arranged in three columns and three rows. The spectral filter 11 shown in FIG. 3 is the same as the spectral filter 11 shown in FIG. The spectral filter 11 is arranged in front of the image sensor 12 so as to be parallel to the image sensor 12 along, for example, the incident direction of the light L. As shown in FIG. Alternatively, the spectral filter 11 is formed integrally with the image sensor 12 .

The light L of the object to be measured that has entered the imaging device 10 passes through the spectral filter 11 and enters the pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 of the image sensor 12, respectively. The light L is converted into electrical signals by pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 of the image sensor 12, respectively. The imaging apparatus 1 having the imaging device 10 splits the light L of the object to be measured based on the imaging signal as the electrical signal output from the imaging device 10 to generate image data.

FIG. 4A is a graph showing a first example of transmittance when light L passes through spectral filter 11 of FIG. The graph of FIG. 4A shows, for example, the wavelength dependence of the transmittance when the light L is transmitted through the pigment formed in the region corresponding to the pixel P1 on the image sensor 12 in the spectral filter 11 of FIG. Here, the area corresponding to the pixel P1 means, for example, an area on the spectral filter 11 having the same size and shape as the pixel P1 of the image sensor 12 and having the same arrangement as the pixel P1.

FIG. 4B is a graph showing a second example of transmittance when the light L is transmitted through the spectral filter 11 of FIG. The graph of FIG. 4B shows the wavelength dependence of the transmittance when the light L is transmitted through the pigment formed in the region corresponding to the pixel P2 on the image sensor 12 in the spectral filter 11 of FIG. 3, for example. Here, the area corresponding to the pixel P2 means, for example, an area on the spectral filter 11 having the same size and shape as the pixel P2 of the image sensor 12 and having the same arrangement as the pixel P2.

FIG. 4C is a graph showing a third example of transmittance when the light L passes through the spectral filter 11 of FIG. The graph of FIG. 4C shows the wavelength dependence of the transmittance when the light L is transmitted through the pigment formed in the region corresponding to the pixel P3 on the image sensor 12 in the spectral filter 11 of FIG. 3, for example. Here, the area corresponding to the pixel P3 means, for example, an area on the spectral filter 11 having the same size and shape as the pixel P3 of the image sensor 12 and having the same arrangement as the pixel P3.

The pigments C1, C2, C3, C4, and C5 formed on the spectral filter 11 have different arrangements from the pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 of the image sensor 12. . Therefore, unlike the wavelength dependence of the transmittance when light is transmitted through only a single dye as shown in FIGS. 7A to 7C, the pixels P1, P2, P3, P4, The transmittance of the light L incident on each of P5, P6, P7, P8, and P9 has wavelength characteristics shown in FIGS. 4A to 4C as an example. More specifically, the transmittance of the light L incident on each of the pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 through the spectral filter 11 in FIG. Different wavelength characteristics for each region corresponding to pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9, corresponding to any number of dyes among C3, C4, and C5 . For example, there are various transmittances when light L is transmitted through an arbitrary number of dyes formed in regions corresponding to pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9. It has transmission wavelength, number of transmission peaks, transmission profile, and the like.

FIG. 5 is a flow chart showing an example of an imaging method using the imaging apparatus 1 of FIG. An example of the flow of the imaging method executed by the imaging apparatus 1 according to one embodiment will be mainly described with reference to FIG. 5 .

In step S101, the control unit 20 calculates the wavelength dependence of the transmittance when the light L is transmitted through the spectral filter 11 by the pixels P1, P2, P3, P4, P5, P6, P7, P8 of the image sensor 12, and Transmittance data associated with each P9 is stored in the storage unit 30 in advance.

In step S<b>102 , the control unit 20 acquires an imaging signal from the imaging device 10 , more specifically, the image sensor 12 .

In step S<b>103 , the control unit 20 calculates the spectral spectrum of the light L incident on the spectral filter 11 based on the transmittance data stored in the storage unit 30 and the imaging signal acquired from the image sensor 12 .

In step S104, the control unit 20 generates image data based on the spectrum of the light L calculated in step S103.

Below, the imaging method shown in the flowchart of FIG. 5 will be described more specifically. The imaging method includes a calibration method corresponding to step S101 in FIG. 5 and a calculation method corresponding to steps S102 and S103 in FIG. Various wavelengths, as shown in FIGS. 4A-4C, for each region corresponding to pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 by calibration and calculation methods as described below. Even when the spectral filter 11 having the characteristic of the transmittance of the light L is used, the light L can be spectrally dispersed, and a spectral image, that is, image data can be generated.

In the following, as shown in FIG. 3, nine pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 arranged in a grid pattern of three columns and three columns are used as an example of the image sensor 12. An image pickup method for performing spectroscopy as one block will be mainly described.

Since the entire image sensor 12 has a large number of pixels arranged, for example, in 3000 columns and 3000 columns, nine pixels P1, P2, P3, P4, P5, P6, P7, When P8 and P9 are one block, it has a large number of blocks arranged in 1000 vertical columns and 1000 horizontal columns. The spectroscopic image will have a resolution based on one block containing 9 pixels P1, P2, P3, P4, P5, P6, P7, P8 and P9 in 3 columns and 3 rows. The light L incident on the region of the spectral filter 11 corresponding to one block is filtered by, for example, an optical low-pass filter and splitter so that it has a uniform light intensity within the region of the spectral filter 11 corresponding to the same block. adjusted. The formation patterns of the above dyes in the spectral filter 11 may be completely different, regular, or identical between different blocks.

An example of the calibration method included in the imaging method will be mainly described.

In the calibration method, the wavelength dependence of the transmittance when the light L passes through the spectral filter 11 is associated with each of the pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 of the image sensor 12. Get transmittance data. For example, the transmittance data of the spectral filter 11 in the region corresponding to the pixel in the m-th column and the n-th row is represented by _Emn . At this time, each of m and n is an arbitrary integer from 1 to 3. For example, if the calibration method uses five calibration wavelengths v ₁ , v ₂ , v ₃ , v ₄ , and v ₅ , the transmittances a _mn , b _mn , c _mn , d _mn , and e _mn are obtained correspondingly. At this time, the transmittance data E _mn includes the values of (a _mn , b _mn , c _mn , d _mn , e _mn ) as vector data. As described above, the wavelength dependence of transmittance is obtained based on multiple calibration wavelengths. The transmittance data E _mn is used as a calibration value for the image sensor 10 . Here, calibration means obtaining values of transmittance (a _mn , b _mn , _cmn , d _mn , e _mn ) for each calibration wavelength in advance of using the imaging device 10. .

The received light intensity _Somn corresponding to the output value of the pixel in the m-th column and the n-th row when the light L is incident on the image sensor 12 is the intensity of the light L incident on the spectral filter 11 with respect to the five calibration wavelengths. P(v ₁ ), P(v ₂ ), P(v ₃ ), P(v ₄ ), and P(v ₅ ), respectively, are expressed as in Equation 1 below.

Here, for example, five lights having respective single wavelengths of calibration wavelengths v ₁ , v ₂ , v ₃ , v ₄ , and v ₅ , and having known light intensities are sent to the imaging element 10 individually. to irradiate. Let P ₀ (v ₁ ), P ₀ (v ₂ ), P ₀ (v ₃ ), P ₀ (v ₄ ), and P ₀ (v ₅ ) be the intensities of these five lights incident on the spectral filter 11 . do. At this time, received light intensities So(v ₁ ) _mn , So(v ₂ ) _mn , So Using (v ₃ ) _mn , So(v ₄ ) _mn and So(v ₅ ) _mn the transmittance data E _mn are obtained as follows.

The above is an example of the calibration method included in the imaging method. The control unit 20 stores the transmittance data _Emn acquired for each block by the calibration method in the storage unit 30 .

An example of the calculation method included in the imaging method will be mainly described.

Here, the calculation means that, for example, when the imaging device 1 equipped with the imaging device 10 performs imaging, five calibration wavelengths of the light L incident on the spectral filter 11 are calculated based on the transmittance data E _mn as calibration values. This means that the control unit 20 calculates the intensities P(v ₁ ), P(v ₂ ), P(v ₃ ), P(v ₄ ), and P(v ₅ ) with respect to . For example, the control unit 20 uses the transmittance data _Emn stored in the storage unit 30 by the calibration method described above as a calibration value when the imaging device 10 is used, thereby obtaining a spectral filter corresponding to each block. The spectral spectrum of the light L is calculated in the 11 regions. For example, the control unit 20 calculates the spectral spectrum of the light L incident on the spectral filter 11 based on the transmittance data _Emn acquired from the storage unit 30 and the imaging signal output from the image sensor 12 .

Nine grid-shaped pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 arranged in three columns and three rows are used as one block of the image sensor 12. is incident on each of nine pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9. Therefore, the received light intensity So _mn when the light L is incident on the image sensor 12 and the intensity P(v ₁ ), P(v ₂ ), P(v ₃ ) of the light L incident on the spectral filter 11 for the five calibration wavelengths ), P(v ₄ ), and P(v ₅ ) are related by nine equations based on Equation 1, as in Equation 3 below.

Here, the received light intensity _Somn is known since it can be converted from the output value of the electrical signal output from the corresponding pixel. Similarly, the transmittances a _mn , b _mn , c _mn , d _mn , and e _mn as calibration values corresponding to each pixel are also known based on the above calibration method. Therefore, the control unit 20 simultaneously solves the nine equations of Equation 3 to obtain the intensities P(v ₁ ), P(v ₂ ), P( v ₃ ), P(v ₄ ), and P(v ₅ ) approximate solutions can be derived. The control unit 20 uses, for example, the method of least squares as the approximation method. The above is an example of the calculation method included in the imaging method.

In such a calculation method, the control section 20 does not have to use the transmittance data E _mn associated with the pixels found to interfere with the calculation based on the calibration method described above. For example, pixels that are found to interfere with computation may be pixels whose transmittances a _mn , b _mn , _cmn , d _mn , and e _mn become too small and pixels that do not work at all when the spectral filter 11 is manufactured. Including pixels, etc.

An example of a method for manufacturing the spectral filter 11 according to one embodiment will be mainly described below.

The spectral filter 11 according to one embodiment is manufactured by, for example, coating or printing at least three types of dyes having different light transmission wavelengths on an arbitrary substrate including a glass substrate and an organic film substrate. The spectral filter 11 according to one embodiment is manufactured such that one region and another region overlap each other among a plurality of regions on the same plane of the spectral filter 11 in which dyes are formed. For example, each region of the manufactured spectral filter 11 is randomly arranged unlike the regular arrangement of pixels of the image sensor 12 .

More specifically, the method for manufacturing the spectral filter 11 includes, for example, spraying a liquid containing at least three kinds of dyes having different light transmission wavelengths onto a substrate using a coating device such as a spray coater having a plurality of nozzles. It includes a step of spraying in a shape. When the coating conditions are adjusted and the solvent is dried, each dye on the substrate has a separated structure in the form of particles or thin films. Each pigment separated into such particles or films has a random arrangement because it is sprayed in the form of a mist.

In the method for manufacturing the spectral filter 11, instead of or in addition to coating by the coating device as described above, at least three types of light having different transmission wavelengths are coated by a printing device such as an inkjet printer having a plurality of heads. A step of printing pigmented ink may be included. The patterns to be printed may be arranged so as not to overlap as much as possible for each pigment.

According to the imaging device 1 according to the embodiment as described above, the manufacturing cost can be reduced even if the types of dyes are increased. More specifically, unlike conventional spectral filters that have the same dye array as the pixels, which complicates the manufacturing process and causes high costs, the spectral filter 11 according to one embodiment is formed with dyes. Overlap of one region and another region is allowed. For example, in the spectral filter 11 according to one embodiment, the pigments are formed in regions that are randomly arranged independently of the pixel columns of the image sensor 12 . Therefore, as described above, the manufacturing method of the spectral filter 11 is simplified and the manufacturing cost is reduced. That is, the spectral filter 11 according to one embodiment can be manufactured more easily than conventional spectral filters.

As in the imaging method according to the embodiment described above, the control unit 20 determines the spectral spectrum of the light L incident on the spectral filter 11 based on the transmittance data _Emn and the imaging signal output from the image sensor 12. calculate. As a result, even when the imaging device 1 uses the spectral filter 11 according to the embodiment manufactured by the above-described simplified manufacturing method, the control unit 20 can accurately calculate the spectral spectrum of the light L. It is possible.

It will be apparent to those skilled in the art that the present disclosure can be embodied in certain other forms than those described above without departing from the spirit or essential characteristics thereof. Accordingly, the preceding description is exemplary, and not limiting. The scope of the disclosure is defined by the appended claims rather than by the foregoing description. Any changes that come within the range of equivalence are included therein.

For example, the shape, arrangement, orientation, number, and the like of each component described above are not limited to the contents illustrated in the above description and drawings. The shape, arrangement, orientation, number, and the like of each component may be arbitrarily configured as long as the function can be realized.

For example, each step and functions included in each step in the imaging method using the imaging device 1 described above can be rearranged so as not to be logically inconsistent, the order of the steps can be changed, or a plurality of steps can be arranged. It can be combined together or split.

For example, the present disclosure can be implemented as a program describing the processing content for realizing each function of the imaging apparatus 1 described above or a storage medium recording the program. It should be understood that the scope of the present disclosure includes these as well.

In the above description, as an example, nine pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9 arranged in a grid of three columns and three rows are used as one block of the image sensor 12 for spectroscopy. The imaging method for performing the above has been described. Without being limited to this, one block of the image sensor 12 may include an arbitrary number of pixels in a grid arranged in an arbitrary number of vertical columns and an arbitrary number of horizontal columns. As the number of pixels included in one block increases, the spectral accuracy and SNR (Signal-to-Noise Ratio) improve. On the other hand, since the light L is adjusted to have a uniform light intensity within the region of the spectral filter 11 corresponding to the same block, the spatial resolution improves as the number of pixels included in one block decreases. .

Although a plurality of pixels included in one block of the image sensor 12 has been described as being arranged in a grid pattern, the present invention is not limited to this. A plurality of pixels included in one block of the image sensor 12 may be arranged in any form other than a grid pattern.

In the imaging method described above, the five types of dyes C1, C2, C3, C4, and C5 are used to disperse the light L with five calibration wavelengths _v1 , _v2 , _v3 , _v4 , and _v5 . Although it is described as being possible, it is not limited to this. Each of the number of dye types and the number of calibration wavelengths may be arbitrary. The number of dye types and the number of calibration wavelengths may be the same or different. At this time, the wavelength resolution in the spectrum of the light L improves as the number of calibration wavelengths increases.

Although the imaging method described above includes the calibration method and the calculation method as a series of flows, the imaging method is not limited to this. For example, different users may perform the calibration method and the calculation method. That is, in the imaging method according to one embodiment, only the calculation method may be executed based on the transmittance data E _mn obtained by the calibration methods executed by different users. For example, the calibration method may be executed when the imaging device 1 is manufactured, and the imaging device 1 may be shipped with the transmittance data _Emn stored in the storage unit 30 . The user may execute the calculation method using the imaging device 1 in which the transmittance data _Emn is stored in the storage unit 30 .

Although the imaging method described above includes the calibration method and the calculation method, it is not limited to this. For example, by using machine learning, when accurate spectral information such as the intensity of the light L for each wavelength is not required, the imaging method according to one embodiment does not include the calibration method as described above. good. At this time, the imaging method may include any method that allows the control unit 20 to generate image data based on the imaging signal output from the imaging element 10 instead of step S103 in FIG. For example, instead of step _S103 in FIG. A step of obtaining spectral information of L may also be included. The imaging method may include the step of generating image data based on the acquired spectral information of the light L after the step.

1 imaging device 10 imaging device 11 spectral filter 12 image sensor 20 control unit 30 storage unit 40 input unit 50 communication unit C1, C2, C3, C4, C5 dye L light P1, P2, P3, P4, P5, P6, P7, P8, P9 pixels

Claims (8)

An imaging device that acquires image data based on an imaging signal,
an imaging device that outputs the imaging signal;
a control unit that generates the image data based on the imaging signal output from the imaging device;
with
The imaging element is
a spectral filter containing at least three dyes having different light transmission wavelengths;
an image sensor that includes a plurality of pixels arranged in a grid pattern, receives light that has passed through the spectral filter, and converts the light into the imaging signal as an electrical signal;
has
The plurality of regions on the same plane of the spectral filter , in which the dye is formed, does not depend on the shape of the pixel, and includes a region with a shape different from the shape of the pixel ,
Imaging device. a storage unit that stores transmittance data in which the wavelength dependence of the transmittance when the light passes through the spectral filter is associated with each pixel of the image sensor;
The control unit
calculating a spectral spectrum of the light incident on the spectral filter based on the transmittance data acquired from the storage unit and the imaging signal output from the image sensor;
generating the image data based on the calculated spectral spectrum;
The imaging device according to claim 1 . the plurality of regions are arranged randomly without depending on the arrangement of the plurality of pixels;
The imaging device according to claim 1 or 2. A spectral filter used in an imaging element included in an imaging device that acquires image data based on an imaging signal,
containing at least three dyes having different light transmission wavelengths,
The plurality of regions on the same plane of the spectral filter in which the dye is formed have shapes different from the shape of the pixels, regardless of the shape of the pixels of the image sensor used together with the spectral filter in the imaging device. including areas ,
spectral filter. the plurality of regions are arranged randomly without depending on the arrangement of the plurality of pixels;
5. A spectral filter according to claim 4. A spectral filter including at least three types of dyes having different light transmission wavelengths, and a plurality of pixels arranged in a lattice, receives light transmitted through the spectral filter and converts it into an imaging signal as an electrical signal. An imaging method for acquiring image data based on the imaging signal output from an imaging device having an image sensor that converts,
obtaining the imaging signal from the image sensor;
Based on the transmittance data in which the wavelength dependence of the transmittance when the light passes through the spectral filter is associated with each pixel of the image sensor, and the imaging signal obtained from the image sensor, calculating a spectral spectrum of the light incident on the filter;
generating the image data based on the calculated spectral spectrum;
including
The plurality of regions on the same plane of the spectral filter , in which the dye is formed, does not depend on the shape of the pixel, and includes a region with a shape different from the shape of the pixel ,
Imaging method. pre-storing the transmittance data prior to acquiring the imaging signal;
The imaging method according to claim 6 . the plurality of regions are arranged randomly without depending on the arrangement of the plurality of pixels;
The imaging method according to claim 6 or 7.

Images