WO2021192891A1 - 信号処理方法、信号処理装置、および撮像システム - Google Patents

信号処理方法、信号処理装置、および撮像システム Download PDF

Info

Publication number
WO2021192891A1
WO2021192891A1 PCT/JP2021/008433 JP2021008433W WO2021192891A1 WO 2021192891 A1 WO2021192891 A1 WO 2021192891A1 JP 2021008433 W JP2021008433 W JP 2021008433W WO 2021192891 A1 WO2021192891 A1 WO 2021192891A1
Authority
WO
WIPO (PCT)
Prior art keywords
wavelength
sub
data
mask
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/008433
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
基樹 八子
石川　篤
久田　和也
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to CN202180018960.9A priority Critical patent/CN115211106B/zh
Priority to EP21775387.0A priority patent/EP4130693A4/en
Priority to JP2022509473A priority patent/JP7720532B2/ja
Publication of WO2021192891A1 publication Critical patent/WO2021192891A1/ja
Priority to US17/930,093 priority patent/US20220414948A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G01J2003/2826—Multispectral imaging, e.g. filter imaging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination

Definitions

  • the present disclosure relates to a signal processing method, a signal processing device, and an imaging system.
  • Hyperspectral cameras are used in various fields such as food inspection, biopsy, drug development, and mineral component analysis.
  • Patent Document 1 discloses an example of a hyperspectral imaging device using compressed sensing.
  • the imaging device includes a coding element which is an array of a plurality of optical filters having different wavelength dependences of light transmittance, an image sensor which detects light transmitted through the coding element, and a signal processing circuit.
  • a coding element is arranged on the optical path connecting the subject and the image sensor.
  • the image sensor acquires one wavelength division multiplexing image by simultaneously detecting light on which components of a plurality of wavelength bands are superimposed for each pixel.
  • the signal processing circuit utilizes the information of the spatial distribution of the spectral transmittance of the coding element and applies compressed sensing to the acquired wavelength-multiplexed image to obtain an image for each of the plurality of wavelength bands. Reconstruct the data.
  • target wavelength range image data for each of all wavelength bands included in the wavelength range of the acquired wavelength division multiplexing image (hereinafter referred to as "target wavelength range") is generated and displayed.
  • target wavelength range image data for each of all wavelength bands included in the wavelength range of the acquired wavelength division multiplexing image
  • the present disclosure provides a technique for efficiently generating an image of a required wavelength band.
  • the method according to one aspect of the present disclosure is a signal processing method executed by a computer.
  • the method is to acquire compressed image data including two-dimensional image information obtained by compressing hyperspectral information in the target wavelength region, and to obtain one or more sub-wavelength regions that are a part of the target wavelength region.
  • the present invention includes the acquisition of setting data for designating the above, and the generation of a plurality of two-dimensional images corresponding to the plurality of wavelength bands included in the one or more sub-wavelength regions based on the compressed image data.
  • the present disclosure may be implemented in recording media such as systems, devices, methods, integrated circuits, computer programs or computer readable recording disks, systems, devices, methods, integrated circuits, etc. It may be realized by any combination of a computer program and a recording medium.
  • the computer-readable recording medium may include a non-volatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory).
  • the device may consist of one or more devices. When the device is composed of two or more devices, the two or more devices may be arranged in one device, or may be separately arranged in two or more separated devices.
  • "device" can mean not only one device, but also a system of multiple devices.
  • FIG. 1A is a diagram schematically showing an exemplary hyperspectral imaging system.
  • FIG. 1B is a diagram schematically showing a first modification of an exemplary hyperspectral imaging system.
  • FIG. 1C is a diagram schematically showing a second modification of an exemplary hyperspectral imaging system.
  • FIG. 1D is a diagram schematically showing a third modification of an exemplary hyperspectral imaging system.
  • FIG. 2A is a diagram schematically showing an example of a filter array.
  • FIG. 2B is a diagram showing an example of the spatial distribution of the light transmittance of each of the plurality of wavelength bands W 1 , W 2 , ..., W N included in the target wavelength region.
  • FIG. 1A is a diagram schematically showing an exemplary hyperspectral imaging system.
  • FIG. 1B is a diagram schematically showing a first modification of an exemplary hyperspectral imaging system.
  • FIG. 1C is a diagram schematically showing a second modification of an exemplary hyperspectral imaging system.
  • FIG. 2C is a diagram showing an example of the spectral transmittance of the region A1 included in the filter array shown in FIG. 2A.
  • FIG. 2D is a diagram showing an example of the spectral transmittance of the region A2 included in the filter array shown in FIG. 2A.
  • FIG. 3A is a diagram showing an example of the relationship between the target wavelength region W and the plurality of wavelength bands W 1 , W 2 , ..., W N included in the target wavelength region W.
  • FIG. 3B is a diagram showing another example of the relationship between the target wavelength region W and the plurality of wavelength bands W 1 , W 2 , ..., W N included therein.
  • FIG. 4A is a diagram for explaining the characteristics of spectral transmittance in a certain region of the filter array.
  • FIG. 4B is a diagram showing the results of averaging the spectral transmittances shown in FIG. 4A for each of the wavelength bands W 1 , W 2 , ..., W N.
  • FIG. 5 is a diagram schematically showing a usage scene of the hyperspectral camera.
  • FIG. 6A is a diagram showing an example of a target wavelength region W and a designated sub-wavelength region Wa.
  • FIG. 6B is a diagram showing an example in which a second sub-wavelength region is designated in addition to the first sub-wavelength region.
  • FIG. 7 is a diagram showing a configuration of an imaging system according to an exemplary embodiment of the present disclosure.
  • FIG. 8 is a flowchart showing the operation of the system.
  • FIG. 9 is a diagram showing an example of mask data before conversion stored in the memory.
  • FIG. 10 is a diagram showing an example of a GUI for inputting imaging conditions.
  • FIG. 11 is a diagram showing an example of a GUI for inputting restoration conditions.
  • FIG. 12 is a diagram showing an example of a GUI for inputting restoration conditions.
  • FIG. 13 is a diagram showing an example of a screen displaying a spectroscopic image generated as a result of the restoration operation.
  • FIG. 14 is a diagram for explaining an example of a method of synthesizing mask information of a plurality of bands and converting it into new mask information.
  • FIG. 15A is a diagram showing an example of the converted mask data recorded in the memory.
  • FIG. 15B is a diagram showing another example of the converted mask data recorded in the memory.
  • FIG. 16 is a diagram showing an example of a method of generating an image for each of a plurality of wavelength bands included in a target wavelength region.
  • FIG. 17 is a diagram showing a system configuration when the signal processing circuit does not convert mask information.
  • FIG. 18 is a diagram showing another example of GUI for setting restoration conditions.
  • FIG. 19 is a diagram showing an example of displaying an image in a wavelength range not specified.
  • FIG. 20 is a diagram showing an example of a method of restoring only a specific sub-wavelength region with high wavelength resolution by performing two-step restoration.
  • all or part of a circuit, unit, device, member or part, or all or part of a functional block in a block diagram is, for example, a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (range scale integration). ) Can be performed by one or more electronic circuits.
  • the LSI or IC may be integrated on one chip, or may be configured by combining a plurality of chips.
  • functional blocks other than the storage element may be integrated on one chip.
  • it is called LSI or IC, but the name changes depending on the degree of integration, and it may be called system LSI, VLSI (very large scale integration), or ULSI (ultra large scale integration).
  • Field Programmable Gate Array (FPGA) which is programmed after the LSI is manufactured, or reconfigurable logistic device, which can reconfigure the junction relationship inside the LSI or set up the circuit partition inside the LSI, can also be used for the same purpose.
  • FPGA Field Programmable Gate Array
  • circuits, units, devices, members or parts can be executed by software processing.
  • the software is recorded on a non-temporary recording medium such as one or more ROMs, optical discs, hard disk drives, etc., and when the software is executed by a processor, the functions identified by the software. Is executed by the processing device and peripheral devices.
  • the system or device may include one or more non-temporary recording media, processing devices, and required hardware devices, such as interfaces, on which the software is recorded.
  • FIG. 1A is a diagram schematically showing an exemplary hyperspectral imaging system.
  • This system includes an imaging device 100 and a processing device 200.
  • the image pickup apparatus 100 has the same configuration as the image pickup apparatus disclosed in Patent Document 1.
  • the image pickup apparatus 100 includes an optical system 140, a filter array 110, and an image sensor 160.
  • the filter array 110 has the same structure and function as the "encoding element" disclosed in Patent Document 1. Therefore, in the following description, the filter array 110 may be referred to as a "coding element".
  • the optical system 140 and the filter array 110 are arranged on the optical path of the light incident from the object 70 which is the subject.
  • the filter array 110 is arranged between the optical system 140 and the image sensor 160.
  • an apple is illustrated as an example of the object 70.
  • the object 70 is not limited to an apple, but can be any object.
  • the processing device 200 generates image data for each of the plurality of wavelength bands included in the target wavelength region based on the image data generated by the image sensor 160.
  • This image data is referred to as "spectroscopic image data" in the present specification.
  • the number of wavelength bands included in the target wavelength region is N (N is an integer of 4 or more).
  • the generated spectroscopic image data of a plurality of wavelength bands are referred to as spectroscopic images 220W 1 , 220W 2 , ..., 220W N, and these are collectively referred to as spectroscopic images 220.
  • a data or signal indicating an image that is, a set of data or signals representing a pixel value of each pixel may be simply referred to as an "image”.
  • the filter array 110 is an array of a plurality of translucent filters arranged in rows and columns.
  • the plurality of filters include a plurality of types of filters in which the spectral transmittance, that is, the wavelength dependence of the light transmittance is different from each other.
  • the filter array 110 modulates the intensity of the incident light for each wavelength and outputs it. This process by the filter array 110 is referred to herein as "coding".
  • the filter array 110 is arranged near or directly above the image sensor 160.
  • the “neighborhood” means that the image of the light from the optical system 140 is close enough to be formed on the surface of the filter array 110 in a state of being clear to some extent. "Directly above” means that they are so close that there is almost no gap.
  • the filter array 110 and the image sensor 160 may be integrated.
  • the optical system 140 includes at least one lens. Although the optical system 140 is shown as one lens in FIG. 1A, the optical system 140 may be a combination of a plurality of lenses. The optical system 140 forms an image on the image pickup surface of the image sensor 160 via the filter array 10.
  • the filter array 110 may be arranged away from the image sensor 160.
  • 1B to 1D are diagrams showing a configuration example of an image pickup apparatus 100 in which the filter array 110 is arranged away from the image sensor 160.
  • the filter array 110 is arranged between the optical system 140 and the image sensor 160 and at a position away from the image sensor 160.
  • the filter array 110 is arranged between the object 70 and the optical system 140.
  • the image pickup apparatus 100 includes two optical systems 140A and 140B, and the filter array 110 is arranged between them.
  • an optical system including one or more lenses may be arranged between the filter array 110 and the image sensor 160.
  • the image sensor 160 is a monochrome type photodetector having a plurality of two-dimensionally arranged photodetectors (also referred to as "pixels" in the present specification).
  • the image sensor 160 may be, for example, a CCD (Charge-Coupled Device), a CMOS (Complementary Metal Oxide Sensor) sensor, an infrared array sensor, a terahertz array sensor, or a millimeter wave array sensor.
  • the photodetector includes, for example, a photodiode.
  • the image sensor 160 does not necessarily have to be a monochrome type sensor.
  • a color type sensor having an R / G / B, R / G / B / IR, or R / G / B / W filter may be used.
  • the wavelength range to be acquired may be arbitrarily determined, and is not limited to the visible wavelength range, but may be the wavelength range of ultraviolet, near infrared, mid-infrared, far infrared, microwave / radio waves.
  • the processing device 200 is a computer including a processor and a storage medium such as a memory.
  • the processing device 200 generates data of a plurality of spectroscopic images 220W 1 , 220W 2 , ... 220W N , each including information of a plurality of wavelength bands, based on the image 120 acquired by the image sensor 160.
  • FIG. 2A is a diagram schematically showing an example of the filter array 110.
  • the filter array 110 has a plurality of regions arranged two-dimensionally. In the present specification, the area may be referred to as a "cell". In each region, an optical filter having individually set spectral transmittance is arranged.
  • the spectral transmittance is represented by a function T ( ⁇ ), where ⁇ is the wavelength of the incident light.
  • the spectral transmittance T ( ⁇ ) can take a value of 0 or more and 1 or less.
  • the filter array 110 has 48 rectangular regions arranged in 6 rows and 8 columns. This is just an example, and in actual use, more areas may be provided. The number may be, for example, about the same as the number of pixels of the image sensor 160. The number of filters included in the filter array 110 is determined according to the application, for example, in the range of tens to tens of millions.
  • FIG. 2B is a diagram showing an example of the spatial distribution of the light transmittance of each of the plurality of wavelength bands W 1 , W 2 , ..., W N included in the target wavelength region.
  • the difference in shade of each region represents the difference in transmittance. The lighter the region, the higher the transmittance, and the darker the region, the lower the transmittance.
  • the spatial distribution of light transmittance differs depending on the wavelength band.
  • 2C and 2D are diagrams showing examples of spectral transmittances of regions A1 and A2 included in the filter array 110 shown in FIG. 2A, respectively.
  • the spectral transmittance of the region A1 and the spectral transmittance of the region A2 are different from each other.
  • the spectral transmittance of the filter array 110 differs depending on the region. However, the spectral transmittances of all regions do not necessarily have to be different.
  • the spectral transmittances of at least a part of the plurality of regions are different from each other.
  • the filter array 110 includes two or more filters having different spectral transmittances.
  • the number of spectral transmittance patterns in the plurality of regions included in the filter array 110 may be equal to or greater than the number N of wavelength bands included in the target wavelength region.
  • the filter array 110 may be designed so that the spectral transmittances of more than half of the regions are different.
  • the target wavelength range W can be set in various ranges depending on the application.
  • the target wavelength range W can be, for example, a visible light wavelength range of about 400 nm to about 700 nm, a near infrared wavelength range of about 700 nm to about 2500 nm, or a near ultraviolet wavelength range of about 10 nm to about 400 nm.
  • the target wavelength region W may be a radio wave region such as mid-infrared, far-infrared, terahertz wave, or millimeter wave.
  • the wavelength range used is not always the visible light range. In the present specification, not only visible light but also invisible light such as near-ultraviolet rays, near-infrared rays, and radio waves are referred to as "light" for convenience.
  • N is an arbitrary integer of 4 or more, and each wavelength range obtained by dividing the target wavelength range W into N equal parts is the wavelength bands W 1 , W 2 , ..., W N.
  • a plurality of wavelength bands included in the target wavelength region W may be arbitrarily set.
  • the bandwidth may be non-uniform depending on the wavelength band.
  • the bandwidth differs depending on the wavelength band, and there is a gap between two adjacent wavelength bands.
  • the plurality of wavelength bands need only be different from each other, and the method of determining the wavelength bands is arbitrary.
  • FIG. 4A is a diagram for explaining the characteristics of spectral transmittance in a certain region of the filter array 110.
  • the spectral transmittance has a plurality of maximum values P1 to P5 and a plurality of minimum values with respect to wavelengths in the target wavelength region W.
  • the maximum value of the light transmittance in the target wavelength region W is 1 and the minimum value is 0.
  • the spectral transmittance has a maximum value in a wavelength region such as the wavelength band W 2 and the wavelength band W N-1.
  • the maximum values P1, P3, P4 and P5 are 0.5 or more.
  • the filter array 110 transmits a large amount of components in a certain wavelength range from the incident light, and does not transmit so much components in another wavelength range. For example, light in k wavelength bands out of N wavelength bands has a transmittance of more than 0.5, and light in the remaining NK wavelength ranges has a transmittance of 0.5. Can be less than. k is an integer that satisfies 2 ⁇ k ⁇ N. If the incident light is white light that evenly contains the wavelength components of all visible light, the filter array 110 sets the incident light to light having a plurality of discrete intensity peaks with respect to the wavelength for each region. It is modulated to and these multi-wavelength light is superimposed and output.
  • FIG. 4B is a diagram showing the results of averaging the spectral transmittances shown in FIG. 4A for each wavelength band W 1 , W 2 , ..., W N.
  • the averaged transmittance is obtained by integrating the spectral transmittance T ( ⁇ ) for each wavelength band and dividing by the bandwidth of that wavelength band.
  • the value of the transmittance averaged for each wavelength band as described above is defined as the transmittance in that wavelength band.
  • the transmittance is remarkably high in the three wavelength regions having maximum values P1, P3 and P5. In particular, the transmittance exceeds 0.8 in the two wavelength regions having maximum values P3 and P5.
  • each region transmits any value of 0 or more and 1 or less.
  • a binary-scale transmittance distribution may be adopted in which the transmittance of each region can take a value of either approximately 0 or approximately 1.
  • each region transmits most of the light in at least two wavelength regions of the plurality of wavelength regions included in the target wavelength region, and transmits most of the light in the remaining wavelength regions. I won't let you.
  • "most" refers to approximately 80% or more.
  • a part of all cells may be replaced with a transparent area.
  • a transparent region transmits light of all wavelength bands W 1 to W N included in the target wavelength region W with a transmittance of the same high level, for example, a transmittance of 80% or more.
  • the plurality of transparent areas may be arranged, for example, in a checkerboard pattern. That is, in the two arrangement directions of the plurality of regions in the filter array 110, regions having different light transmittances depending on the wavelength and transparent regions can be arranged alternately.
  • Data showing the spatial distribution of the spectral transmittance of such a filter array 110 is acquired in advance by design data or actual measurement calibration, and is stored in a storage medium included in the processing device 200. This data is used for arithmetic processing described later.
  • the filter array 110 can be constructed using, for example, a multilayer film, an organic material, a diffraction grating structure, or a fine structure containing a metal.
  • a multilayer film for example, a dielectric multilayer film or a multilayer film including a metal layer can be used.
  • each cell is formed so that at least one of the thickness, material, and stacking order of each multilayer film is different.
  • different spectral characteristics can be realized depending on the cell.
  • Compositions using organic materials can be realized by making the pigments or dyes contained different depending on the cell, or by laminating different materials.
  • a configuration using a diffraction grating structure can be realized by providing a diffraction structure having a different diffraction pitch or depth for each cell.
  • a fine structure containing a metal When a fine structure containing a metal is used, it can be produced by utilizing spectroscopy by the plasmon effect.
  • the processing device 200 reconstructs the multi-wavelength spectroscopic image 220 based on the image 120 output from the image sensor 160 and the spatial distribution characteristic of the transmittance for each wavelength of the filter array 110.
  • the term "multi-wavelength" means, for example, a wavelength range larger than the three-color wavelength range of RGB acquired by a normal color camera.
  • the number of this wavelength range can be, for example, about 4 to 100.
  • the number of wavelength regions is referred to as the number of bands. Depending on the application, the number of bands may exceed 100.
  • the data to be obtained is the data of the spectroscopic image 220, and let that data be f.
  • f is data obtained by integrating the image data f 1 , f 2 , ..., And f N of each band.
  • the horizontal direction of the image is the x direction
  • the vertical direction of the image is the y direction.
  • the number of pixels in the x direction of the image data to be obtained is n and the number of pixels in the y direction is m
  • each of the image data f 1 , f 2 , ..., F N is two-dimensional data of n ⁇ m pixels. Is.
  • the data f is three-dimensional data having the number of elements n ⁇ m ⁇ N.
  • This three-dimensional data is referred to as a "hyperspectral data cube" or a "hyperspectral cube”.
  • the number of elements of the data g of the image 120 obtained by coding and multiplexing by the filter array 110 is n ⁇ m.
  • the data g can be represented by the following equation (1).
  • each of f 1 , f 2 , ..., And f N is data having n ⁇ m elements. Therefore, strictly speaking, the vector on the right side is a one-dimensional vector of n ⁇ m ⁇ N rows and 1 column.
  • the vector g is converted into a one-dimensional vector of n ⁇ m rows and one column, represented, and calculated.
  • the matrix H encodes and intensity-modulates each component f 1 , f 2 , ..., F N of the vector f with different coding information (hereinafter, also referred to as “mask information”) for each wavelength band, and they are used. Represents a transformation that adds. Therefore, H is a matrix of n ⁇ m rows and n ⁇ m ⁇ N columns. In the present specification, the matrix H may be referred to as a "system matrix".
  • the processing device 200 uses the redundancy of the image included in the data f to obtain a solution by using a compressed sensing method. Specifically, the data f to be obtained is estimated by solving the following equation (2).
  • f' represents the estimated data of f.
  • the first term in parentheses in the above equation represents the amount of deviation between the estimation result Hf and the acquired data g, the so-called residual term.
  • the sum of squares is used as the residual term, but the absolute value, the square root of the sum of squares, or the like may be used as the residual term.
  • the second term in parentheses is a regularization term or a stabilization term. Equation (2) means finding f that minimizes the sum of the first term and the second term.
  • the processing device 200 can converge the solution by a recursive iterative operation and calculate the final solution f'.
  • the first term in parentheses in equation (2) means an operation for finding the sum of squares of the difference between the acquired data g and Hf obtained by transforming f in the estimation process by the matrix H.
  • the second term ⁇ (f) is a constraint condition in the regularization of f, and is a function that reflects the sparse information of the estimated data. This function has the effect of smoothing or stabilizing the estimated data.
  • the regularization term can be represented by, for example, the Discrete Cosine Transform (DCT), Wavelet Transform, Fourier Transform, or Total Variation (TV) of f. For example, when the total variation is used, stable guess data that suppresses the influence of noise in the observation data g can be obtained.
  • the sparsity of the object 70 in the space of each regularization term depends on the texture of the object 70. You may choose a regularization term that makes the texture of the object 70 more sparse in the space of the regularization term. Alternatively, a plurality of regularization terms may be included in the operation.
  • ⁇ is a weighting factor. The larger the weighting coefficient ⁇ , the larger the amount of redundant data to be reduced, and the higher the compression rate. The smaller the weighting coefficient ⁇ , the weaker the convergence to the solution.
  • the weighting coefficient ⁇ is set to an appropriate value at which f converges to some extent and does not cause overcompression.
  • the image encoded by the filter array 110 is acquired in a blurred state on the imaging surface of the image sensor 160. Therefore, the spectroscopic image 220 can be reconstructed by holding this blur information in advance and reflecting the blur information in the system matrix H described above.
  • the blur information is represented by a point spread function (Point Spread Function: PSF).
  • PSF is a function that defines the degree of spread of the point image to the peripheral pixels. For example, when a point image corresponding to one pixel on an image spreads over a region of k ⁇ k pixels around the pixel due to blurring, PSF is a coefficient group indicating the influence on the brightness of each pixel in the region. That is, it can be defined as a matrix.
  • the spectroscopic image 220 can be reconstructed by reflecting the influence of the blurring of the coding pattern by the PSF in the system matrix H.
  • the position where the filter array 110 is arranged is arbitrary, but a position where the coding pattern of the filter array 110 is too diffused and does not disappear can be selected.
  • the hyperspectral data cube shows a two-dimensional image of all of a plurality of wavebands W 1 contained in the target wavelength range W of W N are generated.
  • images for all these wavelength bands may not be required. In such cases, it is inefficient to generate images with high wavelength resolution for all wavelength bands.
  • FIG. 5 is a diagram schematically showing a usage scene of the hyperspectral camera.
  • the user may wish to obtain color information in the red wavelength range, eg, 600 nm to 700 nm, in order to estimate the sugar content of apples.
  • the user may wish to obtain color information in the green wavelength range, eg, the wavelength range of 500 nm to 600 nm, in order to know the exact spectrum of the leaves.
  • the user may wish to acquire color information in the blue wavelength range, for example, the wavelength range of 400 nm to 500 nm, in order to know how the blue product has faded.
  • one of the following methods (1) and (2) is used in order to acquire color information in different wavelength ranges.
  • (1) Use an image sensor that can independently acquire information in each wavelength range.
  • (2) A wide range of color information is acquired using a camera capable of acquiring information in a wide wavelength range, and only the information in the required wavelength range is displayed.
  • the user can specify one or more sub-wavelength regions that are a part of the target wavelength region W based on the image data acquired by the hyperspectral imaging device.
  • the signal processor produces a hyperspectral data cube showing a plurality of two-dimensional images for a plurality of wavelength bands contained in one or more designated sub-wavelength regions. This makes it possible to acquire detailed spectral information about the sub-wavelength range desired by the user while suppressing the calculation cost.
  • FIG. 6A is a diagram showing an example of a target wavelength region W and a designated sub-wavelength region Wa.
  • Sub-wavelength range W a is a part of the target wavelength range W, includes a plurality of wavelength bands W a1, W a2, ⁇ , the W ai.
  • i represents the number of wavelengths bands included in the sub-wavelength region W a.
  • the signal processor generates a hyperspectral data cube showing two-dimensional images for each of these wavelength bands Wa1 , Wa2 , ..., Wai.
  • Figure 6B is, in addition to the first sub-wavelength region W a, shows an example in which the second sub-wavelength band W b is specified.
  • the first sub-wavelength region W a and the second sub-wavelength band W b are spaced apart, both included in the target wavelength region W.
  • the second sub-wavelength region W b includes a plurality of wavelength bands W b1 , W b2 , ..., W bj .
  • j represents the number of wavelength bands included in the second sub-wavelength region W b. In this way, a plurality of sub-wavelength regions may be specified.
  • the signal processing device generates images of a plurality of wavelength bands included in the designated sub-wavelength range based on the image data generated by the image pickup device.
  • the generated multiple wavelength band images are displayed on the display.
  • the signal processing method is executed by a computer.
  • the method is to acquire compressed image data including two-dimensional image information obtained by compressing hyperspectral information in the target wavelength region, and to obtain one or more sub-wavelength regions that are a part of the target wavelength region.
  • the present invention includes the acquisition of setting data for designating the above, and the generation of a plurality of two-dimensional images corresponding to the plurality of wavelength bands included in the one or more sub-wavelength regions based on the compressed image data.
  • “Hyperspectral information in the target wavelength range” means information of a plurality of images corresponding to a plurality of wavelength bands included in a predetermined target wavelength range.
  • “Compressing hyperspectral information” means compressing image information of a plurality of wavelength bands as one monochrome two-dimensional image using a coding element such as the filter array 120 described above, and acquiring in advance. It includes compressing image information of a plurality of wavelength bands into one monochrome two-dimensional image by software processing.
  • the above method for example, it is possible to generate data of a plurality of two-dimensional images corresponding to a plurality of wavelength bands included in one or more sub-wavelength regions specified by the user, that is, a hyperspectral data cube. Therefore, it is possible to generate only the necessary hyperspectral data cube according to the application or purpose.
  • the hyperspectral information may be information of four or more wavelength bands included in the target wavelength region, and the two-dimensional image information may be data of a plurality of pixels included in the compressed image data. good. Information on the four or more wavelength bands may be superimposed on the data of each of the plurality of pixels. In other words, the data of each pixel of the compressed image data may include one value on which information of four or more wavelength bands included in the target wavelength region is superimposed. Information on wavelength bands of 10 or more or 100 or more may be superimposed on the data of each pixel of the compressed image data, depending on the intended use.
  • the setting data may include information that specifies wavelength resolution in the one or more sub-wavelength regions.
  • the plurality of two-dimensional images can be generated with the wavelength resolution.
  • the user can specify the wavelength resolution for each sub-wavelength region in addition to the designation of the sub-wavelength region. Therefore, flexible adjustment such as increasing the wavelength resolution of the sub-wavelength region that requires detailed spectral information becomes possible.
  • the one or more sub-wavelength regions may include a first sub-wavelength region and a second sub-wavelength region.
  • the plurality of two-dimensional images can be generated for each of the first sub-wavelength region and the second sub-wavelength region.
  • the one or more sub-wavelength regions may include a first sub-wavelength region and a second sub-wavelength region.
  • the wavelength resolution can be specified independently for each of the first sub-wavelength region and the second sub-wavelength region.
  • the plurality of two-dimensional images can be generated for each of the first sub-wavelength region and the second sub-wavelength region with the corresponding wavelength resolution.
  • the first sub-wavelength region and the second sub-wavelength region may be separated from each other.
  • the first sub-wavelength region and the second sub-wavelength region may be adjacent to each other or may partially overlap each other.
  • the method may further include displaying a graphical user interface (GUI) for allowing the user to input the setting data on a display connected to the computer.
  • GUI graphical user interface
  • the method may further include displaying the plurality of two-dimensional images on a display connected to the computer. This allows the user to easily confirm the generated spectroscopic image for each wavelength band.
  • the compressed image data can be generated by imaging using a filter array including a plurality of types of optical filters having different spectral transmittances and an image sensor.
  • the method may further include acquiring mask data that reflects the spatial distribution of the spectral transmittance of the filter array.
  • the plurality of two-dimensional images can be generated based on the compressed image data and the mask data.
  • mask data including information on a plurality of mask images acquired by capturing a plurality of backgrounds corresponding to a plurality of unit bands included in the target wavelength region with the image sensor through the filter array. May further include obtaining.
  • the plurality of two-dimensional images can be generated based on the compressed image data and the mask data.
  • the mask data can be, for example, data that defines the matrix H in the above equation (2).
  • the format of the mask data may differ depending on the configuration of the imaging system.
  • the mask data may indicate the spatial distribution of the spectral transmittance of the filter array, or may include information for calculating the spatial distribution of the spectral transmittance of the filter array.
  • the mask data may include background image information for each unit band in addition to the mask image information described above. By dividing the mask image by the background image for each pixel, information on the transmittance distribution can be obtained for each unit band.
  • the mask data may include only the information of the mask image.
  • the mask image shows the distribution of values obtained by multiplying the transmittance of the filter array by the sensitivity of the image sensor. Such mask data can be used in configurations where the filter array is placed close to and facing the image sensor.
  • the mask data may include mask information.
  • the mask information may indicate the spatial distribution of the transmittance of the filter array in each of the plurality of unit bands included in the target wavelength region.
  • the method comprises synthesizing a portion of the mask information corresponding to a plurality of unit bands included in a non-designated wavelength region other than the one or more sub wavelength regions in the target wavelength region. It may further include generating composite mask information and generating a composite image corresponding to the non-designated wavelength region based on the compressed image data and the composite mask information.
  • the method is to generate a composite mask image obtained by synthesizing the mask images of a plurality of unit bands included in a non-designated wavelength region other than the designated one or more sub-wavelength regions in the target wavelength region. And, based on the compressed image data and the composite mask image, generating composite image data for the non-designated wavelength region may further be included.
  • a detailed spectroscopic image is not generated for the non-designated wavelength region, and a detailed spectroscopic image is generated only for the designated sub-wavelength region. Therefore, the calculation time required to generate the spectroscopic image can be shortened.
  • the mask data may further include information on a plurality of background images acquired by capturing the plurality of backgrounds with the image sensor without passing through the filter array.
  • the method may further include generating a composite background image by synthesizing the plurality of background images.
  • the composite image can be generated based on the compressed image data, the composite mask image, and the composite background image.
  • the mask data may include a plurality of background images and a plurality of mask images.
  • Each of the plurality of background images is acquired, for example, by capturing a corresponding background among the plurality of backgrounds with the image sensor without passing through the filter array.
  • Each of the plurality of mask images is acquired, for example, by capturing the corresponding background of the plurality of backgrounds with the image sensor through the filter array.
  • the composite mask information can be generated based on the plurality of mask images and the plurality of background images.
  • the method may further include displaying the composite image on a display connected to the computer. This allows the user to easily see a rough image of an unspecified wavelength range.
  • the mask data may include mask information.
  • the mask information may indicate the spatial distribution of the transmittance of the filter array in each of the plurality of unit bands included in the target wavelength region.
  • the setting data provides information for designating a plurality of large sub-wavelength regions, each of which is a part of the target wavelength region, and a plurality of small sub-wavelength regions included in at least one of the plurality of large sub-wavelength regions.
  • the method is the first by synthesizing a part of the mask information corresponding to a plurality of unit bands included in each of the plurality of large sub-wavelength regions for each of the plurality of large sub-wavelength regions.
  • the plurality of two-dimensional images can be generated corresponding to the plurality of small sub-wavelength regions.
  • the method is to generate a second synthetic mask information obtained by synthesizing the mask information for a plurality of unit bands included in the small sub-wavelength region for each of the plurality of small sub-wavelength regions. Based on the first composite image for at least one of the plurality of large sub-wavelength regions and the second composite mask information, a second composite image is generated for each of the small sub-wavelength regions. It may also be included.
  • the target wavelength range may include a visible wavelength range.
  • the method is to generate an image corresponding to a red wavelength region, an image corresponding to a green wavelength region, and an image corresponding to a blue wavelength region based on the compressed image data and the synthetic mask information. Displaying an image corresponding to the red wavelength region, an image corresponding to the green wavelength region, and an RGB image based on the image corresponding to the blue wavelength region on a display connected to the computer. It may also be included. As a result, the user can confirm the RGB image of the object separately from the detailed spectroscopic image of the designated sub-wavelength range.
  • the method according to still another embodiment of the present disclosure is a method of generating mask data.
  • the mask data is used to restore spectroscopic image data for each wavelength band from compressed image data acquired by an imaging apparatus including a filter array including a plurality of types of optical filters having different spectral transmittances. That is, a method of generating mask data used for restoring spectral image data for each wavelength band from compressed image data acquired by an imaging device including a filter array including a plurality of types of optical filters having different spectral transmittances. Is.
  • the method involves acquiring first mask data for reconstructing a first spectral image corresponding to a first wavelength band group in a target wavelength range, and one or more sub-wavelengths that are part of the target wavelength range. Acquiring the setting data for designating the region, and restoring the second spectral image data corresponding to the second wavelength band group in the one or more sub-wavelength regions based on the first mask data and the setting data.
  • the first wavelength band group can be a set of all or a part of wavelength bands included in the target wavelength range.
  • the second wavelength band group can be a set of all or a part of wavelength bands included in the sub-wavelength region.
  • Each of the first wavelength band group and the second wavelength band group may be an aggregate of synthetic bands in which two or more unit wavelength bands are combined.
  • mask data conversion processing is performed according to the band synthesis mode.
  • the setting data may include information regarding a band synthesis mode used in the mask data conversion process.
  • the first mask data and the second mask data may be data that reflect the spatial distribution of the spectral transmittance of the filter array.
  • the first mask data may include first mask information indicating the spatial distribution of the spectral transmittance corresponding to the first wavelength band group.
  • the second mask data may include second mask information indicating the spatial distribution of the spectral transmittance corresponding to the second wavelength band group.
  • the second mask data may further include the third mask information obtained by synthesizing a plurality of pieces of information.
  • Each of the plurality of pieces of information indicates the spatial distribution of the spectral transmittance in the corresponding wavelength band included in the non-designated wavelength region other than the one or more sub-wavelength regions in the target wavelength region.
  • the signal processing device includes a processor and a memory for storing a computer program executed by the processor.
  • the computer program acquires compressed image data including two-dimensional image information obtained by compressing hyperspectral information in the target wavelength region to the processor, and is a part of the target wavelength region. Acquiring the setting data for designating the above sub-wavelength regions and generating a plurality of two-dimensional images corresponding to a plurality of wavelength bands included in the one or more sub-wavelength regions based on the compressed image data. And to execute.
  • the signal processing device includes a processor and a memory for storing a computer program executed by the processor.
  • the computer program acquires the first mask data for restoring the first spectral image data corresponding to the first wavelength band group in the target wavelength region to the processor, and is a part of the target wavelength region. Acquiring the setting data for designating one or more sub-wavelength regions, and based on the first mask data and the setting data, a second corresponding to the second wavelength band group in the one or more sub-wavelength regions. Restoring the second mask data for generating the spectral image data and performing.
  • the image pickup system includes the signal processing device and the image pickup device for generating the compressed image data.
  • a computer program is to obtain compressed image data including two-dimensional image information obtained by compressing hyperspectral information in a target wavelength range to a computer, and to obtain compressed image data in the target wavelength range.
  • a computer program is a first wavelength in a target wavelength range from compressed image data acquired by a computer including a filter array including a plurality of types of optical filters having different spectral transmission rates. Acquiring the first mask data for restoring the first spectral image data corresponding to the band group, and acquiring the setting data for designating one or more sub-wavelength regions that are a part of the target wavelength range. And, based on the first mask data and the setting data, the second mask data for restoring the second spectroscopic image data corresponding to the second wavelength band group in the one or more sub wavelength regions is generated. And to execute.
  • a computer-readable non-temporary storage medium presents a computer with compressed image data including two-dimensional image information obtained by compressing hyperspectral information in a wavelength range of interest. Acquiring, acquiring setting data for designating one or more sub-wavelength regions that are a part of the target wavelength region, and being included in the one or more sub-wavelength regions based on the compressed image data. It stores a program for generating a plurality of two-dimensional images corresponding to a plurality of wavelength bands and executing a process including.
  • a non-transient storage medium readable by a computer is a first mask for restoring to a computer the first spectral image data corresponding to the first wavelength band group in the target wavelength range. Based on the acquisition of data, the acquisition of setting data for designating one or more sub-wave frequencies that are a part of the target wavelength range, and the first mask data and the setting data, the above one A program for generating a second mask data for restoring the second spectral image data corresponding to the second wavelength band group in the above sub-wavelength region and executing a process including the above-mentioned sub-wavelength band group is stored.
  • FIG. 7 is a diagram showing a configuration of an imaging system according to an exemplary embodiment of the present disclosure.
  • This system includes an image pickup device 100, a processing device 200, a display device 300, and an input user interface (UI) 400.
  • the processing device 200 corresponds to the signal processing device in the present disclosure.
  • the image pickup device 100 includes an image sensor 160 and a control circuit 150 that controls the image sensor 160.
  • the imaging apparatus 100 also includes a filter array 110 and at least one optical system 140, as shown in FIGS. 1A-1D.
  • the arrangement of the filter array 110 and the optical system 140 may be any of the arrangements shown in FIGS. 1A to 1D.
  • the image sensor 160 acquires a black-and-white image based on light whose intensity is modulated for each region by the filter array 110.
  • Information on a plurality of wavelength bands within the target wavelength region W is superimposed on the data of each pixel of the monochrome image. Therefore, it can be said that this monochrome image is a two-dimensional image in which the hyperspectral information in the target wavelength region W is compressed.
  • Such a black-and-white image is an example of a "compressed image" in the present specification. Further, in the present specification, data indicating a compressed image is referred to as "compressed image data”.
  • the processing device 200 includes a signal processing circuit 250 and a memory 210 such as a RAM and a ROM.
  • the signal processing circuit 250 may be an integrated circuit including a processor such as a CPU or GPU.
  • the signal processing circuit 250 performs restoration processing based on the compressed image data output from the image sensor 160. This restoration process is basically the same as the processing performed by the processing devices 200 shown in FIGS. 1A to 1D, but in the present embodiment, the restoration process is performed according to the restoration conditions input from the input UI 400.
  • the signal processing circuit 250 generates image data with high wavelength resolution only in a designated sub-wavelength region in the target wavelength region. As a result, the calculation time can be shortened.
  • the memory 210 stores a computer program executed by the processor included in the signal processing circuit 250, various data referenced by the signal processing circuit 250, and various data generated by the signal processing circuit 250.
  • the display device 300 includes a memory 310, an image processing circuit 320, and a display 330.
  • the memory 310 temporarily stores the setting data indicating the restoration condition sent from the input UI 400.
  • the image processing circuit 320 performs necessary processing on the image restored by the signal processing circuit 250 and then displays the image on the display 330.
  • the display 330 can be any display, such as a liquid crystal or organic LED.
  • the input UI 400 includes hardware and software for setting various conditions such as imaging conditions and restoration conditions.
  • Imaging conditions can include, for example, conditions such as resolution, gain, and exposure time.
  • Restoration conditions may include, for example, the lower and upper wavelengths of each sub-wavelength region, the number of wavelength bands included in each sub-wavelength region, and the number of calculations.
  • the input imaging conditions are sent to the control circuit 150 of the imaging device 100.
  • the control circuit 150 causes the image sensor 160 to perform imaging according to the imaging conditions.
  • the image sensor 160 generates a compressed image in which information of a plurality of wavelength bands in the target wavelength region W is superimposed.
  • the input restoration condition is sent to the signal processing circuit 250 and the memory 310 and recorded.
  • the signal processing circuit 250 performs restoration processing according to the set restoration conditions and generates a hyperspectral data cube for the specified sub-wavelength region.
  • the image processing circuit 320 causes the display 330 to display an image for each of the plurality of wavelength bands in the designated sub-wavelength region according to the set restoration conditions.
  • the signal processing circuit 250 uses the mask data pre-recorded in the memory 210 by converting it as necessary according to the restoration conditions input by the input UI 400.
  • the mask data is data showing the spatial distribution of the spectral transmittance of the filter array 110, and includes information corresponding to the matrix H in the above equation (2).
  • the generated spectroscopic image is processed by the image processing circuit 320 as needed.
  • the image processing circuit 320 displays the spectroscopic image on the display 330 after performing processing such as determination of arrangement on the screen, association with band information, or coloring corresponding to the wavelength.
  • the signal processing circuit 250 generates an image for each of a plurality of wavelength bands only for at least one designated sub-wavelength region in the target wavelength region W.
  • the wavelength regions other than the designated sub-wavelength regions are calculated by adding up the continuous wavelength regions as one wavelength region.
  • the signal processing circuit 250 may generate an image for each of the plurality of wavelength bands for the entire target wavelength region W. In that case, the image processing circuit 320 may extract and display the data for the designated sub-wavelength region from the image data input from the signal processing circuit 250.
  • FIG. 8 is a flowchart showing the operation of the system of this embodiment.
  • the user inputs the imaging condition and the restoration condition via the input UI 400 (step S101).
  • the input data indicating the imaging conditions is sent to the control circuit 150.
  • the input data indicating the restoration condition is sent to the signal processing circuit 250 and the memory 310.
  • the memory 310 temporarily stores the restoration conditions. This restoration condition is referred to when the image is displayed in order to associate the image with the condition of the set wavelength band.
  • the imaging device 100 acquires a compressed image by imaging the object according to the imaging conditions (step S102).
  • the signal processing circuit 250 determines whether or not it is necessary to convert the mask data based on the input restoration conditions (step S103). When it is necessary to convert, the signal processing circuit 250 converts the mask data stored in advance in the memory 210 (step S104). Here, the conversion refers to synthesizing mask information for a plurality of wavelength regions and handling it as mask information for one wavelength region. Details of the synthesis of mask information will be described later with reference to FIG. If no conversion is required, step S104 is omitted. The signal processing circuit 250 uses the compressed image and the mask data converted as needed to perform a restoration operation according to the input restoration conditions (step S105). As a result, a spectroscopic image is generated from the compressed image.
  • the image processing circuit 320 of the display device 300 associates the generated spectroscopic image with the restoration conditions stored in the memory 310 and labels them (step S106). For example, image data in which a label indicating a corresponding wavelength range is added to each of the generated spectroscopic images is generated.
  • the image processing circuit 320 outputs the generated image data to the display 330 and displays the image (step S107).
  • FIG. 9 shows an example of mask data before conversion stored in the memory 210.
  • the mask data in this example includes mask information indicating the spatial distribution of transmittance for each of the plurality of unit bands included in the target wavelength region.
  • the mask data in this example includes mask information for each of a large number of unit bands divided by 1 nm, and information regarding acquisition conditions for mask information. Each unit band is specified by a lower limit wavelength and an upper limit wavelength.
  • the mask information includes information on the mask image and the background image.
  • the plurality of mask images shown in FIG. 9 are acquired by capturing a plurality of backgrounds corresponding to the plurality of unit bands with the image sensor 120 through the filter array 110.
  • the plurality of background images are acquired by capturing the plurality of backgrounds with the image sensor 120 without passing through the filter array 110.
  • the data of such a mask image and a background image are recorded in advance for each unit band.
  • the information regarding the acquisition conditions includes the exposure time and gain information.
  • data of a mask image and a background image are recorded for each of a plurality of unit bands having a width of 1 nm.
  • the width of each unit band is not limited to 1 nm and can be determined to any value.
  • the mask data does not have to include the information of the background image.
  • the mask information does not have to include the background image because the mask information substantially matches the mask image.
  • GUI graphical user interface
  • FIG. 10 shows an example of a GUI screen for inputting imaging conditions.
  • the user sets the resolution, gain, exposure time, and frame rate before performing hyperspectral imaging.
  • the resolution represents the number of vertical and horizontal pixels of the displayed image.
  • the resolution can be specified, for example, by the user selecting a name such as VGA, HD, 4K from the pull-down menu, or by directly inputting the number of vertical and horizontal pixels.
  • the gain is specified by a rational number of 0 or more, and may be input by addition, subtraction, multiplication, and division of rational numbers. For example, when 8/3 is input, the gain can be set as 2.6666 ... dB. Both the exposure time and the frame rate do not have to be entered.
  • the user may input at least one of the exposure time and the frame rate, and if a conflict occurs (for example, the exposure time is 100 ms and the frame rate is 30 fps), one of them may be prioritized.
  • a function for automatically adjusting the gain, exposure time, and frame rate may be provided.
  • the average brightness may be automatically adjusted to be 1/2 of the maximum brightness.
  • the GUI for inputting the imaging conditions may have a function of saving and loading the set imaging conditions.
  • the GUI may have a function of displaying a compressed image acquired under set imaging conditions in real time. Here, it is not always necessary to display the compressed image itself. Any image acquired under the imaging conditions set at that time may be displayed.
  • pixels that output only the values of red (R), green (G), and blue (B) may be arranged, and the RGB image acquired using only the values of those pixels may be displayed.
  • restoration is performed in three bands with 400 nm to 500 nm as the first band, 500 nm to 600 nm as the second band, and 600 nm to 700 nm as the third band, and the restoration result is an RGB image. May be displayed as.
  • 11 and 12 are diagrams showing an example of a GUI for inputting restoration conditions.
  • the user inputs the sub-wavelength region, the wavelength resolution or the number of band divisions, and the number of calculations.
  • the number of calculations represents the number of repetitions of the restoration operation shown in the equation (2).
  • the sub-wavelength region can be specified by setting the lower limit wavelength and the upper limit wavelength, for example, by dragging and dropping.
  • a sub-wavelength region of 420 nm to 480 nm and a sub-wavelength region of 600 nm to 690 nm are designated. Instead of specifying by drag and drop, as shown in FIG.
  • the sub-wavelength region and the range of each wavelength band within each sub-wavelength region may be input numerically.
  • the region for inputting the sub-wavelength region and the range of each wavelength band within each sub-wavelength region may be displayed as an independent window, or may be contained in a screen for inputting other setting items.
  • the user inputs either the wavelength resolution or the number of band divisions.
  • the number of calculations is specified by any integer of 1 or more. Typically, about 10 to 10000 times can be specified.
  • the predicted calculation time is also displayed.
  • the estimated calculation time is automatically calculated and displayed from the set resolution, the number of band divisions, and the number of calculations, instead of being input by the user.
  • the functions of inputting the number of calculations and displaying the estimated calculation time may be omitted.
  • a format may be used in which, for example, a pull-down menu is selected from a plurality of modes such as a high precision mode (low speed), a balanced mode (medium speed), and a high speed mode (high speed). As shown in FIG. 11, it may have a function of saving and loading the set restoration conditions.
  • FIG. 13 is a diagram showing an example of a screen displaying a spectroscopic image generated as a result of the restoration operation.
  • the generated spectroscopic image is associated with the set restoration conditions and displayed in a format that can be distinguished for each set band.
  • the lower limit wavelength and the upper limit wavelength of each band can be displayed numerically together with the restored image of each band.
  • each band may be indicated by a number counted from the short wavelength side or the long wavelength side.
  • the image of each band may be displayed in the colors included in the band.
  • all physical quantities expressed in wavelength (nm) may be expressed in wave number (for example, cm -1 ) or frequency (for example, Hz).
  • FIG. 14 is a diagram for explaining an example of a method of synthesizing mask information of a plurality of bands and converting it into new mask information.
  • the mask information before conversion the mask information of the unit bands # 1 to 20 is stored in the memory 210 in advance as shown in FIG.
  • the synthesis processing is not performed on the unit bands # 1 to 5, but the synthesis processing is performed on the unit bands # 6 to 20.
  • the transmittance distribution of the filter array 110 is calculated by dividing the value of each region in the mask image by the value of the corresponding region in the background image.
  • the data of each mask image stored in the memory 210 is referred to as "unit mask image data", and the data of each stored background image is referred to as "unit background image data".
  • the combined transmittance distribution is the sum of the unit mask image data of bands # 6 to 20 for each pixel, and the unit background image data of bands # 6 to 20 for each pixel. Obtained by dividing by the added data. By performing such an operation, mask information can be synthesized for any plurality of bands. Further, when the uniformity of the background image is very high, the mask information substantially matches the mask image.
  • the data obtained by adding the mask image data of the bands # 6 to 20 or the averaged data may be used as the composite mask data of the bands # 6 to 20.
  • mask information is recorded for each of a large number of unit bands, each having a width of 1 nm.
  • the width of each wavelength band specified by the user is relatively wide at 30 nm.
  • the signal processing circuit 250 synthesizes mask information of a plurality of unit bands for each designated wavelength band and restores the information.
  • the mask data conversion process by synthesis may be performed in the environment used by the end user, or may be performed at a manufacturing site such as a factory that manufactures the system.
  • the converted mask data is stored in the memory 210 in advance in place of or in addition to the mask data before conversion.
  • the mask data is used to restore the spectroscopic image data for each wavelength band from the compressed image data acquired by an image pickup apparatus including a filter array including a plurality of types of optical filters having different spectral transmittances. Be done.
  • the method of converting mask data in this embodiment includes the following steps. -Acquire the first mask data for restoring the first spectroscopic image data corresponding to the first wavelength band group in the target wavelength region. -Acquire setting data that specifies one or more sub-wavelength regions that are a part of the target wavelength region. -Based on the first mask data and the setting data, second mask data for restoring the second spectroscopic image data corresponding to the second wavelength band group in the sub-wavelength region is generated.
  • the first mask data can be, for example, data for restoring a spectroscopic image for each of all the unit bands included in the target wavelength range from the compressed image.
  • the second mask data can be, for example, data for restoring a spectroscopic image for each of all the unit bands included in each designated sub-wavelength region from the compressed image.
  • the second mask data may be data for restoring a spectroscopic image for each composite band obtained by synthesizing a plurality of unit bands from a compressed image.
  • the configuration data may include data on the mode of band synthesis.
  • a synthetic band obtained by synthesizing a plurality of unit bands is also referred to as an "edited band".
  • Each of the first mask data and the second mask data is data that reflects the spatial distribution of the spectral transmittance of the filter array.
  • the first mask data includes the first mask information showing the spatial distribution of the spectral transmittance corresponding to the first wavelength band group.
  • the second mask data includes the second mask information showing the spatial distribution of the spectral transmittance corresponding to the second wavelength band group.
  • the second mask data is the third mask information obtained by synthesizing the information indicating the spatial distribution of the spectral transmittance corresponding to the third wavelength band group included in the non-designated wavelength region other than one or more designated sub-wavelength regions. It may also be included.
  • the signal processing circuit 250 is based on the compressed image and the second mask data for the spectral image having a relatively high wavelength resolution for each designated sub-wavelength region and the non-designated non-designated wavelength region. It is possible to generate a spectroscopic image having a relatively low wavelength resolution of.
  • 15A and 15B are diagrams showing an example of the converted second mask data recorded in the memory 210.
  • the mask image and the background image are stored in a combined state as the converted mask information for each of the edited bands having a width of 10 nm. If the uniformity of the background image is very high, the mask data may not include the information of the background image.
  • the converted mask data for each of the edited bands having a width of 10 nm is stored in the state of the composite mask data obtained by dividing the mask image by the background image.
  • the wavelength width of the edited band is not limited to 10 nm and can be set arbitrarily.
  • the wider the bandwidth after compositing the more unit mask images are averaged.
  • the wider the bandwidth after synthesis the more the mask data obtained by dividing more unit mask images by the unit background image becomes the averaged data. Therefore, the wider the bandwidth after composition, the smaller the contrast of the composite mask image or composite mask data tends to be.
  • FIG. 16 is a diagram showing an example of a method of generating an image for each of a plurality of wavelength bands included in a target wavelength range.
  • the target wavelength region includes four sub-wavelength regions.
  • the first sub-wavelength region includes bands # 1 to 5.
  • the second sub-wavelength region includes bands # 6-10.
  • the third sub-wavelength region includes bands # 11 to 15.
  • the fourth sub-wavelength region includes bands # 16 to 20.
  • the signal processing circuit 250 performs a restoration operation by synthesizing mask information of all unit bands that do not belong to the sub-wavelength region for each sub-wavelength region. As shown in FIG.
  • FIG. 17 is a diagram showing a system configuration when the signal processing circuit 250 does not convert mask information.
  • the signal processing circuit 250 reads the restoration conditions given from the input UI 400 and the mask information stored in the memory 210, and generates a spectral image from the compressed image acquired from the image sensor 160.
  • the signal processing circuit 250 generates a spectroscopic image over the entire target wavelength range and outputs it to the image processing circuit 320.
  • the image processing circuit 320 causes the display 330 to display only an image of a part of the wavelength bands from the acquired spectroscopic image according to the set restoration conditions.
  • FIG. 18 is a diagram showing another example of GUI for setting restoration conditions.
  • different wavelength resolutions or the number of band divisions can be specified for each set sub-wavelength range.
  • the user inputs either the wavelength resolution or the number of band divisions for each sub-wavelength region.
  • the signal processing circuit 250 performs the restoration calculation according to the input wavelength resolution or the number of band divisions. With such a configuration, a spectroscopic image can be generated with different resolutions for each sub-wavelength region.
  • FIG. 19 displays an image generated from mask information synthesized for a wavelength range (hereinafter, referred to as “non-designated wavelength range”) that is included in the target wavelength range but is not included in any of the sub-wavelength ranges. It is a figure which shows the example of the UI which performs.
  • one image generated for a non-designated wavelength region is displayed, but images for two or more non-designated wavelength regions may be displayed.
  • An RGB image may be displayed in place of or in addition to the image for the non-designated wavelength range.
  • the target wavelength range includes the visible wavelength range
  • the signal processing circuit 250 synthesizes mask information for each of the red (R), green (G), and blue (B) wavelength ranges.
  • the signal processing circuit 250 uses the combined mask information to generate image data for each of the red, green, and blue wavelength regions from the compressed image data.
  • the image processing circuit 320 causes the generated RGB image to be displayed on the display 330.
  • FIG. 20 is a diagram showing an example of a method of restoring only a specific sub-wavelength region with high wavelength resolution by performing two-step restoration.
  • a compressed image for a target wavelength region including 20 unit bands is acquired.
  • the signal processing circuit 250 may include 4 such as 1st to 5th bands, 6th to 10th bands, 11th to 15th bands, 16th to 20th bands.
  • Restoration is performed for one large sub-wavelength range (hereinafter referred to as "large sub-wavelength range").
  • the signal processing circuit 250 restores the designated specific large sub-wavelength region so as to divide it into a plurality of smaller bands (hereinafter, referred to as “small sub-wavelength region”).
  • the number of large sub-wavelength regions divided into a plurality of small sub-wavelength regions can be arbitrarily determined. In the example of FIG. 20, only one large sub-wavelength region is divided into a plurality of small sub-wavelength regions, but two or more large sub-wavelength regions may be divided into a plurality of small sub-wavelength regions. Further, in the example of FIG. 20, the signal processing circuit 250 performs band division in two stages, but a spectroscopic image may be generated through division in three or more stages. Further, the small sub-wavelength region may be a unit band.
  • each time the band is divided in each stage it may be possible to select which of the divided plurality of wavelength regions is divided into finer sub-wavelength regions. The selection may be made by the user or may be made automatically.
  • the setting data includes a plurality of large sub-wavelength regions, each of which is a part of a target wavelength region, and a plurality of small sub-wavelength regions included in at least one of the plurality of large sub-wavelength regions. Contains the information you specify.
  • the signal processing circuit 250 executes the following processing. -For each of the plurality of large sub-wavelength regions, the first synthetic mask information is generated by synthesizing the mask information for the plurality of unit bands included in the large sub-wavelength region. -Based on the compressed image data and the first composite mask information, the first composite image data is generated for each large sub-wavelength region.
  • the second composite mask information is generated by synthesizing the mask information for the plurality of unit bands included in the small sub-wavelength region. -Based on the first composite image data for the specified large sub-wavelength region and the second composite mask information, the second composite image data is generated for each small sub-wavelength region.
  • the generated hyperspectral data cube contains second composite image data for a plurality of small sub-wavelength regions.
  • detailed spectral information can be obtained only for a specific large sub-wavelength region specified by the user.
  • the configuration of the imaging device, the hyperspectral information compression algorithm, and the hyperspectral data cube reconstruction algorithm are not limited to the above-described embodiments.
  • the arrangement of the filter array 110, the optical system 140, and the image sensor 160 is not limited to the arrangement shown in FIGS. 1A to 1D, and may be appropriately modified.
  • the characteristics of the filter array 110 are not limited to the characteristics exemplified with reference to FIGS. 2A to 4B, and the filter array 110 having the optimum characteristics according to the application or purpose is used.
  • a spectroscopic image for each wavelength band may be generated by a method other than the calculation using the compressed sensing shown in the above equation (2). For example, other statistical methods such as maximum likelihood estimation or Bayesian estimation may be used.
  • the compressed image data is generated by the image pickup apparatus 100 provided with the filter array 110, but the compressed image data may be generated by another method.
  • compressed image data may be generated by allowing a coded matrix corresponding to the matrix H in the above equation (1) to act on a hyperspectral data cube generated by an arbitrary hyperspectral camera. If it is necessary to reduce the amount of data for storage or transmission of data, such software processing may generate compressed image data. Even for the compressed image data generated by such software processing, the processing in each of the above embodiments can be applied to restore the image for each wavelength band.
  • the technique of the present disclosure is useful, for example, in cameras and measuring devices that acquire multi-wavelength images.
  • the technology of the present disclosure can be applied to, for example, sensing for living organisms / medical care / beauty, foreign matter / residual pesticide inspection system for foods, remote sensing system and in-vehicle sensing system.
  • Imaging device 70
  • Filter array 120
  • Image 140 Optical system 150
  • Control circuit 160
  • Image sensor 200
  • Processing device 210
  • Memory 220 Spectral image
  • Signal processing circuit 300
  • Display device 310
  • Memory 320 Image processing circuit
  • Display 400 Input UI

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Spectrometry And Color Measurement (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
PCT/JP2021/008433 2020-03-26 2021-03-04 信号処理方法、信号処理装置、および撮像システム Ceased WO2021192891A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202180018960.9A CN115211106B (zh) 2020-03-26 2021-03-04 信号处理方法、信号处理装置及摄像系统
EP21775387.0A EP4130693A4 (en) 2020-03-26 2021-03-04 Signal processing method, signal processing device, and image-capturing system
JP2022509473A JP7720532B2 (ja) 2020-03-26 2021-03-04 信号処理方法、信号処理装置、および撮像システム
US17/930,093 US20220414948A1 (en) 2020-03-26 2022-09-07 Signal processing method, signal processing device, and imaging system

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2020056353 2020-03-26
JP2020-056353 2020-03-26
JP2021023067 2021-02-17
JP2021-023067 2021-02-17

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/930,093 Continuation US20220414948A1 (en) 2020-03-26 2022-09-07 Signal processing method, signal processing device, and imaging system

Publications (1)

Publication Number Publication Date
WO2021192891A1 true WO2021192891A1 (ja) 2021-09-30

Family

ID=77890052

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/008433 Ceased WO2021192891A1 (ja) 2020-03-26 2021-03-04 信号処理方法、信号処理装置、および撮像システム

Country Status (5)

Country Link
US (1) US20220414948A1 (https=)
EP (1) EP4130693A4 (https=)
JP (1) JP7720532B2 (https=)
CN (1) CN115211106B (https=)
WO (1) WO2021192891A1 (https=)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022163421A1 (ja) * 2021-01-26 2022-08-04 パナソニックＩｐマネジメント株式会社 検査対象に含まれる異物を検出する方法および装置
WO2022176686A1 (ja) * 2021-02-22 2022-08-25 パナソニックＩｐマネジメント株式会社 異物を検出する方法および装置
WO2022196351A1 (ja) * 2021-03-15 2022-09-22 ソニーグループ株式会社 情報処理装置、情報処理方法及びプログラム
WO2023106142A1 (ja) * 2021-12-08 2023-06-15 パナソニックＩｐマネジメント株式会社 信号処理方法、プログラム、およびシステム
JP2023083837A (ja) * 2021-12-06 2023-06-16 豊田合成株式会社 乗員モニタリングシステム用撮影装置及び乗員モニタリングシステム
WO2025084081A1 (ja) * 2023-10-16 2025-04-24 パナソニックＩｐマネジメント株式会社 撮像システムおよびそれに用いられる方法
WO2025121318A1 (ja) * 2023-12-06 2025-06-12 パナソニックＩｐマネジメント株式会社 情報処理方法及び撮像システム
WO2026034167A1 (ja) * 2024-08-06 2026-02-12 パナソニックＩｐマネジメント株式会社 画像処理方法及び画像処理システム

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6945195B2 (ja) * 2019-01-16 2021-10-06 パナソニックＩｐマネジメント株式会社 光学フィルタ、光検出装置、および光検出システム
WO2022270355A1 (ja) * 2021-06-24 2022-12-29 パナソニックＩｐマネジメント株式会社 撮像システム、撮像システムに用いられる方法、および撮像システムに用いられるコンピュータプログラム
WO2024053302A1 (ja) * 2022-09-08 2024-03-14 パナソニックＩｐマネジメント株式会社 情報処理方法および撮像システム

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016130727A (ja) * 2015-01-08 2016-07-21 パナソニックＩｐマネジメント株式会社 撮像装置
JP2016156801A (ja) * 2014-11-19 2016-09-01 パナソニックＩｐマネジメント株式会社 撮像装置および分光システム
JP2017208642A (ja) * 2016-05-17 2017-11-24 キヤノン株式会社 圧縮センシングを用いた撮像装置、撮像方法および撮像プログラム
JP2018511253A (ja) * 2015-03-24 2018-04-19 ザ ユニバーシティ オブ ユタ リサーチ ファウンデイション 分散して空間的符号化画像を作成する画像を伴う撮像素子
JP2018107731A (ja) * 2016-12-27 2018-07-05 パナソニックＩｐマネジメント株式会社 画像生成装置及び撮像装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7230738B2 (en) * 2002-10-09 2007-06-12 Xerox Corporation System for spectral multiplexing of source image to provide a composite image with noise encoding to increase image confusion in the composite image, for rendering the composite image, and for spectral demultiplexing of the composite image
JP2005201693A (ja) * 2004-01-13 2005-07-28 Olympus Corp 色票処理装置、色票処理方法及び色票処理プログラム
US8428372B2 (en) * 2010-04-09 2013-04-23 The Boeing Company Method, apparatus and computer program product for compressing data
CN102589695B (zh) * 2012-02-27 2014-02-26 苏州大学 一种光谱成像方法及其系统
CN104244018B (zh) * 2014-09-19 2018-04-27 重庆邮电大学 快速压缩高光谱信号的矢量量化方法
CN104732566B (zh) * 2015-03-16 2017-08-29 西北工业大学 基于非分离稀疏先验的高光谱图像压缩感知方法
CN105719322A (zh) * 2016-01-27 2016-06-29 四川用联信息技术有限公司 一种基于自乘矩阵变换的多光谱图像压缩方法
CN108960333B (zh) * 2018-07-12 2021-12-17 西安电子科技大学 基于深度学习的高光谱图像无损压缩方法
CN110501071B (zh) * 2019-08-02 2021-07-13 杭州电子科技大学 一种基于模糊编码的压缩高光谱掩膜优化方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016156801A (ja) * 2014-11-19 2016-09-01 パナソニックＩｐマネジメント株式会社 撮像装置および分光システム
US9599511B2 (en) 2014-11-19 2017-03-21 Panasonic Intellectual Property Management Co., Ltd. Imaging apparatus comprising coding element and spectroscopic system comprising the imaging apparatus
JP2016130727A (ja) * 2015-01-08 2016-07-21 パナソニックＩｐマネジメント株式会社 撮像装置
JP2018511253A (ja) * 2015-03-24 2018-04-19 ザ ユニバーシティ オブ ユタ リサーチ ファウンデイション 分散して空間的符号化画像を作成する画像を伴う撮像素子
JP2017208642A (ja) * 2016-05-17 2017-11-24 キヤノン株式会社 圧縮センシングを用いた撮像装置、撮像方法および撮像プログラム
JP2018107731A (ja) * 2016-12-27 2018-07-05 パナソニックＩｐマネジメント株式会社 画像生成装置及び撮像装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4130693A4

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022163421A1 (ja) * 2021-01-26 2022-08-04 パナソニックＩｐマネジメント株式会社 検査対象に含まれる異物を検出する方法および装置
WO2022176686A1 (ja) * 2021-02-22 2022-08-25 パナソニックＩｐマネジメント株式会社 異物を検出する方法および装置
WO2022196351A1 (ja) * 2021-03-15 2022-09-22 ソニーグループ株式会社 情報処理装置、情報処理方法及びプログラム
JP2023083837A (ja) * 2021-12-06 2023-06-16 豊田合成株式会社 乗員モニタリングシステム用撮影装置及び乗員モニタリングシステム
JP7647520B2 (ja) 2021-12-06 2025-03-18 豊田合成株式会社 乗員モニタリングシステム
WO2023106142A1 (ja) * 2021-12-08 2023-06-15 パナソニックＩｐマネジメント株式会社 信号処理方法、プログラム、およびシステム
WO2025084081A1 (ja) * 2023-10-16 2025-04-24 パナソニックＩｐマネジメント株式会社 撮像システムおよびそれに用いられる方法
WO2025121318A1 (ja) * 2023-12-06 2025-06-12 パナソニックＩｐマネジメント株式会社 情報処理方法及び撮像システム
WO2026034167A1 (ja) * 2024-08-06 2026-02-12 パナソニックＩｐマネジメント株式会社 画像処理方法及び画像処理システム

Also Published As

Publication number Publication date
CN115211106B (zh) 2024-12-10
EP4130693A1 (en) 2023-02-08
JP7720532B2 (ja) 2025-08-08
EP4130693A4 (en) 2023-09-06
CN115211106A (zh) 2022-10-18
JPWO2021192891A1 (https=) 2021-09-30
US20220414948A1 (en) 2022-12-29

Similar Documents

Publication Publication Date Title
JP7720532B2 (ja) 信号処理方法、信号処理装置、および撮像システム
JP6952277B2 (ja) 撮像装置および分光システム
JP7668452B2 (ja) 信号処理方法、信号処理装置、および撮像システム
US11843876B2 (en) Optical filter array, photodetection device, and photodetection system
JP7829156B2 (ja) 画像処理装置、撮像システム、および復元画像の誤差を推定する方法
JP2016130727A (ja) 撮像装置
WO2021241171A1 (ja) フィルタアレイおよび光検出システム
WO2023286613A1 (ja) フィルタアレイ、光検出装置、および光検出システム
CN117460947A (zh) 信号处理装置及信号处理方法
WO2022163421A1 (ja) 検査対象に含まれる異物を検出する方法および装置
WO2023106142A1 (ja) 信号処理方法、プログラム、およびシステム
WO2024043009A1 (ja) 信号処理方法および信号処理装置
JP7122636B2 (ja) フィルタアレイおよび光検出システム
JP2024020922A (ja) 復元画像の評価方法および撮像システム、
US20250386084A1 (en) Imaging system, matrix data, and matrix data generation method
WO2026088780A1 (ja) 情報処理方法、処理装置、およびコンピュータプログラム
WO2025192485A1 (ja) 撮像システムおよび撮像方法
WO2023106143A1 (ja) 分光画像を生成するシステムに用いられる装置およびフィルタアレイ、分光画像を生成するシステム、ならびにフィルタアレイの製造方法
WO2025100176A1 (ja) 画像を補正する方法およびシステム
WO2024203282A1 (ja) 撮像システムおよび方法
WO2025084081A1 (ja) 撮像システムおよびそれに用いられる方法
WO2024142683A1 (ja) 光検出装置、光検出システム、およびフィルタアレイ
WO2024053302A1 (ja) 情報処理方法および撮像システム
WO2025192316A1 (ja) 撮像装置、撮像システム、および撮像方法
JP2025071703A (ja) 画像を生成するシステムに用いられる装置およびフィルタアレイ

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21775387

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022509473

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021775387

Country of ref document: EP

Effective date: 20221026