Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
  • G01B11/00—Measuring arrangements characterised by the use of optical techniques
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
  • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
  • G01J3/28—Investigating the spectrum
  • G01J3/30—Measuring the intensity of spectral lines directly on the spectrum itself
  • G01J3/36—Investigating two or more bands of a spectrum by separate detectors
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
  • H04N23/13—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with multiple sensors
  • H04N23/15—Image signal generation with circuitry for avoiding or correcting image misregistration
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/50—Constructional details
  • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules

Definitions

• the technology disclosed herein relates to a processing device, an inspection device, a processing method, and a program.
• JP 2018-004509 A discloses a color measurement system equipped with an inspection device having an illumination device that irradiates light onto the measurement object and an imaging device that acquires spectral information of the measurement object.
• the inspection device has a contact part that comes into contact with the measurement object and whose positional relationship with the illumination device and imaging device is known, and the imaging device acquires the two-dimensional distribution of the spectral information of the measurement object in one shot.
• JP2015-194368A discloses a defect inspection method.
• an inspection object is illuminated with an illumination device selected from a plurality of illumination devices capable of illuminating the inspection object from different directions, and a surface image of the inspection object is acquired.
• the defect inspection method also compares, for each pixel, the luminance of the acquired surface image with the luminance of a reference image of an inspection object having no defects, and corrects the position of the inspection object by moving the inspection object to a position where the imaging positions of the surface image and the reference image match.
• the defect inspection method then acquires a surface image of the inspection object after the correction, compares the luminance of the surface image with the reference image for each pixel, and determines that a defect exists in a specified area if the degree of match of luminance in the specified area is lower than a threshold value.
• JP Patent Publication 08-240535 A discloses an appearance inspection device that includes a light source, a reflector, an imaging unit, and a defect determination unit.
• the light source irradiates light from diagonally above the inspection object being transported.
• the reflectors are provided below the inspection object and diagonally above the inspection object facing the light source across the inspection object, and diffusely reflect the light.
• the imaging unit is provided vertically above the inspection object.
• the defect determination unit performs binarization of the image signal obtained by A/D converting the output signal from the imaging unit and integration of the density value, etc., to determine chipping defects of the inspection object, streaky paint defects on the surface of the inspection object that occur in the same direction as the transport direction, and band-like paint defects on the surface of the inspection object that occur in the same direction as the transport direction.
• One embodiment of the technology disclosed herein provides a processing device, inspection device, processing method, and program that can improve measurement accuracy compared to measuring the spectral reflectance of a subject by directly using imaging data obtained by imaging the subject with an imaging device.
• a first aspect of the technology disclosed herein is a processing device that includes a processor, and the processor acquires first imaging data obtained by imaging a subject having a first surface and a second surface with an imaging device, acquires reference imaging data in a reference region spanning the first surface and the second surface from the first imaging data, and corrects second imaging data obtained by imaging with the imaging device based on the reference imaging data.
• a second aspect of the technology disclosed herein is a processing device according to the first aspect, in which the first imaging data includes the second imaging data.
• a third aspect of the technology disclosed herein is a processing device according to the first or second aspect, in which the second imaging data is processed by a processing device different from the processing device used for the first imaging data.
• a fourth aspect of the technology disclosed herein is a processing device according to any one of the first to third aspects, in which correcting the second imaging data includes correcting data related to non-uniformity in the spectral reflectance of the subject.
• a fifth aspect of the technology disclosed herein is a processing device according to any one of the first to fourth aspects, in which the imaging device is a spectroscopic imaging device.
• a sixth aspect of the technology disclosed herein is a processing device according to the fifth aspect, in which the spectroscopic imaging device includes a plurality of spectroscopic filters, and the center of gravity of the plurality of spectroscopic filters is located at a position different from the optical axis.
• a seventh aspect of the technology disclosed herein is a processing device according to any one of the first to sixth aspects, in which the processor acquires reference imaging data when a calculation result is obtained based on the first imaging data, indicating that the first and second surfaces have an angle.
• An eighth aspect of the technology disclosed herein is a processing device according to the seventh aspect, in which the imaging device is a spectroscopic imaging device, the spectroscopic imaging device includes a plurality of spectroscopic filters, the centers of gravity of which are located at a position different from the optical axis, and the calculation result is a processing device in which a first area, which is the area of an image region corresponding to a first surface derived based on a first wavelength band image corresponding to a first spectral filter of the plurality of spectroscopic filters, is different from a second area, which is the area of an image region corresponding to the first surface derived based on a second wavelength band image corresponding to a second spectral filter of the plurality of spectroscopic filters.
• a ninth aspect of the technology disclosed herein is a processing device according to the eighth aspect, in which the first area is an area derived based on the luminance distribution of the first wavelength band image, and the second area is an area derived based on the luminance distribution of the second wavelength band image.
• a tenth aspect of the technology disclosed herein is a processing device according to any one of the first to ninth aspects, in which the reference area is an area that is determined based on the boundary between the first and second surfaces.
• An eleventh aspect of the technology disclosed herein is a processing device according to any one of the first to ninth aspects, in which correcting the second imaging data includes calculating an average value of the spectral reflectance in the reference region, and correcting data relating to the non-uniformity of the spectral reflectance in a first evaluation region of the first surface that is different from the reference region based on the average value.
• a twelfth aspect of the technology disclosed herein is a processing device according to the eleventh aspect, in which correcting the second imaging data includes correcting data relating to the non-uniformity of the spectral reflectance of a second evaluation area of the second surface that is different from the reference area, based on the average value.
• a thirteenth aspect of the technology disclosed herein is a processing device according to the eleventh or twelfth aspect, in which the processor outputs a first evaluation result regarding the degree of unevenness in the spectral reflectance of the first evaluation area obtained by performing the correction.
• a fourteenth aspect of the technology disclosed herein is a processing device according to the twelfth aspect or the thirteenth aspect dependent on the twelfth aspect, in which the processor outputs a second evaluation result regarding the degree of unevenness in the spectral reflectance of the second evaluation area obtained by performing the correction.
• a fifteenth aspect of the technology disclosed herein is a processing device according to any one of the first to fourteenth aspects, in which the first surface and the second surface are irradiated with light under different conditions.
• a sixteenth aspect of the technology disclosed herein is an inspection device that includes a processing device according to any one of the first to fifteenth aspects and an imaging device.
• a seventeenth aspect of the technology disclosed herein is a processing method that includes acquiring first imaging data obtained by imaging an object having a first surface and a second surface with an imaging device, acquiring reference imaging data in a reference region spanning the first surface and the second surface from the first imaging data, and correcting second imaging data obtained by imaging with the imaging device based on the reference imaging data.
• An 18th aspect of the technology disclosed herein is a program for causing a computer to execute a process including obtaining first imaging data obtained by imaging an object having a first surface and a second surface with an imaging device, obtaining reference imaging data in a reference region spanning the first surface and the second surface from the first imaging data, and correcting, based on the reference imaging data, second imaging data obtained by imaging with the imaging device.
• FIG. 1 is a block diagram showing an example of an inspection device.
• FIG. 1 is a perspective view showing an example of an imaging device.
• FIG. 2 is an exploded perspective view showing an example of a pupil division filter.
• FIG. 2 is a block diagram showing an example of a hardware configuration of the imaging apparatus.
• FIG. 2 is an exploded perspective view showing an example of a portion of a photoelectric conversion element.
• FIG. 2 is a block diagram showing an example of a functional configuration of the imaging device.
• 11 is a block diagram showing an example of the operation of an output value acquisition unit and an interference removal processing unit.
• FIG. FIG. 2 is a block diagram showing an example of a functional configuration of the processing device.
• 5 is a block diagram showing an example of the operation of an imaging data acquisition unit and an image area setting unit.
• FIG. 1 is a perspective view showing an example of an imaging device.
• FIG. 2 is an exploded perspective view showing an example of a pupil division filter.
• FIG. 2 is a block
• FIG. 11 is a block diagram showing an example of the operation of an area derivation unit and an area determination unit
• FIG. 13 is a block diagram showing an example of the operation of a boundary setting unit and a reference image area setting unit.
• FIG. 13 is a block diagram showing an example of the operation of an average value derivation unit and an evaluation image area setting unit.
• FIG. 11 is a block diagram showing an example of the operation of an evaluation value derivation unit and an evaluation result output unit.
• FIG. 11 is a flowchart showing an example of the flow of a spectral image generation process.
• 13 is a flowchart showing an example of the flow of a measurement process.
• RGB is an abbreviation for "Red Green Blue”.
• LED is an abbreviation for "light emitting diode”.
• CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor”.
• CCD is an abbreviation for "Charge Coupled Device”.
• I/F is an abbreviation for "Interface”.
• RAM is an abbreviation for "Random Access Memory”.
• CPU is an abbreviation for "Central Processing Unit”.
• GPU is an abbreviation for "Graphics Processing Unit”.
• EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory”.
• HDD is an abbreviation for "Hard Disk Drive”.
• EL is an abbreviation for "Electro Luminescence”.
• TPU is an abbreviation for "Tensor processing unit”.
• SSD is an abbreviation for "Solid State Drive”.
• USB is an abbreviation for "Universal Serial Bus”.
• ASIC is an abbreviation for "Application Specific Integrated Circuit”.
• FPGA is an abbreviation for "Field-Programmable Gate Array”.
• PLD is an abbreviation for "Programmable Logic Device”.
• SoC is an abbreviation for "System-on-a-Chip”.
• IC is an abbreviation for "Integrated Circuit”.
• “same” refers to sameness in the sense of including, in addition to perfect sameness, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and an error that does not go against the spirit of the technology of the present disclosure.
• “orthogonal” refers to orthogonal in the sense of including, in addition to perfect orthogonality, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and an error that does not go against the spirit of the technology of the present disclosure.
• straight line refers to a straight line in the sense of including, in addition to perfect straight line, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and an error that does not go against the spirit of the technology of the present disclosure.
• the inspection device 130 includes a first light source 132A, a second light source 132B, a housing 134, an imaging device 10, and a processing device 90.
• the inspection device 130 is an example of an "inspection device” according to the technology of the present disclosure.
• the imaging device 10 is an example of an “imaging device” according to the technology of the present disclosure.
• the processing device 90 is an example of a "processing device” according to the technology of the present disclosure.
• the first light source 132A and the second light source 132B are examples of a "light source” according to the technology of the present disclosure.
• the first light source 132A and the second light source 132B are, for example, an LED light source, a laser light source, or an incandescent light bulb.
• the light emitted from the first light source 132A and the second light source 132B is unpolarized.
• the first light source 132A and the second light source 132B are arranged at the upper part inside the housing 134.
• the number of the first light source 132A and the second light source 132B is two.
• the first light source 132A and the second light source 132B are arranged on both sides of the imaging device 10.
• the number of the first light source 132A and the second light source 132B may be any number. Instead of the first light source 132A and the second light source 132B, a single light source may be used.
• the housing 134 is configured to cover the imaging space 136.
• a first light source 132A, a second light source 132B, an entrance portion 10A of the imaging device 10, and a subject 200 are arranged in the imaging space 136.
• the imaging device 10 is provided on a ceiling portion 134A of the housing 134.
• the subject 200 is a triangular prism.
• the subject 200 has a first surface 202A, a second surface 202B, and a third surface 202C.
• the first surface 202A, the second surface 202B, and the third surface 202C are the side surfaces of the triangular prism.
• the subject 200 is placed on the bottom 134B of the housing 134 with the third surface 202C in contact with the bottom 134B and with the first surface 202A and the second surface 202B facing the imaging device 10.
• the subject 200 is placed, for example, on the optical axis OA of the imaging device 10.
• the first surface 202A and the second surface 202B are flat surfaces.
• the first surface 202A is angled with respect to the second surface 202B.
• a ridge line 204 is formed between the first surface 202A and the second surface 202B, extending in the height direction of the triangular prism.
• the imaging device 10 is, as an example, a multispectral camera.
• the imaging device 10 may be a spectral camera such as a hyperspectral camera.
• the imaging device 10 may also be an RGB camera with a spectral filter.
• the imaging device 10 is a multispectral camera.
• the imaging device 10 is an example of a "spectral imaging device" according to the technology of the present disclosure.
• the imaging device 10 includes an optical system 26 and an image sensor 28.
• the optical system 26 includes a first lens 30, a pupil division filter 16, and a second lens 32.
• the first lens 30, the pupil division filter 16, and the second lens 32 are arranged in this order along the optical axis OA from the subject 200 side to the image sensor 28 side.
• the pupil division filter 16 has spectral filters 20A to 20C.
• Each of the spectral filters 20A to 20C is a bandpass filter that transmits light in a specific wavelength band.
• the spectral filters 20A to 20C have different wavelength bands. Specifically, the spectral filter 20A has a first wavelength band ⁇ 1 , the spectral filter 20B has a second wavelength band ⁇ 2 , and the spectral filter 20C has a third wavelength band ⁇ 3 .
• each of the spectral filters 20A to 20C will be referred to as a "spectral filter 20.”
• each of the first wavelength band ⁇ 1 , the second wavelength band ⁇ 2 , and the third wavelength band ⁇ 3 will be referred to as a "wavelength band ⁇ .”
• the spectral filter 20 is an example of a "spectral filter” according to the technology of the present disclosure.
• the spectral filter 20A corresponding to the first wavelength band ⁇ 1 is an example of a "first spectral filter” according to the technology of the present disclosure.
• the spectral filter 20B corresponding to the second wavelength band ⁇ 2 is an example of a "second spectral filter” according to the technology of the present disclosure.
• spectral images 72A to 72C corresponding to each wavelength band ⁇ are generated based on an image 70 obtained by imaging the subject 200.
• the spectral image 72A is a spectral image corresponding to the first wavelength band ⁇ 1
• the spectral image 72B is a spectral image corresponding to the second wavelength band ⁇ 2
• the spectral image 72C is a spectral image corresponding to the third wavelength band ⁇ 3.
• each of the spectral images 72A to 72C will be referred to as a "spectral image 72.”
• three spectral images 72 are generated based on light that has been dispersed into three wavelength bands ⁇ .
• three wavelength bands ⁇ are merely an example, and two or more wavelength bands ⁇ may be used.
• the imaging device 10 has a zoom function.
• the angle of view of the imaging device 10 is adjusted by the zoom function.
• the angle of view of the imaging device 10 is set to an angle of view in which the imaging range of the imaging device 10 includes the subject 200.
• the processing device 90 is connected to the imaging device 10 so as to be able to communicate with it.
• the processing device 90 is, for example, an information processing device such as a personal computer or a server.
• the processing device 90 is equipped with a display device 122.
• the display device 122 is, for example, a liquid crystal display or an EL display.
• the processing device 90 measures the spectral reflectance of the first surface 202A and the second surface 202B of the subject 200 based on the multiple spectral images 72, and displays the measurement result 124, which is the result of the measurement, on the display device 122.
• the imaging device 10 includes a lens device 12 and an imaging device body 14.
• the lens device 12 has a pupil division filter 16.
• the imaging device 10 is a multispectral camera that captures light that has been dispersed into multiple wavelength bands ⁇ by the pupil division filter 16, thereby generating and outputting multiple spectral images 72A-72C.
• the pupil division filter 16 has a frame 18, spectral filters 20A-20C, and polarizing filters 22A-22C.
• the frame 18 has openings 24A-24C.
• the openings 24A-24C are formed in a line around the optical axis OA.
• each opening 24A-24C will be referred to as "opening 24.”
• the spectral filters 20A-20C are provided in the openings 24A-24C, respectively, and are arranged in a line around the optical axis OA. As a result, the center of gravity of each of the spectral filters 20A-20C is located at a position different from the optical axis OA.
• the spectral filters 20A-20C are an example of “multiple spectral filters” according to the technology disclosed herein.
• Polarizing filters 22A to 22C are provided corresponding to spectral filters 20A to 20C, respectively. Specifically, polarizing filter 22A is provided in opening 24A and is superimposed on spectral filter 20A. Polarizing filter 22B is provided in opening 24B and is superimposed on spectral filter 20B. Polarizing filter 22C is provided in opening 24C and is superimposed on spectral filter 20C.
• Each of the polarizing filters 22A to 22C is an optical filter that transmits light that vibrates in a specific direction.
• the polarizing filters 22A to 22C have a polarization axis with a different polarization angle.
• the polarizing filter 22A has a first polarization angle ⁇ 1
• the polarizing filter 22B has a second polarization angle ⁇ 2
• the polarizing filter 22C has a third polarization angle ⁇ 3 .
• the polarization axis may be referred to as a transmission axis.
• the first polarization angle ⁇ 1 is set to 0°
• the second polarization angle ⁇ 2 is set to 45°
• the third polarization angle ⁇ 3 is set to 90°.
• each of the polarizing filters 22A to 22C will be referred to as the "polarizing filter 22.” Furthermore, when there is no need to distinguish between the first polarization angle ⁇ 1 , the second polarization angle ⁇ 2 , and the third polarization angle ⁇ 3 , each of the first polarization angle ⁇ 1 , the second polarization angle ⁇ 2 , and the third polarization angle ⁇ 3 will be referred to as the "polarization angle ⁇ .”
• the number of the multiple openings 24 is three, corresponding to the number of the multiple wavelength bands ⁇ , but the number of the multiple openings 24 may be greater than the number of the multiple wavelength bands ⁇ (i.e., the number of the multiple spectral filters 20). Furthermore, the unused openings 24 among the multiple openings 24 may be blocked by a shielding member (not shown). Furthermore, in the example shown in FIG. 3, the multiple spectral filters 20 have different wavelength bands ⁇ , but the multiple spectral filters 20 may include spectral filters 20 having the same wavelength band ⁇ .
• the lens device 12 includes an optical system 26, and the imaging device body 14 includes an image sensor 28.
• the optical system 26 includes a pupil division filter 16, a first lens 30, and a second lens 32.
• the first lens 30 causes the light reflected by the subject 200 to enter the pupil division filter 16.
• the second lens 32 causes the light transmitted through the pupil division filter 16 to form an image on the light receiving surface 34A of the photoelectric conversion element 34 provided in the image sensor 28.
• the pupil division filter 16 is disposed at the pupil position of the optical system 26.
• the pupil position refers to the diaphragm surface that limits the brightness of the optical system 26.
• the pupil position here includes nearby positions, and nearby positions refer to the range from the entrance pupil to the exit pupil.
• the configuration of the pupil division filter 16 is as described using Figure 3.
• Figure 4 shows multiple spectral filters 20 and multiple polarizing filters 22 arranged in a straight line along a direction perpendicular to the optical axis OA.
• the image sensor 28 includes a photoelectric conversion element 34 and a signal processing circuit 36.
• the image sensor 28 is a CMOS image sensor.
• a CMOS image sensor is exemplified as the image sensor 28, but the technology of the present disclosure is not limited to this, and the technology of the present disclosure is also applicable even if the image sensor 28 is another type of image sensor, such as a CCD image sensor.
• FIG. 4 shows a schematic configuration of the photoelectric conversion element 34.
• FIG. 5 specifically shows a portion of the configuration of the photoelectric conversion element 34.
• the photoelectric conversion element 34 has a pixel layer 38, a polarizing filter layer 40, and a spectral filter layer 42. Note that the configuration of the photoelectric conversion element 34 shown in FIG. 5 is only an example, and the technology disclosed herein can be applied even if the photoelectric conversion element 34 does not have the spectral filter layer 42.
• the pixel layer 38 has a plurality of pixels 44.
• the plurality of pixels 44 are arranged in a matrix and form the light receiving surface 34A of the photoelectric conversion element 34.
• Each pixel 44 is a physical pixel having a photodiode (not shown), which photoelectrically converts the received light and outputs an electrical signal according to the amount of received light.
• the pixels 44 provided in the photoelectric conversion element 34 will be referred to as “physical pixels 44" to distinguish them from the pixels that form the spectral image. Also, the pixels that form the spectral image 72 will be referred to as “image pixels.”
• the photoelectric conversion element 34 outputs the electrical signals output from the multiple physical pixels 44 to the signal processing circuit 36 as imaging data.
• the signal processing circuit 36 digitizes the analog imaging data input from the photoelectric conversion element 34.
• the imaging data is image data that represents the captured image 70.
• the multiple physical pixels 44 form multiple pixel blocks 46.
• Each pixel block 46 is formed by a total of four physical pixels 44, two vertically and two horizontally.
• the four physical pixels 44 that form each pixel block 46 are shown arranged in a straight line along a direction perpendicular to the optical axis OA, but the four physical pixels 44 are arranged adjacent to each other in the vertical and horizontal directions of the photoelectric conversion element 34 (see FIG. 5).
• the polarizing filter layer 40 has a plurality of types of polarizers 48A to 48D.
• Each of the polarizers 48A to 48D is an optical filter that transmits light vibrating in a specific direction.
• the polarizers 48A to 48D have polarization axes with different polarization angles. Specifically, the polarizer 48A has a first polarization angle ⁇ 1 , the polarizer 48B has a second polarization angle ⁇ 2 , the polarizer 48C has a third polarization angle ⁇ 3 , and the polarizer 48D has a fourth polarization angle ⁇ 4.
• the first polarization angle ⁇ 1 is set to 0°
• the second polarization angle ⁇ 2 is set to 45°
• the third polarization angle ⁇ 3 is set to 90°
• the fourth polarization angle ⁇ 4 is set to 135°.
• each of the polarizers 48A to 48D will be referred to as a "polarizer 48.” Furthermore, when there is no need to distinguish between the first polarization angle ⁇ 1 , the second polarization angle ⁇ 2 , the third polarization angle ⁇ 3 , and the fourth polarization angle ⁇ 4 , each of the first polarization angle ⁇ 1 , the second polarization angle ⁇ 2 , the third polarization angle ⁇ 3 , and the fourth polarization angle ⁇ 4 will be referred to as a "polarization angle ⁇ .”
• the spectral filter layer 42 has a B filter 50A, a G filter 50B, and an R filter 50C.
• the B filter 50A is a blue-range filter that transmits the most light in the blue wavelength band of light in a plurality of wavelength bands.
• the G filter 50B is a green-range filter that transmits the most light in the green wavelength band of light in a plurality of wavelength bands.
• the R filter 50C is a red-range filter that transmits the most light in the red wavelength band of light in a plurality of wavelength bands.
• the B filter 50A, the G filter 50B, and the R filter 50C are assigned to each pixel block 46.
• the B filter 50A, the G filter 50B, and the R filter 50C are shown arranged in a line in a direction perpendicular to the optical axis OA, but as an example, as shown in FIG. 5, the B filter 50A, the G filter 50B, and the R filter 50C are arranged in a matrix in a default pattern arrangement.
• the B filter 50A, the G filter 50B, and the R filter 50C are arranged in a matrix in a Bayer array, which is an example of a default pattern arrangement.
• the default pattern arrangement may be an RGB stripe array, an R/G checkerboard array, an X-Trans (registered trademark) array, a honeycomb array, or the like, in addition to the Bayer array.
• filter 50 when there is no need to distinguish between the B filter 50A, the G filter 50B, and the R filter 50C, they will each be referred to as "filter 50.”
• the imaging device body 14 includes, in addition to the image sensor 28, a control driver 52, an input/output I/F 54, a computer 56, and a communication device 58.
• the input/output I/F 54 is connected to the signal processing circuit 36, the control driver 52, the computer 56, and the communication device 58.
• the computer 56 has a processor 60, storage 62, and RAM 64.
• the processor 60 controls the entire imaging device 10.
• the processor 60 is, for example, a processing device including a CPU and a GPU, and the GPU operates under the control of the CPU and is responsible for executing image-related processing.
• a processing device including a CPU and a GPU is given as an example of the processor 60, but this is merely one example, and the processor 60 may be one or more CPUs that integrate a GPU function, or one or more CPUs that do not integrate a GPU function.
• the processor 60, storage 62, and RAM 64 are connected via a bus 66, which is connected to the input/output I/F 54.
• the storage 62 is a non-transitory storage medium, and stores various parameters and programs.
• the storage 62 is a flash memory (e.g., an EEPROM).
• EEPROM e.g., EEPROM
• the RAM 64 temporarily stores various information and is used as a work memory. Examples of the RAM 64 include a DRAM and/or an SRAM.
• the processor 60 reads the necessary programs from the storage 62 and executes the read programs on the RAM 64.
• the processor 60 controls the control driver 52 and the signal processing circuit 36 according to the programs executed on the RAM 64.
• the control driver 52 controls the photoelectric conversion element 34 under the control of the processor 94.
• the communication device 58 is connected to the processor 60 via the input/output I/F 54 and the bus 66.
• the communication device 58 is also connected to the processing device 90 so that it can communicate with it via wired or wireless communication.
• the communication device 58 is responsible for sending and receiving information to and from the processing device 90. For example, the communication device 58 transmits data to the processing device 90 in response to a request from the processor 60.
• the communication device 58 also receives data transmitted from the processing device 90 and outputs the received data to the processor 60 via the bus 66.
• a spectral image generation program 80 is stored in the storage 62.
• the processor 60 reads the spectral image generation program 80 from the storage 62 and executes the read spectral image generation program 80 on the RAM 64.
• the processor 60 executes a spectral image generation process for generating a plurality of spectral images 72 in accordance with the spectral image generation program 80 executed on the RAM 64.
• the spectral image generation process is realized by the processor 60 operating as an output value acquisition unit 82 and an interference removal processing unit 84 in accordance with the spectral image generation program 80.
• the output value acquisition unit 82 acquires the output value Y of each physical pixel 44 based on the imaging data.
• the output value Y of each physical pixel 44 corresponds to the luminance value of each image pixel included in the captured image 70 represented by the imaging data.
• the output value Y of each physical pixel 44 is a value including interference (i.e., crosstalk). That is, since light of each wavelength band ⁇ of the first wavelength band ⁇ 1 , the second wavelength band ⁇ 2 , and the third wavelength band ⁇ 3 is incident on each physical pixel 44, the output value Y is a mixture of a value corresponding to the amount of light of the first wavelength band ⁇ 1 , a value corresponding to the amount of light of the second wavelength band ⁇ 2 , and a value corresponding to the amount of light of the third wavelength band ⁇ 3 .
• the processor 60 In order to obtain the spectral image 72, the processor 60 must perform a process of separating and extracting values corresponding to each wavelength band ⁇ from the output value Y for each physical pixel 44, that is, a process of removing interference, on the output value Y. Therefore, in this embodiment, in order to obtain the spectral image 72, the interference removal processing unit 84 performs interference removal processing on the output value Y of each physical pixel 44 acquired by the output value acquisition unit 82.
• the output value Y of each physical pixel 44 includes, for red, green, and blue, each luminance value for each polarization angle ⁇ as a component of the output value Y.
• the output value Y of each physical pixel 44 is expressed by equation (1).
• Y ⁇ 1_R is the luminance value of the red component of the output value Y whose polarization angle is the first polarization angle ⁇ 1
• Y ⁇ 2_R is the luminance value of the red component of the output value Y whose polarization angle is the second polarization angle ⁇ 2
• Y ⁇ 3_R is the luminance value of the red component of the output value Y whose polarization angle is the third polarization angle ⁇ 3
• Y ⁇ 4_R is the luminance value of the red component of the output value Y whose polarization angle is the fourth polarization angle ⁇ 4 .
• Y ⁇ 1_G is the luminance value of the green component of the output value Y whose polarization angle is the first polarization angle ⁇ 1
• Y ⁇ 2_G is the luminance value of the green component of the output value Y whose polarization angle is the second polarization angle ⁇ 2
• Y ⁇ 3_G is the luminance value of the green component of the output value Y whose polarization angle is the third polarization angle ⁇ 3
• Y ⁇ 4_G is the luminance value of the green component of the output value Y whose polarization angle is the fourth polarization angle ⁇ 4 .
• Y ⁇ 1_B is the luminance value of the blue component of the output value Y whose polarization angle is the first polarization angle ⁇ 1
• Y ⁇ 2_B is the luminance value of the blue component of the output value Y whose polarization angle is the second polarization angle ⁇ 2
• Y ⁇ 3_B is the luminance value of the blue component of the output value Y whose polarization angle is the third polarization angle ⁇ 3
• Y ⁇ 4_B is the luminance value of the blue component of the output value Y whose polarization angle is the fourth polarization angle ⁇ 4 .
• the pixel value X of each image pixel forming the spectral image 72 includes, as components of the pixel value X, a luminance value X ⁇ 1 of polarized light in a first wavelength band ⁇ 1 having a first polarization angle ⁇ 1 (hereinafter referred to as the “first wavelength band polarized light”), a luminance value X ⁇ 2 of polarized light in a second wavelength band ⁇ 2 having a second polarization angle ⁇ 2 (hereinafter referred to as the “second wavelength band polarized light”), and a luminance value X ⁇ 3 of polarized light in a third wavelength band ⁇ 3 having a third polarization angle ⁇ 3 (hereinafter referred to as the “third wavelength band polarized light”).
• the pixel value X of each image pixel is expressed by equation (2).
• Interference matrix A is the interference matrix.
• Interference matrix A (not shown) is a matrix that indicates the characteristics of interference.
• Interference matrix A is determined in advance based on multiple known values such as the spectrum of the incident light, the spectral transmittance of the first lens 30, the spectral transmittance of the second lens 32, the spectral transmittances of the multiple spectral filters 20, and the spectral sensitivity of the image sensor 28.
• the interference cancellation matrix A + is a matrix defined based on the spectrum of the incident light, the spectral transmittance of the first lens 30, the spectral transmittance of the second lens 32, the spectral transmittances of the multiple spectral filters 20, and the spectral sensitivity of the image sensor 28.
• the interference cancellation matrix A + is stored in advance in the storage 62.
• the interference cancellation processing unit 84 acquires the interference cancellation matrix A + stored in the storage 62 and the output value Y of each physical pixel 44 acquired by the output value acquisition unit 82, and outputs the pixel value X of each image pixel according to equation (4) based on the acquired interference cancellation matrix A + and the output value Y of each physical pixel 44.
• the pixel value X of each image pixel includes, as its components, a luminance value X ⁇ 1 of the first wavelength band polarized light, a luminance value X ⁇ 2 of the second wavelength band polarized light, and a luminance value X ⁇ 3 of the third wavelength band polarized light.
• the spectral image 72A of the captured image 70 corresponds to the luminance value X ⁇ 1 of the light in the first wavelength band ⁇ 1 (i.e., an image based on the luminance value X ⁇ 1 ).
• the spectral image 72B of the captured image 70 corresponds to the luminance value X ⁇ 2 of the light in the second wavelength band ⁇ 2 (i.e., an image based on the luminance value X ⁇ 2 ).
• the spectral image 72C of the captured image 70 corresponds to the luminance value X ⁇ 3 of the light in the third wavelength band ⁇ 3 (i.e., an image based on the luminance value X ⁇ 3 ).
• the interference removal process is performed by the interference removal processor 84, whereby the captured image 70 is separated into a spectral image 72A corresponding to the luminance value X ⁇ 1 of the first wavelength band polarized light, a spectral image 72B corresponding to the luminance value X ⁇ 2 of the second wavelength band polarized light, and a spectral image 72C corresponding to the luminance value X ⁇ 3 of the third wavelength band polarized light. That is, the captured image 70 is separated into spectral images 72 for each wavelength band ⁇ of the multiple spectral filters 20.
• the processing device 90 includes a computer 92.
• the computer 92 includes a processor 94, storage 96, and RAM 98.
• the processor 94, storage 96, and RAM 98 are realized by hardware similar to the processor 60, storage 62, and RAM 64 described above (see FIG. 4).
• the processor 94 is an example of a "processor" according to the technology of the present disclosure.
• the storage 96 stores a measurement program 100.
• the measurement program 100 is an example of a "program" according to the technology of the present disclosure.
• the processor 94 reads the measurement program 100 from the storage 96 and executes the read measurement program 100 on the RAM 98.
• the processor 94 executes a measurement process according to the measurement program 100 executed on the RAM 98.
• the measurement process is realized by the processor 94 operating as an image data acquisition unit 102, an image area setting unit 104, an area derivation unit 106, an area determination unit 108, a boundary setting unit 110, a reference image area setting unit 112, an average value derivation unit 114, an evaluation image area setting unit 116, an evaluation value derivation unit 118, and an evaluation result output unit 120 according to the measurement program 100.
• the first surface 202A and the second surface 202B of the subject 200 are irradiated onto the first surface 202A and the second surface 202B of the subject 200 by the first light source 132A and the second light source 132B.
• the first surface 202A and the second surface 202B are irradiated with light under different conditions.
• the light intensity of the first light source 132A located on the first surface 202A side is set to be higher than the light intensity of the second light source 132B located on the second surface 202B side.
• the intensity of the light irradiated onto the first surface 202A is higher than the intensity of the light irradiated onto the second surface 202B.
• the imaging device 10 captures an image of the subject 200 while the first surface 202A and the second surface 202B are irradiated with light from the first light source 132A and the second light source 132B.
• the imaging device 10 then transmits imaging data obtained by capturing an image of the subject 200 to the processing device 90.
• the imaging data is image data representing multiple spectral images.
• the imaging data is an example of the "first imaging data" and "second imaging data" related to the technology of the present disclosure.
• the imaging data acquisition unit 102 acquires imaging data received by the processing device 90.
• the image area setting unit 104 selects two spectral images 72 from the multiple spectral images 72 included in the imaging data acquired by the imaging data acquisition unit 102.
• the two spectral images 72 are selected based on an image selection instruction given to the processing device 90 by, for example, a user or the like.
• a spectral image 72A corresponding to a first wavelength band ⁇ 1 and a spectral image 72B corresponding to a second wavelength band ⁇ 2 are selected as two spectral images 72.
• the spectral images 72A and 72B include an object image 140 corresponding to an object 200.
• the spectral image 72A is an example of a "first wavelength band image” according to the technology of the present disclosure.
• the spectral image 72B is an example of a "first wavelength band image” according to the technology of the present disclosure.
• the image region setting unit 104 divides the image region of the subject image 140 based on the luminance distribution of the image pixels for each selected spectral image 72.
• the image region setting unit 104 detects the luminance distribution of the image pixels for the spectral image 72A, and divides the subject image 140 into a first image region 142A whose luminance is equal to or greater than a preset value, and a second image region 142B whose luminance is less than the preset value.
• the image area setting unit 104 detects the luminance distribution of image pixels for the spectral image 72B, and divides the subject image 140 into a first image area 142A whose luminance is equal to or greater than a preset value, and a second image area 142B whose luminance is less than the preset value.
• the preset value is set to a luminance corresponding to the median value of the difference between the light intensity of the first light source 132A and the light intensity of the second light source 132B, for example.
• the first image area 142A is an image area corresponding to the first surface 202A
• the second image area 142B is an image area corresponding to the second surface 202B.
• the size of the first image area 142A and the second image area 142B differs for each spectral image 72 because the spectral filters 20 corresponding to each spectral image 72 are provided in openings 24 formed at positions different from the optical axis OA, causing parallax between the multiple spectral filters 20.
• a position shift occurs due to parallax in the optical image formed on the light receiving surface 34A for each wavelength band ⁇ .
• the area derivation unit 106 derives a first area A1, which is the area of the first image region 142A, based on one of the two selected spectral images 72.
• the area derivation unit 106 also derives a second area A2, which is the area of the first image region 142A, based on the other of the two selected spectral images 72.
• the area derivation unit 106 derives the first area A1, which is the area of the first image region 142A, based on the spectral image 72A.
• the area derivation unit 106 derives the second area A2, which is the area of the first image region 142A, based on the spectral image 72B.
• the first area A1 is an area derived based on the luminance distribution of the spectral image 72A
• the second area A2 is an area derived based on the luminance distribution of the spectral image 72B.
• the first area A1 is an example of a "first area” according to the technology of the present disclosure.
• the second area A2 is an example of a "second area” according to the technology of the present disclosure.
• the area determination unit 108 determines whether the first area A1 and the second area A2 derived by the area derivation unit 106 are different. In other words, the area determination unit 108 determines whether the area derivation unit 106 has obtained a calculation result that the first area A1 and the second area A2 are different.
• the calculation result is an example of a "calculation result" related to the technology of the present disclosure.
• the area derivation unit 106 obtains a calculation result that the first area A1 and the second area A2 are different.
• the first area A1 derived based on the spectral image 72A and the second area A2 derived based on the spectral image 72B will be the same. Therefore, when the first surface 202A and the second surface 202B do not have an angle, the area derivation unit 106 obtains a calculation result that the first area A1 and the second area A2 are the same.
• the first surface 202A and the second surface 202B have an angle, so in the example shown in FIG. 10, the first area A1 derived based on the spectral image 72A is different from the second area A2 derived based on the spectral image 72B.
• the first area A1 may be the area of the second image region 142B derived based on the spectral image 72A.
• the second area A2 may be the area of the second image region 142B derived based on the spectral image 72B.
• the area determination unit 108 may determine whether the first area A1, which is the area of the second image region 142B derived based on the spectral image 72A, differs from the second area A2, which is the area of the second image region 142B derived based on the spectral image 72B.
• the boundary setting unit 110 selects one of the spectral images 72 obtained by dividing the image area of the subject image 140 by the image area setting unit 104 (see FIG. 9).
• any of the spectral images 72 is selected based on an image selection instruction given to the processing device 90 by, for example, a user or the like.
• spectral image 72A is selected.
• spectral image 72B may be selected, and thereafter, a process similar to that for spectral image 72A may be performed on spectral image 72B.
• the boundary setting unit 110 sets a boundary 144 between the first image area 142A and the second image area 142B for the subject image 140 included in any of the selected spectral images 72.
• the boundary 144 is a line corresponding to the ridge line 204 between the first surface 202A and the second surface 202B.
• the ridge line 204 is an example of a "boundary" according to the technology of the present disclosure.
• the reference image area setting unit 112 sets the image area on the boundary 144 side of the first image area 142A as the first reference image area 146A, and sets the image area on the boundary 144 side of the second image area 142B as the second reference image area 146B. The reference image area setting unit 112 then sets a reference image area 146 including the first reference image area 146A and the second reference image area 146B for the subject image 140.
• the first reference image area 146A is an image area corresponding to the first reference area 206A on the edge 204 side of the first surface 202A
• the second reference image area 146B is an image area corresponding to the second reference area 206B on the edge 204 side of the second surface 202B.
• the first reference area 206A and the second reference area 206B form the reference area 206.
• the reference image area 146 is an image area corresponding to the reference area 206.
• the reference area 206 is an area including the first reference area 206A and the second reference area 206B, and is an area that spans the first surface 202A and the second surface 202B of the subject 200.
• the reference area 206 is an area in the first surface 202A and the second surface 202B where there is little difference between the way the light irradiated from the first light source 132A hits the object and the way the light irradiated from the second light source 132B hits the object.
• the reference imaging data shown in FIG. 11 is imaging data representing the reference image area 146, and is imaging data in the reference area 206.
• the reference image area setting unit 112 sets the reference image area 146, and the reference imaging data is obtained from the imaging data (see FIG. 9).
• the reference imaging data is an example of "reference imaging data" related to the technology of the present disclosure.
• the average value derivation unit 114 derives the average value S(REF) ave of the luminance of the image pixels included in the reference image region 146 by equation (5), where S(REF) a is the average value of the luminance of the image pixels included in the first reference image region 146A, and S(REF) b is the average value of the luminance of the image pixels included in the second reference image region 146B.
• the evaluation image area setting unit 116 sets the image area of the first image area 142A other than the first reference image area 146A as the first evaluation image area 148A for the subject image 140, and sets the image area of the second image area 142B other than the second reference image area 146B as the second evaluation image area 148B.
• the evaluation value derivation unit 118 derives an evaluation value S(EVE) a+ for each image pixel included in the first evaluation image area 148A based on S(EVE) a using formula ( 6 ), where S(EVE) a is the luminance value for each image pixel included in the first evaluation image area 148A.
• the evaluation value derivation unit 118 derives an evaluation value S(EVE) b+ for each image pixel included in the second evaluation image area 148B based on S(EVE) b using equation (7), where S(EVE) b is the luminance value of each image pixel included in the second evaluation image area 148B.
• the evaluation value derivation unit 118 derives the evaluation value S(EVE) a+ for each image pixel included in the first evaluation image region 148A, thereby correcting the luminance value S(EVE) a for each image pixel included in the first evaluation image region 148A.
• the evaluation value derivation unit 118 derives the evaluation value S(EVE) b+ for each image pixel included in the second evaluation image region 148B, thereby correcting the luminance value S(EVE) b for each image pixel included in the second evaluation image region 148B.
• first evaluation image area 148A is an image area corresponding to first evaluation area 208A of subject 200, and luminance value S(EVE) a corresponds to the spectral reflectance of first evaluation area 208A.
• First evaluation area 208A is an area of first surface 202A other than first reference area 206A.
• second evaluation image area 148B is an image area corresponding to second evaluation area 208B of subject 200, and luminance value S(EVE) b corresponds to the spectral reflectance of second evaluation area 208B.
• Second evaluation area 208B is an area of second surface 202B other than second reference area 206B.
• the average value S(REF) ave corresponds to the average value of the spectral reflectance in the reference region 206.
• the reference imaging data includes data on the average value of the spectral reflectance in the reference region 206
• the imaging data includes data on the non-uniformity of the spectral reflectance in the first evaluation region 208A and data on the non-uniformity of the spectral reflectance in the second evaluation region 208B.
• the evaluation value derivation unit 118 corrects the luminance value S(EVE) a included in the first evaluation image region 148A based on the average value S(REF) ave , thereby correcting data related to the non-uniformity of the spectral reflectance of the first evaluation region 208A based on the reference imaging data.
• the evaluation value derivation unit 118 corrects the luminance value S(EVE) b included in the second evaluation image region 148B based on the average value S(REF) ave , thereby correcting data related to the non-uniformity of the spectral reflectance of the second evaluation region 208B based on the reference imaging data.
• non-uniformity of the spectral reflectance refers to unevenness in the spectral reflectance.
• the evaluation result output unit 120 outputs a first evaluation result 126A which is a determination result of whether or not the evaluation value S(EVE) a+ derived for each image pixel included in the first evaluation image area 148A falls within a first predetermined range.
• the first predetermined range is set to a range within which a user or the like can visually evaluate that there is no unevenness in the spectral reflectance of the first surface 202A of the subject 200.
• the first evaluation result 126A is an evaluation result regarding the degree of unevenness in the spectral reflectance of the first evaluation area 208A.
• the evaluation result output unit 120 outputs a second evaluation result 126B which is a determination result of whether or not the evaluation value S(EVE) b+ derived for each image pixel included in the second evaluation image region 148B falls within a second predetermined range.
• the second predetermined range is set to a range in which a user or the like can visually evaluate that there is no unevenness in the spectral reflectance of the second surface 202B of the subject 200.
• the second evaluation result 126B is an evaluation result regarding the degree of unevenness in the spectral reflectance of the second evaluation region 208B.
• the first evaluation result 126A and the second evaluation result 126B are included in the measurement result 124, and the measurement result 124 including the first evaluation result 126A and the second evaluation result 126B is displayed, for example, on the display device 122 (see FIG. 1).
• Figure 14 shows an example of the flow of the spectral image generation process according to this embodiment.
• step ST10 the output value acquisition unit 82 acquires the output value Y of each physical pixel 44 based on the imaging data output from the image sensor 28 (see FIG. 7). After the process of step ST10 is executed, the spectral image generation process proceeds to step ST12.
• step ST12 the interference elimination processing unit 84 acquires the interference elimination matrix A + stored in the storage 62 and the output value Y of each physical pixel 44 acquired in step ST10, and outputs the pixel value X of each image pixel based on the acquired interference elimination matrix A + and the output value Y of each physical pixel 44 (see FIG. 7).
• the captured image 70 is separated into a spectral image 72A corresponding to the luminance value X ⁇ 1 of the first wavelength band polarized light, a spectral image 72B corresponding to the luminance value X ⁇ 2 of the second wavelength band polarized light, and a spectral image 72C corresponding to the luminance value X ⁇ 3 of the third wavelength band polarized light.
• the spectral image generation process ends.
• Figure 15 shows an example of the flow of the measurement process according to this embodiment.
• step ST20 the imaging data acquisition unit 102 acquires imaging data received by the processing device 90 (see FIG. 9). After the process of step ST20 is executed, the measurement process proceeds to step ST22.
• step ST22 the image area setting unit 104 selects two of the multiple spectral images 72 included in the imaging data acquired in step ST20. Then, for each selected spectral image 72, the image area setting unit 104 divides the subject image 140 into a first image area 142A and a second image area 142B based on the luminance distribution of the image pixels (see FIG. 9). After the processing of step ST22 is executed, the measurement processing proceeds to step ST24.
• step ST24 the area derivation unit 106 derives a first area A1, which is the area of the first image region 142A, based on one of the two spectral images 72 selected in step ST22.
• the area derivation unit 106 also derives a second area A2, which is the area of the first image region 142A, based on the other of the two spectral images 72 selected in step ST22 (see FIG. 10).
• step ST26 the area determination unit 108 determines whether the first area A1 and the second area A2 derived in step ST24 are different (see FIG. 10). If the first area A1 and the second area A2 are different, the determination is affirmative, and the measurement process proceeds to step ST28. If the first area A1 and the second area A2 are the same, the determination is negative, and the measurement process ends.
• step ST28 the boundary setting unit 110 selects one of the two spectral images 72 selected in step ST22. Then, the boundary setting unit 110 sets a boundary 144 between the first image area 142A and the second image area 142B for the subject image 140 included in one of the selected spectral images 72 (see FIG. 11). After the processing of step ST28 is performed, the measurement processing proceeds to step ST30.
• step ST30 the reference image area setting unit 112 sets the image area on the boundary 144 side of the first image area 142A as the first reference image area 146A, and sets the image area on the boundary 144 side of the second image area 142B as the second reference image area 146B. Then, the reference image area setting unit 112 sets a reference image area 146 including the first reference image area 146A and the second reference image area 146B for the subject image 140 (see FIG. 11). After the processing of step ST30 is executed, the measurement processing proceeds to step ST32.
• step ST32 average value derivation section 114 derives average value S(REF) ave of luminance of image pixels included in reference image region 146 (see FIG. 12). After the process of step ST30 is executed, the measurement process proceeds to step ST34.
• step ST34 the evaluation image area setting unit 116 sets the image area of the first image area 142A other than the first reference image area 146A as the first evaluation image area 148A for the subject image 140, and sets the image area of the second image area 142B other than the second reference image area 146B as the second evaluation image area 148B (see FIG. 12).
• step ST34 the measurement processing proceeds to step ST36.
• step ST36 the evaluation value derivation unit 118 derives an evaluation value S(EVE) a+ for each image pixel included in the first evaluation image region 148A. Similarly, the evaluation value derivation unit 118 derives an evaluation value S(EVE) b+ for each image pixel included in the second evaluation image region 148B using equation (7) (see FIG. 13). After the process of step ST36 is executed, the measurement process proceeds to step ST38.
• step ST38 the evaluation result output unit 120 outputs a first evaluation result 126A which is a result of the determination as to whether or not the evaluation value S(EVE) a+ derived for each image pixel included in the first evaluation area 208A falls within a first predetermined range.
• the evaluation result output unit 120 outputs a second evaluation result 126B which is a result of the determination as to whether or not the evaluation value S(EVE) b+ derived for each image pixel included in the second evaluation area 208B falls within a second predetermined range (see FIG. 13).
• the processor 94 acquires imaging data obtained by imaging the subject 200 having the first surface 202A and the second surface 202B using the imaging device 10, and acquires reference imaging data in the reference region 206 spanning the first surface 202A and the second surface 202B from the imaging data.
• the processor 94 then corrects the imaging data based on the reference imaging data. Therefore, the measurement accuracy can be improved compared to the case where the spectral reflectance of the subject 200 is measured by using the imaging data as is.
• the imaging data obtained by imaging the subject 200 with the imaging device 10 is corrected based on the reference imaging data. Therefore, the imaging data obtained each time the subject 200 is imaged with the imaging device can be corrected.
• correcting the imaging data includes correcting data related to the non-uniformity of the spectral reflectance of the subject 200. Therefore, the measurement accuracy can be improved compared to measuring the non-uniformity of the spectral reflectance of the subject 200 by using the imaging data as is.
• correcting the imaging data includes deriving an average value of the spectral reflectance in the reference area 206, and correcting data regarding the non-uniformity of the spectral reflectance of the first evaluation area 208A, which is different from the reference area 206 of the first surface 202A, based on the average value. Therefore, it is possible to improve the measurement accuracy compared to measuring the non-uniformity of the spectral reflectance of the first evaluation area 208A by using the imaging data as is.
• correcting the imaging data includes correcting data regarding the non-uniformity of the spectral reflectance of the second evaluation area 208B, which is different from the reference area 206 of the second surface 202B, based on the average value. Therefore, it is possible to improve the measurement accuracy compared to the case where the non-uniformity of the spectral reflectance of the second evaluation area 208B is measured by using the imaging data as is.
• the processor 94 also outputs a first evaluation result 126A relating to the degree of unevenness in the spectral reflectance of the first evaluation area 208A obtained by the correction. Therefore, the degree of unevenness in the spectral reflectance of the first evaluation area 208A can be provided to a user, etc.
• the processor 94 also outputs a second evaluation result 126B relating to the degree of unevenness in the spectral reflectance of the second evaluation area 208B obtained by the correction. Therefore, the degree of unevenness in the spectral reflectance of the second evaluation area 208B can be provided to a user, etc.
• the center of gravity of the multiple spectral filters 20 is located at a position different from the optical axis OA. Therefore, by utilizing the parallax occurring between the multiple spectral filters 20 to determine whether the first area A1 and the second area A2 are different, it is possible to determine whether the first surface 202A and the second surface 202B have an angle.
• the processor 94 acquires reference imaging data when a calculation result is obtained based on the imaging data that indicates that the first surface 202A and the second surface 202B have an angle. This eliminates the need to execute a process to acquire reference imaging data even when the first surface 202A and the second surface 202B do not have an angle.
• the calculation result is that the first area A1 derived based on one of the two selected spectral images 72 is different from the second area A2 derived based on the other of the two selected spectral images 72. Therefore, it is possible to determine whether the first surface 202A and the second surface 202B have an angle based on the two selected spectral images 72.
• the first area A1 is an area derived based on the luminance distribution of one of the two selected spectral images 72
• the second area A2 is an area derived based on the luminance distribution of the other of the two selected spectral images 72. Therefore, the first area A1 and the second area A2 can be derived based on the spectral reflectance of the subject 200.
• the reference area 206 is an area that is determined based on the boundary 144 between the first surface 202A and the second surface 202B. Therefore, the area of the first surface 202A and the second surface 202B where there is little difference between the way the light irradiated from the first light source 132A hits and the way the light irradiated from the second light source 132B hits can be set as the reference area 206.
• the subject image 140 can be divided into a first image region 142A and a second image region 142B based on the luminance distribution of the image pixels.
• the imaging device 10 is a multispectral camera. Therefore, multiple spectral images 72 can be obtained in one imaging session.
• the imaging data obtained by imaging the subject 200 with the imaging device 10 is corrected based on the reference imaging data.
• the imaging data may be used as first imaging data for obtaining the reference imaging data.
• second imaging data obtained separately from the first imaging data i.e., imaging data different from the first imaging data
• imaging data may be corrected based on the reference imaging data.
• the subject 200 is a triangular prism, but it may be an object other than a triangular prism as long as it has a first surface 202A and a second surface 202B.
• the first surface 202A and the second surface 202B may be selected from the multiple surfaces by a user or the like.
• first surface 202A and the second surface 202B are flat surfaces, but may be curved surfaces. Furthermore, if the first surface 202A and the second surface 202B are curved surfaces, they may be approximated to flat surfaces.
• the processor 60 is exemplified for the imaging device 10, but at least one other CPU, at least one GPU, and/or at least one TPU may be used in place of the processor 60 or together with the processor 60.
• the processor 94 is exemplified as the processing device 90, but instead of the processor 94 or together with the processor 94, at least one other CPU, at least one GPU, and/or at least one TPU may be used.
• the imaging device 10 has been described with an example in which the spectral image generation program 80 is stored in the storage 62, but the technology of the present disclosure is not limited to this.
• the spectral image generation program 80 may be stored in a portable non-transient computer-readable storage medium (hereinafter simply referred to as a "non-transient storage medium") such as an SSD or USB memory.
• the spectral image generation program 80 stored in the non-transient storage medium may be installed in the computer 56 of the imaging device 10.
• the spectral image generation program 80 may be stored in a storage device such as another computer or server device connected to the imaging device 10 via a network, and the spectral image generation program 80 may be downloaded in response to a request from the imaging device 10 and installed in the computer 56 of the imaging device 10.
• spectral image generation program 80 it is not necessary to store the entire spectral image generation program 80 in a storage device such as another computer or server device connected to the imaging device 10, or in the storage 62; only a part of the spectral image generation program 80 may be stored.
• the processing device 90 has been described with an example in which the measurement program 100 is stored in the storage 96, but the technology of the present disclosure is not limited to this.
• the measurement program 100 may be stored in a non-transitory storage medium.
• the measurement program 100 stored in the non-transitory storage medium may be installed in the computer 92 of the processing device 90.
• the measurement program 100 may be stored in a storage device such as another computer or server device connected to the processing device 90 via a network, and the measurement program 100 may be downloaded in response to a request from the processing device 90 and installed in the computer 92 of the processing device 90.
• the entire measurement program 100 in a storage device such as another computer or server device connected to the processing device 90, or in the storage 96; only a portion of the measurement program 100 may be stored therein.
• the imaging device 10 has a built-in computer 56
• the technology of the present disclosure is not limited to this, and for example, the computer 56 may be provided outside the imaging device 10.
• processing device 90 has a built-in computer 92
• the technology disclosed herein is not limited to this, and for example, the computer 92 may be provided outside the processing device 90.
• a computer 56 including a processor 60, storage 62, and RAM 64 is exemplified for the imaging device 10, but the technology of the present disclosure is not limited to this, and a device including an ASIC, FPGA, and/or PLD may be applied instead of the computer 56. Also, a combination of a hardware configuration and a software configuration may be used instead of the computer 56.
• a computer 92 including a processor 94, storage 96, and RAM 98 is exemplified as the processing device 90, but the technology of the present disclosure is not limited to this, and a device including an ASIC, FPGA, and/or PLD may be applied instead of the computer 92. Also, a combination of a hardware configuration and a software configuration may be used instead of the computer 92.
• processors listed below can be used as hardware resources for executing the various processes described in the above embodiment.
• An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing various processes by executing software, i.e., a program.
• Another example of a processor is a dedicated electronic circuit, which is a processor with a circuit configuration designed specifically for executing a specific process, such as an FPGA, PLD, or ASIC. All of the processors have built-in or connected memory, and all of the processors use the memory to execute various processes.
• the hardware resources that execute the various processes may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resources that execute the various processes may be a single processor.
• a configuration using a single processor first, there is a configuration in which one processor is configured by combining one or more CPUs with software, and this processor functions as a hardware resource that executes various processes. Secondly, there is a configuration in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute various processes, on a single IC chip, as typified by SoCs. In this way, various processes are realized using one or more of the above-mentioned various processors as hardware resources.
• these various processors can be electronic circuits that combine circuit elements such as semiconductor elements.
• the above gaze detection process is merely one example. It goes without saying that unnecessary steps can be deleted, new steps can be added, and the processing order can be changed without departing from the spirit of the invention.
• a and/or B is synonymous with “at least one of A and B.”
• a and/or B means that it may be just A, or just B, or a combination of A and B.
• the same concept as “A and/or B” is also applied when three or more things are expressed by linking them with “and/or.”

Landscapes

• Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Spectroscopy & Molecular Physics (AREA)
• General Physics & Mathematics (AREA)
• Chemical & Material Sciences (AREA)
• Analytical Chemistry (AREA)
• Biochemistry (AREA)
• General Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Immunology (AREA)
• Pathology (AREA)
• Health & Medical Sciences (AREA)
• Investigating Or Analysing Materials By Optical Means (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=90830564

Family Applications (1)

Country Status (2)

Citations (3)

• 2023
  • 2023-09-28 WO PCT/JP2023/035530 patent/WO2024090133A1/ja not_active Ceased
  • 2023-09-28 JP JP2024552904A patent/JPWO2024090133A1/ja active Pending

Patent Citations (3)

Also Published As

Similar Documents

Legal Events