JP2022007978A - Control method for spectral imaging device, spectral imaging device, computer program, control method for display system, control method for projector, display system, and projector - Google Patents

Abstract

To provide a control method for a spectral imaging device, the spectral imaging device, and a computer program with which light in a predetermined wavelength can be efficiently taken out.SOLUTION: A control method for a spectral imaging device comprising an image pick-up device and a spectroscopic element, and the method includes, when the spectral imaging device is in a high accuracy mode, creating, with the spectral imaging device, a first measurement spectrum consisting of N1 wavelengths obtained by picking up an image of an object with different output wavelengths of the spectroscopic element, and when the spectral imaging device is in a high speed mode, creating, with the spectral imaging device, a second measurement spectrum consisting of N2 wavelengths obtained by picking up an image of the object with different output wavelengths of the spectroscopic element. N1 is an integer of 2 or more, and N2 is an integer smaller than N1.SELECTED DRAWING: Figure 4

Description

The present invention relates to a control method for a spectroscopic image pickup device, a spectroscopic image pickup device, a computer program, a control method for a display system, a control method for a projector, a display system, and a projector.

For example, Patent Document 1 discloses a technique of capturing an image of an object to be inspected and correcting the image based on the imaging result. Further, Patent Document 2 describes that the projected image of the projector is sequentially imaged while switching the filters of the bands corresponding to the plurality of primary colors, and the correction data is calculated based on the imaging result. With these techniques, the more images in different wavelength ranges are captured, the easier it is to obtain highly accurate correction data, but the longer the time required for imaging. In particular, in the case of a spectroscopic image pickup device (also referred to as a spectroscopic camera), it is necessary to secure a predetermined exposure time in order to obtain a high S / N ratio in each pixel of the image sensor, so that the time required for image correction tends to be long.

Japanese Unexamined Patent Publication No. 2011-24842 Japanese Unexamined Patent Publication No. 2005-20581

However, depending on the usage scene, it is not always necessary to obtain a high S / N ratio, and if high-precision correction data is not required, such as when you want to perform a simple check, you may only need to make a rough measurement in a short time. be. That is, there is a demand for a spectroscopic imaging device having both high-precision measurement and low-precision measurement.

The control method of the spectroscopic image pickup device includes an image pickup element and a spectroscopic element, and when the spectroscopic image pickup device is in the first mode, the spectroscopic image pickup device is used to image an object with different output wavelengths of the spectroscopic element. When the spectroscopic image pickup device is in the second mode, the first measurement spectrum consisting of the wavelengths is generated, and when the spectroscopic image pickup device is in the second mode, the output wavelengths of the spectroscopic elements are made different by the spectroscopic image pickup device to image the object. Generates a second measurement spectrum consisting of wavelengths, where N1 is an integer greater than or equal to 2 and N2 is an integer smaller than N1.

The spectroscopic image pickup device includes an image pickup element and a spectroscopic element, and when the spectroscopic image pickup device is in the first mode, the spectroscopic image pickup device has different output wavelengths of the spectroscopic element to image an object. When the spectroscopic image pickup device is in the second mode, the second measurement of N2 pieces in which the object is imaged by different output wavelengths of the spectroscopic element by the spectroscopic image pickup device. Generates a spectrum, where N1 is an integer greater than or equal to 2 and N2 is an integer smaller than N1.

The computer program is a computer program for identifying an object based on image pickup data by a spectroscopic image pickup device including an image pickup element and a spectroscopic element, and the output wavelength is different in the first mode of the spectroscopic image pickup device. In the process of imaging an object and generating a first measurement spectrum consisting of N1 wavelengths, which are integers of 2 or more, and in the second mode of the spectroscopic imaging device, the object is imaged with different output wavelengths. , A process of generating a second measurement spectrum consisting of N2 wavelengths, which is an integer smaller than N1, and at least one of them is performed by a computer.

Further, one aspect of solving the above-mentioned problems is a control method of a display system including an image pickup element, a spectroscopic image pickup device including a spectroscopic element, and a projector that projects a projected image based on image data onto a projection surface. When the system is in the first mode, the projection image is imaged by the spectroscopic image pickup device with different spectral wavelengths of the spectroscopic elements to generate N1 first image pickup data, and the projector is used to generate the N1 first image pickup data. The first corrected image data obtained by correcting the first image data is generated based on the first captured image data, and the projector projects the first projected image based on the first corrected image data onto the projection surface and displays the display. When the system is in the second mode, the projection image is imaged by the spectroscopic image pickup device with different spectral wavelengths of the spectroscopic elements to generate N2 second image pickup data, and the projector is used to generate the N2 second image pickup data. Based on the second image pickup data, the second corrected image data obtained by correcting the first image data is generated, and the projector projects the second projected image based on the second corrected image data onto the projection surface. Including, N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

In the control method of the display system, the first corrected image data is data in which predetermined information regarding color is measured based on the N1 first image pickup data and the first image data is corrected based on the measurement result. You may.

In the control method of the display system, in the second mode, the predetermined measurement result obtained from the N2 second imaging data is obtained from the N1 first imaging data based on the conversion data. The data obtained by converting the information corresponding to the predetermined measurement result and correcting the first image data based on the converted information may be used as the second corrected image data.

In the control method of the display system, the projector may project a projected image including an OSD menu including the options of the first mode and the second mode.

In the control method of the display system, the spectroscopic element has a pair of reflective films and a gap changing portion capable of changing the gap size of the pair of reflective films, and is placed on an optical path of light incident on the image pickup element. It may be an arranged wavelength variable interference filter.

Another aspect that solves the above problems is a method of controlling a projector that projects a projected image based on image data onto a projection surface, and when the projector is in the first mode, the projected image is captured by a spectroscopic image pickup device. The first correction, in which the first image data of N1 images taken with different spectral wavelengths of the spectral elements of the spectroscopic image pickup device is acquired and the first image data is corrected based on the first image pickup data of the N1 images. Image data is generated, a first projected image based on the first corrected image data is projected onto the projection surface, and when the projector is in the second mode, the projected image is captured by the spectral imaging device by the spectral imaging device. The second image data of N2 images taken with different spectral wavelengths of the spectroscopic elements of the above is acquired, and the second corrected image data obtained by correcting the first image data is generated based on the second image data of the N2 images. Then, including projecting a second projected image based on the second corrected image data onto the projection surface, N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

Another aspect of solving the above problems is a display system including an image pickup element, a spectroscopic image pickup device including a spectroscopic element, and a projector that projects a projected image based on image data onto a projection surface, wherein the display system includes a display system. In one mode, the spectroscopic image pickup device generates N1 first image pickup data obtained by capturing the projected image with different spectral wavelengths of the spectroscopic elements, and the projector is the N1 first image pickup image. Based on the captured data, the first corrected image data corrected by the first image data is generated, the first projected image based on the first corrected image data is projected on the projection surface, and the display system is in the second mode. At one point, the spectroscopic imaging device generates N2 second imaging data obtained by imaging the projected image at different spectral wavelengths of the spectroscopic element, and the projector uses the N2 second imaging data. Based on this, a second corrected image data obtained by correcting the first image data is generated, and a second projected image based on the second corrected image data is projected onto the projection surface, in which N1 is an integer of 2 or more. Yes, N2 is an integer smaller than N1.

Another aspect that solves the above problems is a projector that projects a projected image based on image data onto a projection surface, and when the projector is in the first mode, the projected image is subjected to the spectroscopy by a spectroscopic image pickup device. The first corrected image data obtained by acquiring the first image pickup data of N1 images taken with different spectral wavelengths of the spectroscopic elements of the image pickup apparatus and correcting the first image data based on the first image pickup data of the N1 images. A first projected image generated and based on the first corrected image data is projected onto the projection surface, and when the projector is in the second mode, the projected image is captured by the spectral image pickup device and the spectral element of the spectral image pickup device. The second image data of N2 images taken with different spectral wavelengths is acquired, and the second corrected image data obtained by correcting the first image data is generated based on the second image data of N2 images. Including projecting a second projected image based on the second corrected image data onto the projection surface, N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

The block diagram which shows the structure of the spectroscopic image pickup apparatus of 1st Embodiment. The cross-sectional view which shows the structure of the image pickup apparatus. The figure which shows the specific example of the operation mode. A flowchart showing a procedure for determining freshness in a high-precision mode. A flowchart showing a procedure for determining freshness in a high-speed mode. A flowchart showing a procedure for determining freshness in the optimum mode. A graph showing the relationship between the wavelength of light and the reflectance when the freshness is different. The cross-sectional view which shows the structure of the spectroscopic element of the modification. The block diagram which shows the structure of the display system of 2nd Embodiment. Schematic diagram of the spectroscopic imaging unit. The figure which shows the specific example of a high precision mode and a high speed mode. A flowchart showing the operation of the display system. The figure which changed the configuration of the display system. The block diagram which shows the structure of the display system of 3rd Embodiment. The figure which changed the configuration of the display system.

[First Embodiment]
Hereinafter, embodiments will be described with reference to the accompanying drawings.
First, the configuration of the spectroscopic imaging device 400A will be described with reference to FIG.

As shown in FIG. 1, the spectroscopic image pickup apparatus 400A is for determining, for example, the freshness of a green vegetable (for example, spinach, Japanese mustard spinach, green pepper, etc.) which is an object T. The spectroscopic image pickup device 400A includes an image pickup device 10, a display device 30, a storage device 40, and a processing device 50.

The image pickup device 10 includes an incident optical system 11, a spectroscopic element 12, an image pickup element 13, and a light source unit 14. The incident optical system 11 includes, for example, an autofocus mechanism. The spectroscopic element 12 is, for example, a wavelength selection filter, and a Fabry-Perot type filter capable of changing the transmission wavelength band is used.

The image pickup element 13 includes a first image pickup element and a second image pickup element (not shown). The first image pickup device is a CCD (Charge Coupled Device), which is an image pickup device that obtains an electric signal representing an object by photoelectrically converting the light transmitted through the spectroscopic element 12. The second image sensor is, for example, CCDAFE (Analog Front End), and is for digitizing the detection signal of the first image sensor. The light source unit 14 is for irradiating the object T.

In the image pickup apparatus 10, the transmission wavelength range of the spectroscopic element 12 is sequentially changed by sequentially receiving instructions of a plurality of measurement bands (multi-band) from the processing apparatus 50 in the spectroscopic element 12. In this way, the image pickup apparatus 10 takes an image of the object T with sensitivities in a plurality of wavelength bands.

The display device 30 is a device for displaying information on the screen. The storage device 40 is an external device for storing data, for example, a hard disk drive device.

The processing device 50 is a device that determines the freshness of the object T by processing the image pickup data obtained by imaging with the image pickup device 10. The processing device 50 includes a control unit 60 and a processing unit 70 that function as a computer, and a storage unit 80.

The control unit 60 is configured to include one or a plurality of processors, and for example, by operating according to a control program stored in the storage unit 80, the operation of the spectroscopic image pickup apparatus 400A is collectively controlled.

The processing unit 70 performs various processes by executing a control program as a computer program. The storage unit 80 includes a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The RAM is used for temporary storage of various data and the like, and the ROM stores a control program, control data and the like for controlling the operation of the spectroscopic image pickup apparatus 400A.

The storage unit 80 includes a measurement band data storage unit 81, a setting data storage unit 82, and a conversion data storage unit 83. Further, although not shown, the storage unit 80 stores a freshness determination program. The data stored in these storage units 80 will be described in detail later.

The processing unit 70 executes the processing of the measurement band indicating unit 71 by executing the freshness determination program stored in the storage unit 80. The processing unit 70 executes each process using the data and parameters stored in the corresponding storage unit 80.

The image pickup control unit 61 of the control unit 60 causes the image pickup device 10 to perform image pickup. In this case, the image pickup control unit 61 sets the image pickup conditions of the high-precision mode M1 as the first mode, the high-speed mode M2 as the second mode, and the optimum mode M3 based on the setting data in the setting data storage unit 82. ..

As a result of each processing, the processing device 50 acquires the image pickup data obtained by the image pickup device 10, determines the freshness of the object T from the image pickup data, and transmits the determination result to the display device 30 and the storage device 40. As a result, the result of the freshness determination is displayed on the display device 30 and stored in the storage device 40.

Next, the configuration of the image pickup apparatus 10 will be described with reference to FIG. 2.

The image pickup apparatus 10 includes an incident optical system 11 to which external light is incident, a spectroscopic element 12 that disperses the incident light, and an image pickup element 13 that captures the light dispersed by the spectroscopic element 12. The incident optical system 11 is composed of, for example, a telecentric optical system or the like, and guides the incident light to the spectroscopic element 12 and the image pickup element 13 so that the optical axis and the main ray are parallel or substantially parallel.

The spectroscopic element 12 includes a pair of substrates 14a and 14b, a pair of reflective films 15 and 16 facing each other, and a gap changing portion 17 capable of changing the gap size of the reflective films 15 and 16. Interference filters are used. The gap changing portion 17 is configured by, for example, an electrostatic actuator. Tunable interference filters are also called etalons. The spectroscopic element 12 is arranged on an optical path of light incident on the image pickup element 13.

The spectroscopic element 12 changes the gap dimensions of the reflective films 15 and 16 by changing the voltage applied to the gap changing unit 17 under the control of the processing device 50, and uses the wavelength of the light transmitted through the reflective films 15 and 16. A certain output wavelength λi (i = 1, 2, ..., N) is changed.

The image pickup device 13 is a device that captures light transmitted through the spectroscopic element 12, and is composed of, for example, a CCD, CMOS, or the like. The image pickup device 10 sequentially switches the wavelength of the light dispersed by the spectroscopic element 12 under the control of the control unit 60, captures the light transmitted through the spectroscopic element 12 by the image pickup element 13, and outputs the image pickup data.

The image pickup data is data output for each pixel constituting the image pickup element 13, and is data indicating the intensity of light received by the pixel, that is, the amount of light. The image pickup data output by the image pickup device 10 is input to the processing device 50. Since the image pickup apparatus 10 is a wavelength scanning type, it is possible to obtain high-resolution image pickup data as compared with the case of the wavelength dispersion type.

Next, with reference to FIG. 3, an operation mode that defines an operation state at the time of measurement will be described. In the following, the term "spectral spectrum" may be simply referred to as "spectrum".

As shown in FIG. 3, the operation modes include a high-precision mode M1, a high-speed mode M2, and an optimum mode M3. The high-precision mode M1 is an operation mode that enables high-precision measurement by setting the number of wavelengths of light (output wavelength λi) dispersed by the spectroscopic element 12 to N1, which is relatively large. On the other hand, the high-speed mode M2 is an operation mode in which the time required for measurement is shortened by setting the number of wavelengths of light (output wavelength λi) dispersed by the spectroscopic element 12 to N2, which is smaller than N1. Further, the optimum mode M3 selectively increases or decreases the number of wavelengths of light (output wavelength λi) dispersed by the spectroscopic element 12, so that the time required for measurement can be reduced to the high precision mode M1. This is an operation mode that is substantially intermediate with the mode M2.

Specifically, in the high-precision mode M1, as shown in FIG. 3, for example, the output wavelength λi is set in the range of 400 nm to 700 nm in increments of 10 nm, and the number of measurement wavelengths N1 is 31. The high-precision mode M1 is, for example, when it is desired to measure the concentration of a specific substance from a mixture of many kinds with high accuracy, when there are a plurality of objects T to be measured, and when the spectral spectrum of the object T is a steep peak shape. It is preferable to select when the object T has an object T, or when the shape of the spectral spectrum of the object T is unknown.

Further, in the high-speed mode M2, as shown in FIG. 3, for example, the output wavelength λi is set in the range of 400 nm to 680 nm in increments of 40 nm, and the number of measurement wavelengths N2 is eight. In the high-speed mode M2, for example, when low-precision measurement often emphasizes short tact time, when the shape of the spectral spectrum of the object T is gentle, and when the shape of the spectral spectrum of the object T is known. Therefore, it is preferable to select it when it is sufficient to see a specific small number of waveforms.

Further, as shown in FIG. 3, in the optimum mode M3, for example, the output wavelength λi is set in the range of 400 nm to 600 nm in increments of 40 nm, the number of measurement wavelengths is 5, and the output wavelength λi is in the range of 600 nm to 700 nm. The number of measured wavelengths is set to 11 in increments of 10 nm. The optimum mode M3 is, for example, when the shape of the spectral spectrum of the object T is known, which wavelength should be viewed in detail, and which wavelength can be thinned out, and also in the high precision mode M1 first. When selecting the output wavelength λi to be measured from the measured information, it is preferable to select the output wavelength λi.

In either operation mode, the exposure time is set so that the image pickup device 13 can obtain a sufficient S / N ratio, and high-precision image pickup data can be obtained. For example, when the exposure time is set to 60 msec, the measurement time in the high-precision mode M1 is 2.07 seconds, the measurement time in the high-speed mode M2 is 0.53 seconds, and the measurement time in the optimum mode M3 is 1. It will be 06 seconds. If N1 is an integer of 2 or more and N2 is in the range of an integer smaller than N1, the values of N1 and N2 can be changed as appropriate. Further, the high-precision mode M1 is an example of the first mode, and the high-speed mode M2 is an example of the second mode.

Next, a method of determining the freshness of green vegetables according to each operation mode M1, M2, M3 will be described with reference to FIGS. 4 to 6.

First, with reference to FIG. 4, a method of determining the freshness of the object T in the high-precision mode M1 will be described. For example, the operation panel (not shown) provided in the display device 30 is operated to select the measurement in the high accuracy mode M1.

In step S11, a database is generated and saved. Specifically, a database required for freshness determination is generated, and the database is stored in the storage unit 80 of the processing device 50. The database referred to here corresponds to various data stored in the storage unit 80 (see FIG. 1) in the processing device 50.

The contents and generation method of various data as a database will be explained. Various data are measurement band data. The database may be generated and stored before the spectroscopic image pickup apparatus 400A is shipped from the factory, or may be omitted when the object T whose freshness is known is used. In addition, a worker may perform a part of the procedure.

Further, the setting data is data in which processing conditions for various processes executed by the processing unit 70 are set, and is, for example, data including imaging conditions of each operation mode M1, M2, M3 including N1 and N2.

As the transformation data, the transformation matrix M of the following equation (1) can be adopted. p is a vector representing the measurement spectrum (first measurement spectrum) in the high-precision mode M1, and is composed of 31 elements in the present embodiment. x is a vector representing the measurement spectrum in the high-speed mode M2, and is composed of eight elements in the present embodiment. The transformation matrix M is determined to estimate a spectrum consisting of m1 spectra from a spectrum consisting of m2 wavelengths. From this conversion data (transformation matrix M), the measurement spectrum that will be obtained in the high-precision mode M1 can be estimated from the measurement spectrum (second measurement spectrum) obtained in the high-speed mode M2. As a method of deriving the transformation matrix M, the method described in JP2012-242270A can be adopted. Further, the equation (1) is established for each pixel in this embodiment.
p = Mx ... (1)

In another embodiment, a conversion matrix Mq for approximating the measurement spectrum p obtained in the high-precision mode M1 to the measurement spectrum obtained by a spectroscope with higher accuracy (for example, measuring at a wavelength higher than 31) is used. It may be used to estimate the spectrum of the object. As a method for determining the transformation matrix Mq, the method described in JP2012-242270A can be adopted. The same applies to the case of the high-speed mode M2. The "measurement spectrum" of the present embodiment is obtained by arranging the luminance values output by any pixel (or pixel group) of the image pickup device 13 along the wavelength.

Next, the measurement band data will be described. When green vegetables get old, chlorophyll (chlorophyll) is decomposed and the bright green color disappears. From this, it can be seen that the freshness of green vegetables can be determined from the amount of chlorophyll. In the present embodiment, the freshness of the green vegetable which is the object T is determined by estimating the amount of chlorophyll as the characteristic amount of the object T.

Here, with reference to FIG. 7, the relationship between the wavelength of light and the reflectance will be described for green vegetables having different freshness.

As shown in FIG. 7, in fresh vegetables (or slightly withered vegetables), light absorption by chlorophyll occurs at around 700 nm. Therefore, a plurality of wavelengths in the range of 500 nm to 1100 nm including the wavelength at which light absorption by chlorophyll occurs (about 700 nm) are stored as measurement band data in order to instruct the image pickup apparatus 10 as the measurement band.

Returning to FIG. 4, in step S12, the output wavelength λi, which is an imaging condition, is acquired. Specifically, the control unit 60 acquires the imaging conditions of the high-precision mode M1 included in the setting data storage unit 82.

Specifically, for example, 31 measurement band data are acquired from 400 nm to 700 nm in increments of 10 nm. The measurement band data is, for example, 400 nm, 410 nm, 420 nm, ..., 700 nm.

The measurement band data does not necessarily have to be 31 at 10 nm intervals, but may be at 20 nm intervals. Further, the wavelength range is not limited to 400 nm to 700 nm, and may be in the range of 350 nm to 1100 nm.

In step S13, the object T is imaged. Specifically, the image pickup control unit 61 controls the image pickup device 10 and makes the output wavelength λi different according to the acquired image pickup conditions to image the object T.

In step S14, N1 imaging data are generated. Specifically, N1 imaging data captured by differentiating the output wavelengths λi of the spectroscopic element 12 in order are generated. The N1 imaging data of this embodiment is 31 imaging data. In this embodiment, 31 imaging data represent the first spectral spectrum.

In step S15, the freshness is determined. Specifically, the processing device 50 determines the freshness of the object T from the image pickup data output from the image pickup device 10.

In step S16, the result is displayed and saved. Specifically, the freshness determination result obtained in step S15 is output to the display device 30 and the storage device 40.

In this way, by performing the freshness determination of the green vegetable as the object T in the high-precision mode M1, for example, it is possible to perform rigorous measurement, and if there is a part that has withered even a little, it is determined that the freshness is poor. can do. Thereby, for example, only fresh vegetables can be selected.

Next, a method of determining the freshness of the object T in the high-speed mode M2 will be described with reference to FIG. For example, the operation panel (not shown) provided in the display device 30 is operated to select the measurement in the high-speed mode M2.

First, as in the high-precision mode M1, in step S11, a database is generated and saved.

Next, in step S21, the output wavelength λi, which is an imaging condition, is acquired. Specifically, the control unit 60 acquires the imaging conditions of the high-speed mode M2 included in the setting data storage unit 82.

Specifically, for example, eight measurement band data are acquired from 400 nm to 680 nm in increments of 40 nm. The measurement band data is, for example, 400 nm, 440 nm, 480 nm, ..., 680 nm.

The measurement band data does not necessarily have to be eight at 40 nm intervals, and it is preferable that the measurement can be performed in a shorter time than the high-precision mode M1, and the measurement intervals may be changed or the wavelength range may be changed. ..

In step S22, the object T is imaged. Specifically, the image pickup control unit 61 controls the image pickup device 10 and makes the output wavelength λi different according to the acquired image pickup conditions to image the object T.

In step S23, N2 image pickup data are generated. Specifically, N2 imaging data captured by sequentially differentiating the output wavelengths λi of the spectroscopic element 12 are generated. The N2 imaging data of this embodiment are eight imaging data.

In step S24, as an option, the spectral spectrum as the second spectral spectrum is estimated. Specifically, the spectral spectrum obtained from the N1 imaged data is estimated based on the converted data M stored in the converted data storage unit 83 of the storage unit 80.

After that, steps S25 and S26 are performed in the same manner as steps S15 and S16 in the high precision mode M1.

As described above, the color spectroscopic spectrum result obtained from the N1 image pickup data in the high-precision mode M1 is associated with the color spectroscopic spectrum result obtained from the N2 image pickup data in the high-speed mode M2. The conversion data is acquired in advance. Therefore, in the high-speed mode M2, the color spectroscopic spectrum result obtained from the N2 imaging data is converted into the information corresponding to the color spectral spectrum result obtained from the N1 imaging data based on the conversion data. .. Then, the image pickup data input based on the converted information is used as the corrected image pickup data. As a result, the deviation of the peak wavelength of each color that occurs when the spectral spectrum is estimated with only a relatively small number of imaging data can be set within an allowable range, the measurement accuracy can be sufficiently maintained, and appropriate correction can be performed.

In this way, by performing the freshness determination of the green vegetable as the object T in the high-speed mode M2, it is possible to determine the freshness in a short time as compared with the case of the high-precision mode M1, for example, and the rough freshness can be determined. If the judgment is acceptable, the efficiency of measurement can be improved.

Next, a method of determining freshness in the optimum mode M3 will be described with reference to FIG. For example, the optimum mode M3 is selected by operating the operation panel arranged on the display device 30.

In step S31, it is determined whether or not the freshness determination process is the first time. If it is the first measurement, the process shifts to step S32. If it is the second or more measurement, the process shifts to step S33.

In step S32, freshness is measured in the high-precision mode M1 in the same manner as the flow shown in FIG. In the case of the second or more measurement, since the process of step S32 has already been performed, the process proceeds to step S33.

Next, in step S33, the output wavelength λi, which is an imaging condition, is acquired. Specifically, the control unit 60 acquires the imaging conditions of the optimum mode M3 included in the setting data storage unit 82.

Specifically, for example, five measurement band data are acquired from 400 nm to 600 nm in increments of 40 nm. The measurement band data is, for example, 400 nm, 440 nm, ... nm, 600 nm. Further, 11 measurement band data are acquired from 600 nm to 700 nm in increments of 10 nm. The measurement band data is, for example, 600 nm, 610 nm, ... nm, 700 nm.

The measurement band data is not limited to the above. For example, the intervals of the output wavelengths λi may be different based on at least one spectral spectrum of the first spectral spectrum obtained in the high-precision mode M1 and the second spectral spectrum obtained in the high-speed mode M2 (in other words). For example, it may be unequally spaced). Here, the intervals of the output wavelengths λi may be different based on the first spectral spectrum.

For example, as shown in FIG. 7, a wavelength band in which the change in reflectance is large (in other words, a wavelength band in which the undulations of the spectral spectrum are large), for example, a range of 500 nm to 600 nm is measured in 10 nm increments, or 700 nm to 800 nm. The range may be set to be measured in increments of 10 nm. Further, in other wavelength bands (in other words, a wavelength band having a flat spectral spectrum), the measurement may be performed in increments of 40 nm. Further, the wavelength interval to be measured is not limited to 40 nm increments or 10 nm increments. By measuring in this way, it is possible to perform the measurement in a shorter time than in the high-precision mode M1 and in which the decrease in accuracy is suppressed.

In step S34, the object T is imaged. Specifically, the image pickup control unit 61 controls the image pickup device 10 and makes the output wavelength λi different according to the acquired image pickup conditions to image the object T.

In step S35, imaging data is generated. Specifically, the imaging data captured by differentiating the output wavelengths λi of the spectroscopic element 12 in order is generated. In this embodiment, for example, 16 imaging data.

In step S36, the spectroscopic spectrum is estimated based on the above calculation formula. As a result, an image of the spectral spectrum for each set wavelength (for each color) can be obtained.

After that, steps S37 and S38 are performed in the same manner as steps S15 and S16 in the high precision mode M1.

In this way, by performing the freshness determination of the green vegetable as the object T in the optimum mode M3, for example, the wavelength band that greatly affects the freshness determination is known as compared with the case of the high precision mode M1. The wavelength band can be measured in detail, and the other parts can be roughly measured. As a result, the freshness can be determined in a short time as compared with the high accuracy mode M1, and it is possible to suppress the deterioration of the measurement accuracy. As a result, the efficiency of measurement can be improved.

As described above, the control method of the spectroscopic image pickup device 400A of the present embodiment includes the image pickup element 13 and the spectroscopic element 12, and when the spectroscopic image pickup device 400A is in the high precision mode M1, the spectroscopic image pickup device 400A is used for the spectroscopic element. When the spectroscopic imager 400A is in the high-speed mode M2, the spectroscopic element 12 is generated by the spectroscopic imager 400A by generating a first measurement spectrum consisting of N1 wavelengths in which the object T is imaged by making the output wavelengths λi of 12 different. A second measurement spectrum consisting of N2 wavelengths obtained by imaging the object T with different output wavelengths λi is generated, N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

According to this method, N1 is an integer of 2 or more, and N2 is an integer smaller than N1, so that the high-precision mode M1 can be used for exact measurement. On the other hand, in the case of simple measurement, by setting the high-speed mode M2, the object T can be measured in a shorter time than in the case of the high-precision mode M1. Therefore, regarding the measurement of various scenes, it is possible to improve the convenience and efficiency of the user, and it is possible to selectively perform high-precision measurement and low-precision measurement.

Further, it is preferable to derive, that is, estimate the spectrum of the object T based on the first measurement spectrum generated in the high-precision mode M1 or the second measurement spectrum generated in the high-speed mode M2.

According to this method, the spectrum of the object T is derived, that is, estimated based on the first measurement spectrum and the second measurement spectrum, so that it is almost the same as the spectral spectrum obtained from N1 image data and N2 image data. It is possible to derive a spectral spectrum in which the peak wavelength of each color is located at the same wavelength.

Further, the spectrum of the object T is estimated by converting the second measurement spectrum obtained in the high-speed mode M2 using the conversion data for estimating the spectrum consisting of N1 wavelengths from the spectrum consisting of N2 wavelengths. It is preferable to do so.

According to this method, since the conversion data is acquired in advance, it is possible to obtain the imaging data corresponding to N1 from the imaging data of N2 smaller than N1.

Further, the spectrum of the object T has a first measurement spectrum obtained in the high-precision mode M1 and a second measurement spectrum obtained in the high-speed mode M2, and has a first measurement spectrum and a second measurement spectrum. It is preferable to make the intervals of the output wavelengths λi different based on at least one of them.

According to this method, for example, wavelengths that can be thinned out from the shape of the spectrum, such as acquiring one or two imaging data in a wavelength band with a flat spectrum and acquiring fine imaging data in a wavelength band with a peak. By selecting the band, it is possible to suppress the accuracy of the imaging data and perform efficient measurement.

Further, the spectroscopic element 12 has a pair of reflective films 15 and 16 and a gap changing portion 17 capable of changing the gap size of the pair of reflective films 15 and 16, and is on the optical path of light incident on the image pickup element 13. It is preferably an arranged wavelength variable interference filter.

According to this method, since a wavelength variable interference filter is used, for example, it is possible to perform high-precision measurement to short-time measurement while maintaining high resolution as compared with the case of wavelength dispersion type.

The spectroscopic image pickup device 400A includes an image pickup element 13 and a spectroscopic element 12, and when the spectroscopic image pickup device 400A is in the high-precision mode M1, the spectroscopic image pickup device 400A makes the output wavelength λi of the spectroscopic element 12 different to obtain the object T. A first measurement spectrum consisting of N1 wavelengths imaged is generated, and when the spectroscopic image pickup device 400A is in the high-speed mode M2, the spectroscopic image pickup device 400A images the object T with different output wavelengths λi of the spectroscopic element 12. N2 second measurement spectra are generated, N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

According to this configuration, N1 is an integer of 2 or more, and N2 is an integer smaller than N1, so that the high-precision mode M1 can be used for exact measurement. On the other hand, in the case of simple measurement, by setting the high-speed mode M2, the object T can be measured in a shorter time than in the case of the high-precision mode M1. Therefore, regarding the measurement of various scenes, it is possible to improve the convenience and efficiency of the user, and it is possible to selectively perform high-precision measurement and low-precision measurement.

The computer program is a computer program for identifying the object T based on the image pickup data by the spectroscopic image pickup device 400A including the image pickup element 13 and the spectroscopic element 12, and is an output wavelength in the high-precision mode M1 of the spectroscopic image pickup device 400A. The output wavelength λi is different in the process of imaging the object T with different λi and generating the first measurement spectrum consisting of N1 wavelengths which are integers of 2 or more, and in the high-speed mode M2 of the spectroscopic image pickup device 400A. The object T is imaged, and at least one of the processes of generating a second measurement spectrum consisting of N2 wavelengths, which is an integer smaller than N1, is executed by a computer as the processing unit 70.

According to this computer program, N1 is an integer of 2 or more, and N2 is an integer smaller than N1, so that the high-precision mode M1 can be used for exact measurement. On the other hand, in the case of simple measurement, by setting the high-speed mode M2, the object T can be measured in a shorter time than in the case of the high-precision mode M1. Therefore, regarding the measurement of various scenes, it is possible to improve the convenience and efficiency of the user, and it is possible to selectively perform high-precision measurement and low-precision measurement.

Hereinafter, a modified example of the above embodiment will be described.

The spectroscopic element 12 is not limited to the above-mentioned configuration, and may have a configuration as shown in FIG. FIG. 8 is a cross-sectional view showing the structure of the spectroscopic element 112 of the modified example. The spectroscopic element 112 of the modified example is different from the spectroscopic element 12 of the above embodiment in a portion composed of the first substrate 101, the second substrate 102, and the third substrate 103.

As shown in FIG. 8, in the spectroscopic element 112 of the modified example, as described above, the first substrate 101, the second substrate 102, and the third substrate 103 are bonded via, for example, a bonding layer 106. There is. A pair of reflective films 104 are arranged on the surfaces of the second substrate 102 and the third substrate 103 that face each other. An electrostatic actuator 105 capable of changing the gap size of the reflective film 104 is arranged on the surfaces of the first substrate 101 and the second substrate 102 facing each other. Even in such a structure, it is possible to provide the spectroscopic element 112 having the same function as the spectroscopic element 12.

Further, in the optimum mode M3 of the above embodiment, the measurement is performed in the high-precision mode M1 at the time of the first measurement, and then the imaging conditions of the optimum mode M3 are acquired to image the object T. , Not limited to this. For example, if various information when the measurement is already performed in the high accuracy mode M1 is obtained, the process may be started from step S33 (see FIG. 6) even for the first measurement.

As described above, the spectroscopic image pickup device 400A has been used for determining the freshness of an object T such as vegetables, but the present invention is not limited to this, and for example, exterior parts made of resin or metal, printed matter, and dyeing are used for color determination. It may be used for a display body such as a fiber or a display. Further, as the component analysis by spectroscopy, the presence / absence of water and organic substances, concentration calibration and the like can be mentioned. In addition, not limited to these, all that can be identified by spectroscopy are targeted. Examples of applications using the spectroscopic image pickup device 400A include a colorimeter for a printer, an image quality inspection camera, and the like. Examples of the image quality inspection camera include color inspection, stain and stain (adhesion) inspection, and the like.

[Second Embodiment]
Hereinafter, embodiments of the display system will be described with reference to the attached drawings.
FIG. 9 is a block diagram showing a configuration of a display system 1 having a projector 100. In this display system 1, a configuration corresponding to a spectroscopic imaging device is integrally provided with the projector 100. The spectroscopic image pickup device is also referred to as a spectroscopic camera, and is not limited to hardware such as the spectroscopic image pickup unit 137 described later, and includes software and a processor for realizing the operation of the spectroscopic image pickup device.

The projector 100 is displayed on a screen SC, an image projection system that generates image light and projects it on a screen SC that constitutes a projection surface, an image processing system that electrically processes image data that is a source of an optical image, and a screen SC. It includes a spectral imaging unit 137 that captures image light. Further, the projector 100 includes an image projection system, an image processing system, and a control unit 150 that controls a spectroscopic imaging unit 137.

[Image projection system]
The image projection system includes a projection unit 110 and a drive unit 120. The projection unit 110 is an example of a display unit that displays an image corresponding to the projection image. The projection unit 110 includes a light source 111, an optical modulation device 113, and an optical unit 117. The drive unit 120 includes a light source drive circuit 121 and an optical modulation device drive circuit 123. The light source drive circuit 121 and the optical modulator drive circuit 123 are connected to the bus 180 and perform data communication with other components of the projector 100 also connected to the bus 180 via the bus 180. The other components include, for example, the control unit 150 and the image processing unit 143 shown in FIG.

As the light source 111, a solid-state light source such as an LED (Light Emitting Diode) or a laser light source is used. Further, as the light source 111, it is also possible to use a lamp such as a halogen lamp, a xenon lamp, or an ultrahigh pressure mercury lamp. A light source drive circuit 121 is connected to the light source 111. The light source drive circuit 121 supplies a drive current or pulse to the light source 111 to turn on the light source 111, stops the supplied drive current or pulse, and turns off the light source 111.

The light modulation device 113 includes a light modulation element that modulates the light emitted by the light source 111 to generate image light. As the light modulation element, for example, a transmissive type or reflective type liquid crystal panel, a digital mirror device, or the like can be used. In the present embodiment, a case where the optical modulation device 113 includes a transmissive liquid crystal panel 115 as an optical modulation element will be described as an example. The optical modulation device 113 includes three liquid crystal panels 115 corresponding to the three primary colors of red, green, and blue. The light modulated by the liquid crystal panel 115 is incident on the optical unit 117 as image light. In the following, red is referred to as "R", green is referred to as "G", and blue is referred to as "B".

An optical modulation device drive circuit 123 is connected to the optical modulation device 113. The optical modulation device drive circuit 123 drives the optical modulation device 113 to draw an image on the liquid crystal panel 115 in frame units. The optical unit 117 includes an optical element such as a lens or a mirror, and projects the image light modulated by the optical modulation device 113 toward the screen SC. An image based on the image light projected by the optical unit 117 is formed on the screen SC. An image formed on the screen SC by the image light projected by the projection unit 110 is called a projection image.

[Operation input system]
The projector 100 includes an operation panel 131, a remote control light receiving unit 133, and an input interface 135. The input interface 135 is connected to the bus 180 and performs data communication with the control unit 150 and the like via the bus 180. The operation panel 131 is arranged in the housing of the projector 100, for example, and includes various switches. When the switch on the operation panel 131 is operated, the input interface 135 outputs an operation signal corresponding to the operated switch to the control unit 150.

The remote control light receiving unit 133 receives an infrared signal transmitted by the remote controller (remote control). The remote control light receiving unit 133 outputs an operation signal corresponding to the received infrared signal. The input interface 135 outputs the input operation signal to the control unit 150. This operation signal is a signal corresponding to the switch of the operated remote controller.

[Spectroscopic imaging unit]
The spectroscopic imaging unit 137 images the projected image displayed on the screen SC by the projection unit 110 and outputs the spectral imaging data.

FIG. 10 is a schematic configuration diagram of the spectroscopic imaging unit 137. The spectroscopic image pickup unit 137 is an example of the "spectral image pickup device" of the present invention. The spectroscopic imaging unit 137 includes an incident optical system 301 on which external light is incident, a spectroscopic element 302 that disperses the incident light, and an image pickup element 303 that captures the light dispersed by the spectroscopic element 302.

The incident optical system 301 is composed of, for example, a telecentric optical system or the like, and guides the incident light to the spectroscopic element 302 and the image pickup element 303 so that the optical axis and the main ray are parallel or substantially parallel. For the spectroscopic element 302, a wavelength variable interference filter including a pair of reflective films 304 and 305 facing each other and a gap changing portion 306 capable of changing the gap size of the reflective films 304 and 305 is used. The gap changing portion 306 is configured by, for example, an electrostatic actuator. Tunable interference filters are also called etalons. The spectroscopic element 302 is arranged on the optical path of the light incident on the image pickup element 303.

The spectroscopic element 302 changes the gap dimensions of the reflective films 304 and 305 by changing the voltage applied to the gap changing unit 306 under the control of the control unit 150, and uses the wavelength of the light transmitted through the reflective films 304 and 305. A certain spectral wavelength λi (i = 1, 2, ..., N) can be changed. The image pickup device 303 is a device that captures light transmitted through the spectroscopic element 302, and is composed of, for example, a CCD, CMOS, or the like. The spectroscopic image pickup unit 137 sequentially switches the wavelength of the light dispersed by the spectroscopic element 302 under the control of the control unit 150, captures the light transmitted through the spectroscopic element 302 by the image pickup element 303, and outputs the spectroscopic image pickup data. The spectroscopic image pickup data is data output for each pixel constituting the image pickup element 303, and is data indicating the intensity of light received by the pixel, that is, the amount of light. The spectral imaging data output by the spectral imaging unit 137 is input to the control unit 150. Since the spectroscopic imaging unit 137 is a wavelength scanning type, it becomes easier to obtain high-resolution spectroscopic imaging data as compared with the case of the wavelength dispersion type.

[Communication Department]
As shown in FIG. 9, the projector 100 includes a communication unit 139. The communication unit 139 is connected to the bus 180. As shown in FIG. 14, which will be described later, the communication unit 139 functions as an interface for mutual data communication between the projectors 100 when a plurality of projectors 100 are connected. The communication unit 139 of the present embodiment is a wired interface for connecting a cable, but may be a wireless communication interface for executing wireless communication such as a wireless LAN or Bluetooth. "Bluetooth" is a registered trademark.

[Image processing system]
Next, the image processing system of the projector 100 will be described.
As shown in FIG. 9, the projector 100 includes an image interface 141, an image processing unit 143, and a frame memory 145 as an image processing system. The image processing unit 143 is connected to the bus 180 and performs data communication with the control unit 150 and the like via the bus 180.

The image interface 141 is an interface for receiving an image signal, and includes a connector to which the cable 3 is connected and an interface circuit for receiving the image signal via the cable 3. The image interface 141 extracts image data and synchronization signals from the received image signals, and outputs the extracted image data and synchronization signals to the image processing unit 143. Further, the image interface 141 outputs a synchronization signal to the control unit 150. The control unit 150 controls other components of the projector 100 in synchronization with the synchronization signal. The image processing unit 143 performs image processing on the image data in synchronization with the synchronization signal.

An image supply device 200 is connected to the image interface 141 via a cable 3. As the image supply device 200, for example, a notebook PC (Personal Computer), a desktop PC, a tablet terminal, a smartphone, or a PDA (Personal Digital Assistant) can be used. Further, the image supply device 200 may be a video playback device, a DVD player, a Blu-ray disc player, or the like. Further, the image signal input to the image interface 141 may be a moving image or a still image, and the data format is arbitrary. The connection is not limited to the wired connection using the cable 3, and may be a wireless connection using wireless communication.

The image processing unit 143 and the frame memory 145 are configured by, for example, an integrated circuit. Integrated circuits include LSIs (Large-Scale Integrated Circuits), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), FPGAs (Files-System) included. Further, an analog circuit may be included as a part of the configuration of the integrated circuit.

The image processing unit 143 is connected to the frame memory 145. The image processing unit 143 expands the image data input from the image interface 141 into the frame memory 145, and performs image processing on the expanded image data.

The image processing unit 143 executes various processes including, for example, a geometric correction process for correcting trapezoidal distortion of a projected image, an OSD process for superimposing an OSD (On Screen Display) menu, and the like. Further, the image processing unit 143 performs image processing such as image adjustment processing for adjusting the brightness and hue of the image data, resolution conversion processing for adjusting the aspect ratio and resolution of the image data according to the optical modulation apparatus 113, and frame rate conversion. To execute.

The image processing unit 143 outputs the image data after image processing to the optical modulation device drive circuit 123. The optical modulator drive circuit 123 generates drive signals for each of the red, green, and blue colors based on the image data input from the image processing unit 143. The optical modulation device drive circuit 123 drives the liquid crystal panel 115 of the corresponding color of the optical modulation device 113 based on the generated drive signal of each color, and draws an image on the liquid crystal panel 115 of each color. When the light emitted from the light source 111 passes through the liquid crystal panel 115, the image light corresponding to the image of the image data is generated.

[Control / Storage]
The control unit 150 includes a storage unit 160 and a processor 170.
The storage unit 160 is composed of, for example, a non-volatile semiconductor memory such as a flash memory or EEPROM, or an SSD (Solid State Drive) using the flash memory. In the present embodiment, the case where the control unit 150 includes the storage unit 160 will be described, but for example, the storage unit 160 configured by the hard disk drive may be provided outside the control unit 150. The storage unit 160 stores control program 161, image data such as adjustment image data 162 and pattern image data 163, setting data 164, parameter 165, correction parameter 166, calibration data 167, and the like. The control unit 150 and the spectroscopic image pickup unit 137 correspond to an example of the "spectral image pickup device" of the present invention.

The control program 161 is a program such as an OS (Operating System) executed by the processor 170 and an application program. The processor 170 measures information about the color and the like of the image projected on the screen SC by controlling each part and performing arithmetic processing according to the control program 161 and performs a process of using the measurement result for correction of the projected image. .. In the following description, this process will be appropriately referred to as "measurement / image quality adjustment process".

The operation mode that defines the operation state when performing measurement / image quality adjustment processing includes a high-precision mode and a high-speed mode. As shown in FIG. 11, the high-precision mode is an operation mode that enables high-precision measurement by setting the number of wavelengths of light (spectral wavelength λi) dispersed by the spectroscopic element 302 to N1, which is relatively large. Is. On the other hand, the high-speed mode is an operation mode in which the time required for measurement is shortened by setting the number of wavelengths of light (spectral wavelength λi) dispersed by the spectroscopic element 302 to N2, which is smaller than N1.

FIG. 11 shows a case where the spectral wavelength λi is set in the range of 400 nm to 700 nm in increments of 10 nm as an example of the high-precision mode, and N1 is 31. Further, FIG. 11 shows a case where the spectral wavelength λi is set in the range of 400 nm to 680 nm in increments of 40 nm as an example of the high-speed mode, and N2 is 8. In either operation mode, the exposure time is set so that a sufficient S / N ratio can be obtained by the image pickup device 303, and highly accurate spectral imaging data can be obtained. For example, when the exposure time is set to 60 msec, the measurement time in the high-precision mode is 2.07 seconds, and the measurement time in the high-speed mode is 0.53 seconds. If N1 is an integer of 2 or more and N2 is in the range of an integer smaller than N1, the values of N1 and N2 can be changed as appropriate. The high-precision mode is an example of the "first mode" of the present invention, and the high-speed mode is an example of the "second mode" of the present invention.

Returning to FIG. 9, the setting data 164 is data in which processing conditions for various processes executed by the processor 170 are set, and is, for example, data including imaging conditions of each operation mode including the above N1 and N2. The parameter 165 is, for example, an image processing parameter to be executed by the image processing unit 143.

The image data stored in the storage unit 160 is data that is the source of the image displayed on the screen SC by the projector 100, and includes, for example, pattern image data 163 and adjustment image data 162. The pattern image data 163 is, for example, image data in which marks having a predetermined shape are arranged at four corners. The processor 170 acquires imaging data (may be spectral imaging data) when an image corresponding to the pattern image data 163 is projected onto the screen SC. Further, the processor 170 acquires information (in this configuration, a projective transformation matrix 167b) that associates the pixels projected on the screen SC with the pixels of the liquid crystal panel 115 based on the acquired imaging data.

The adjustment image data 162 is, for example, monochrome image data of each color of RGB. The control unit 150 acquires spectral imaging data for each spectral wavelength λi when an image corresponding to the adjustment image data 162 is projected onto the screen SC, and corrects the spectral imaging data 167a based on the spectral imaging data. To get. In this case, in the high-precision mode, N1 spectral imaging data are acquired, and in the high-speed mode, N2 spectral imaging data, which is less than N1, are acquired. A predetermined measurement target is measured based on the acquired spectral imaging data. The measurement targets are the absolute value of each color of RGB and the color unevenness in the projection surface.

The correction parameter 166 is a parameter generated by "measurement / image quality adjustment processing", and is an image processing parameter for correcting the absolute value and color unevenness of each color in the input image data by the image processing unit 143. ..
The calibration data 167 includes correction data 167a, projective transformation matrix 167b, estimation matrix M, and transformation data 167c. The correction data 167a is data that corrects the sensitivity distribution of the image pickup element 303 and corrects the spectral imaging data so that the output of the image pickup element 303 becomes uniform.

In the image pickup element 303, the output of each pixel constituting the image pickup element 303 is not uniform due to the influence of the lens aberration of the lens included in the incident optical system 301, and the output differs depending on the position of the pixel. That is, a sensitivity distribution is generated in the image sensor 303. The output of this sensitivity distribution is lower in the peripheral portion than the center of the image sensor 303. Therefore, when the color or the like of an image is corrected based on the spectroscopic image pickup data captured by the spectroscopic image pickup unit 137, it may not be possible to correct the color accurately due to the influence of an error. Further, the sensitivity distribution of the spectroscopic imaging unit 137 is affected by the ultraviolet rays coated on the lens surface and the optical filter that cuts infrared rays. That is, the output of the spectroscopic imaging unit 137 also differs depending on the color of the image captured by the spectroscopic imaging unit 137.

The correction data 167a is generated at the time of manufacturing the projector 100 or the like, and is also generated for each pixel of the image pickup device 303. Further, a plurality of correction data 167a are generated corresponding to the light of each color of RGB projected by the projection unit 110 and the spectral wavelength λi set in the spectroscopic imaging unit 137. By generating the correction data 167a for each color and for each spectral wavelength λi, the correction accuracy of the spectral sensitivity of the spectral imaging unit 137 is improved.

The projective transformation matrix 167b is a transformation matrix that converts the coordinates set in the liquid crystal panel 115 of the optical modulation apparatus 113 into the coordinates set in the spectral imaging data. The liquid crystal panel 115 has a configuration in which a plurality of pixels are arranged in a matrix. The coordinates set on the liquid crystal panel 115 are the coordinates for specifying each pixel arranged in this matrix.

The estimation matrix M is a matrix used for estimating the spectrum. The estimation matrix M is generated at the time of manufacturing the projector 100 and is stored in the storage unit 160 as a part of the calibration data 167. The estimation matrix M is generated based on the spectroscopic imaging data captured by the spectroscopic imaging unit 137. An optical component is mounted on the optical unit 117, and the optical component is also used as a component that guides the light emitted from the light source 111 to the liquid crystal panel 115 of the optical modulation device 113. Further, in the liquid crystal panel 115 as an optical component, each pixel constituting the liquid crystal panel 115 has a spectral characteristic, and an error occurs in the wavelength of the image light transmitted by the pixel. Due to the optical characteristics of these optical components, an error occurs in the value of the spectroscopic imaging data generated by the spectroscopic imaging unit 137, and the color measurement accuracy is lowered. As the method for calculating the correction data 167a, the projective transformation matrix 167b, and the estimation matrix M described above, known methods can be widely applied.

The conversion data 167c in the storage unit 160 includes a color measurement result obtained from N1 spectral imaging data in the high-precision mode and a color measurement result obtained from N2 spectral imaging data in the high-speed mode. It is the data to associate with. The N2 spectral imaging data has a smaller number of data than the N1 spectral imaging data, or the interval between the spectral wavelengths λi is wide, so that the error is large when the spectrum of each color of RGB is estimated. For example, the error of the peak wavelength of each color becomes large. When the error of the peak wavelength of each color becomes large, the measurement accuracy of the absolute value of each color becomes low. In the present embodiment, the conversion data 167c is measured in advance at the time of manufacturing the projector 100, and when the spectrum is estimated from the spectral imaging data of N2 pieces, the conversion data 167c is used to obtain the peak wavelength of each color and the like. Keep the deviation within the allowable range. This improves the measurement accuracy such as the absolute value of each color and the correction accuracy.

[processor]
The processor 170 is an arithmetic processing unit including, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a microcomputer, and the like. The processor 170 may be configured by a single processor or may be configured by combining a plurality of processors. The processor 170 functions as a projection control unit 171, an image pickup control unit 173, a calculation unit 175, and the like by executing the control program 161 stored in the storage unit 160.

The projection control unit 171 controls the image displayed on the screen SC by the projection unit 110. Specifically, the projection control unit 171 controls the image processing unit 143 to execute image processing on the image data input from the image interface 141. At this time, the projection control unit 171 may read the parameter 165 required for image processing and the information of the OSD menu from the storage unit 160 and output the information to the image processing unit 143. Further, the projection control unit 171 can control the light source drive circuit 121 to adjust the brightness of the light source 111.

The image pickup control unit 173 causes the spectroscopic image pickup unit 137 to perform imaging. In this case, the image pickup control unit 173 sets the image pickup conditions of the high-precision mode or the high-speed mode based on the setting data 164 in the storage unit 160. The arithmetic unit 175 performs arithmetic processing for measuring the absolute value of each color of RGB and the color unevenness in the projection surface based on the plurality of spectroscopic imaging data output from the spectroscopic imaging unit 137. Further, the calculation unit 175 generates a correction parameter 166 for correcting the absolute value and color unevenness of each of the above colors based on the measurement result obtained by the calculation process. The image processing unit 143 can generate corrected image data in which the absolute value of each color and color unevenness are corrected by using the correction parameter 166 when processing the image data input from the image interface 141.

[Projector operation]
FIG. 12 is a flowchart showing the operation of the display system 1 regarding the measurement / image quality adjustment process.
When the control unit 150 inputs an operation signal corresponding to the display instruction of the OSD menu via the input interface 135, the projection control unit 171 performs a process of superimposing the OSD menu on the projection image to display the OSD menu ( Step S101). The OSD menu includes a key for instructing the image quality adjustment in the high-precision mode and a key for instructing the image quality adjustment in the high-speed mode, and the user can select the high-precision mode and the high-speed mode by the OSD menu. The OSD menu and operation method for selecting the high-precision mode and the high-speed mode may be appropriately changed. For example, each mode may be selectable by using the operation panel 131, the remote controller, or the like.

The image quality adjustment in this flowchart is to correct the image by using the image pickup result of the spectroscopic image pickup unit 137, and more specifically, the absolute value and color unevenness of each RGB color of the projected image are set to predetermined conditions. It is to correct. The predetermined conditions are, for example, conditions corresponding to the image quality at the time of manufacturing.
When the image quality adjustment in the high-precision mode is instructed (step S102 / high-precision mode), the control unit 150 acquires the imaging conditions in the high-precision mode included in the setting data 164 (step S103). Next, the control unit 150 performs an imaging process for imaging the projected image according to the acquired imaging conditions (step S104), and generates N1 spectral imaging data (step S105).

In the imaging process, first, the control unit 150 displays the pattern image on the screen SC based on the pattern image data 163 in the storage unit 160, and controls the spectroscopic imaging unit 137 to capture the pattern image. The data obtained by capturing the pattern image is, for example, data captured by the spectroscopic imaging unit 137 in a state where the wavelength is fixed to a predetermined wavelength. Next, the control unit 150 calculates a projective transformation matrix 167b showing the correspondence between the imaging coordinates and the panel coordinates based on the data obtained by capturing the pattern image, and stores it in the storage unit 160. Next, the control unit 150 displays the adjustment image on the screen SC based on the adjustment image data 162 in the storage unit 160, and makes the spectral wavelength λi of the spectral imaging unit 137 different according to the acquired imaging conditions to obtain N1 pieces. Generate spectroscopic imaging data. The N1 spectroscopic imaging data is an example of the "N1 first imaging data" of the present invention.

Subsequently, the control unit 150 measures the absolute value of each color of RGB and the color unevenness in the projection plane based on the spectroscopic imaging data of N1 pieces by the calculation unit 175 (step S106). For example, the spectrum of each color is estimated from N1 spectral imaging data, and the absolute value of each color is acquired from each spectrum. Further, the color unevenness in the projection surface is acquired by detecting the color unevenness (difference in luminance, difference in the estimated spectrum, etc.) for each pixel from the spectroscopic imaging data. A known method can be widely applied to the method for measuring the absolute value of each color and the color unevenness.

The control unit 150 generates a correction parameter 166 for correcting the absolute value of each measured color and the color unevenness (step S107). The generated correction parameter 166 is stored in the storage unit 160. The correction parameter 166 may be a correction parameter in which the pixels constituting the liquid crystal panel 115 are set as one unit, or may be a correction parameter in which a plurality of pixels are set as one unit. When the correction parameter 166 with a plurality of pixels as one unit is generated, the correction parameter 166 of the pixels for which the correction parameter 166 is not generated may be obtained by, for example, an interpolation calculation by linear interpolation.

When the image signal supply is started from the image supply device 200, the control unit 150 reads the correction parameter 166 from the storage unit 160 and outputs the correction parameter 166 to the image processing unit 143. The image processing unit 143 expands the input image data into the frame memory 145 when the image interface 141 starts receiving the image signal and the image data is input from the image interface 141. The image processing unit 143 corrects the image data by the correction parameter 166 or the like input from the control unit 150, and displays the image corresponding to the image data on the screen SC (step S108). The supplied image signal is an example of the "first image data" of the present invention, and the corrected image data is an example of the "first corrected image data" of the present invention. The projected image corresponding to the corrected image data is an example of the "first projected image" of the present invention.

When the high-speed mode image quality adjustment is instructed in step S102 (step S102 / high-speed mode), the control unit 150 acquires the high-speed mode imaging conditions included in the setting data 164 (step S109). Next, the control unit 150 performs an imaging process for imaging the projected image according to the acquired imaging conditions (step S110), and generates N2 spectral imaging data (step S111). This imaging process is the same as the imaging process in step S104, except that the control of the spectral wavelength λi of the spectroscopic element 302 is different. In the high-speed mode, the number of spectral wavelengths λi is smaller than in the high-precision mode, so that the measurement can be completed in a short time. The N2 spectroscopic imaging data is an example of the "N2 second imaging data" of the present invention.

Subsequently, the control unit 150 estimates the spectrum from the N2 spectroscopic imaging data by the calculation unit 175 (step S112). In this case, the calculation unit 175 converts the spectrum of each color obtained from the N2 spectral imaging data into the spectrum equivalent to the spectrum obtained from the N1 spectral imaging data based on the conversion data 167c in the storage unit 160. As a result, a spectrum in which the peak wavelength of each color is located at substantially the same wavelength as the spectrum obtained from the N1 spectroscopic imaging data is estimated.

The control unit 150 measures the absolute value of each color and the color unevenness in the projection surface based on the spectroscopic imaging data of N2 pieces (step S113). For example, the absolute value of each color is acquired from the spectrum estimated in step S112. Further, the color unevenness in the projection surface is acquired by detecting the color unevenness (difference in brightness, etc.) for each pixel from the spectral imaging data. As a method for measuring the absolute value and color unevenness of each color, a known method can be widely applied.

When the control unit 150 measures the absolute value of each color and the color unevenness in the projection surface based on the spectroscopic imaging data of N2 pieces, the processing of steps S107 and S108 is executed. by this. The control unit 150 generates a correction parameter 166 for correcting the absolute value and color unevenness of each measured color, corrects the image data by the correction parameter 166, and displays the image corresponding to the corrected image data on the screen SC. Let me. The corrected image data is an example of the "second corrected image data" of the present invention, and the projected image corresponding to the corrected image data is an example of the "second projected image" of the present invention.

As described above, in the display system 1 of the second embodiment, when the display system 1 is in the high precision mode, the spectroscopic image pickup unit 137 provided with the image pickup element 303 and the spectroscopic element 302 displays the projected image and the spectroscopic wavelength of the spectroscopic element 302. Generates N1 spectroscopic imaging data captured differently. The projector 100 generates corrected image data by correcting the input image data based on N1 spectral imaging data. Then, the projector 100 generates a projected image based on the corrected image data and projects it on the screen SC which is a projection surface.

On the other hand, when the display system 1 is in the high-speed mode, the spectroscopic imaging unit 137 generates N2 spectral imaging data in which the projected image is captured by different spectral wavelengths of the spectral elements 302. The projector 100 generates corrected image data by correcting the input image data based on N2 spectral imaging data. Then, the projector 100 generates a projected image based on the corrected image data and projects it on the screen SC.

As described above, N1 is an integer of 2 or more, and N2 is an integer smaller than N1, so that the high-precision mode is used when strict image quality adjustment is desired or when image quality is important such as for theater applications. Can be handled with. On the other hand, if you want to perform a simple check, you can check the image quality in a shorter time than in the high-precision mode by setting the high-speed mode. Therefore, the convenience of the user is improved in adjusting and checking the image quality. In addition, when you want to perform a simple check, for example, when you want to check for abnormalities in projection, especially when you want to check in a short time between projections, or when you want to perform a check in a short time between projections, image quality in offices, educational sites, etc. is not relatively important. If you want to check in the case.

Further, in the high-precision mode, predetermined information regarding color is measured based on N1 first imaging data (spectral imaging data), and the corrected data obtained by correcting the input image data based on the measurement result is corrected. It is used as image data. In the present embodiment, the predetermined information regarding the color is the absolute value of each color of RGB and the color unevenness in the projection surface, but either one may be used, and stains due to stains, deposits, etc. may be detected. Information may be used, and appropriate information can be applied. Thereby, the image quality adjustment regarding the color can be performed by using the spectroscopic element 302.

Further, the conversion data 167c that associates the color spectral result obtained from the N1 spectral imaging data in the high-precision mode with the color spectral result obtained from the N2 spectral imaging data in the high-speed mode is previously provided. Have acquired. In the high-speed mode, the color spectral results obtained from the N2 spectral imaging data are converted into information corresponding to the color spectral results obtained from the N1 spectral imaging data based on the conversion data 167c. Then, the data obtained by correcting the image data input based on the converted information is used as the corrected image data. As a result, the deviation of the peak wavelength of each color that occurs when the spectrum is estimated using only a relatively small number of spectral imaging data can be set within an allowable range, the measurement accuracy can be sufficiently maintained, and appropriate correction can be performed.

Further, since the projector 100 projects a projected image including an OSD menu including options for a high-precision mode and a high-speed mode, it becomes easy to select each mode. As a result, the number of switches can be reduced as compared with the case where the switch for selecting each mode is provided on the operation panel 131 or the remote controller, and the existing operation panel or remote controller can be easily used.

Further, the spectroscopic element 302 has a pair of reflective films 304 and 305 and a gap changing portion 306 capable of changing the gap dimensions of the pair of reflective films 304 and 305, and is on the optical path of light incident on the image pickup element 303. It is an arranged wavelength variable interference filter. This makes it easier to perform high-precision measurement to short-time measurement while maintaining high resolution as compared with the case of the wavelength dispersive type. Although the case of providing two operation modes consisting of a high-precision mode and a high-speed mode is illustrated, an operation mode having different imaging conditions such as the number of spectral wavelengths λi and an exposure time is added, and the number of operation modes is set to three or more. You may.

For example, in the example of FIG. 11, even in the high-speed mode, the S / N ratio of the measured value is increased and the measurement reproducibility is improved by making the exposure time the same as that in the high-precision mode. Not limited. The exposure time may be shorter than the high-precision mode, and another high-speed mode with a shorter measurement time may be further provided. When the high-speed mode shown in FIG. 11 is described as the second mode, other high-speed modes may be described as the third mode. Further, the high-speed mode shown in FIG. 11 may be omitted, and another high-speed mode may be used as an example of the "second mode" of the present invention.

In the second embodiment, a case where a configuration corresponding to a spectroscopic image pickup device is integrally provided with the projector 100 has been illustrated, but as shown in FIG. 13, the display system 1A has a spectroscopic image pickup device 400 that is separate from the projector 100. You may prepare. Further, although FIG. 13 shows a case where the projector 100 and the spectroscopic image pickup device 400 are connected by wire using a cable 4, the projector 100 and the spectroscopic image pickup device 400 may be wirelessly connected by using wireless communication.

[Third Embodiment]
FIG. 14 is a diagram showing a display system 1 according to a third embodiment.
In the third embodiment, two projectors 100 are connected by a cable 5, data communication is performed between the projectors 100, and the colors of the images displayed on the screen SC by each projector 100 are matched. The projector 100 that displays an image on the left side of the screen SC when viewed from the direction facing the screen SC is referred to as a projector 100A, and the projector 100 that displays an image on the right side of the screen SC is referred to as a projector 100B. The number of projectors 100 connected is not limited to two, and may be three or more. In addition, instead of the wired connection using the cable 3, a wireless connection using wireless communication may be used.

Since the configurations of the projector 100A and the projector 100B are the same as the configurations of the projector 100 shown in FIG. 9, the illustration is omitted. Hereinafter, for convenience of explanation, the components of the projector 100A are indicated by the reference numerals “A” to the components in FIG. 9 corresponding to the components, and the components of the projector 100B correspond to the components. Reference numerals “B” are added to the components in FIG. For example, the control unit 150 of the projector 100A is referred to as "control unit 150A", and the control unit 150 of the projector 100B is referred to as "control unit 150B". The projector 100A and the projector 100B are each connected to the image supply device 200 via the cable 3, and an image based on the image signal supplied from the image supply device 200 is displayed on the screen SC.

The projector 100A operates as a master machine, and the projector 100B operates as a slave machine. That is, the projector 100B operates according to the control of the projector 100A. The projector 100A, which is a master machine, instructs the projector 100B to calculate each value necessary for generating the correction parameter 166, and instructs the projector 100B to perform image processing using the correction parameter 166.

The area where the projector 100A projects the image light is referred to as "projection area 20A", and the area where the projector 100B projects the image light is referred to as "projection area 20B". A part of the projection area 20A and the projection area 20B overlap each other.

In the third embodiment, the projector 100A has a function of accepting color matching between the projectors 100A and 100B connected to each other. For example, the control unit 150A displays an OSD menu including a key for instructing color matching by the projection control unit 171A, and receives a color matching instruction from the user. In this case, the control unit 150A displays a key for instructing color matching in the high-precision mode and a key for instructing color matching in the high-speed mode as keys for instructing color matching. The key for instructing color matching in high-precision mode is an example of "key for instructing image quality adjustment in high-precision mode", and the key for instructing color matching in high-speed mode is "key for instructing image quality adjustment in high-speed mode". This is an example.

When the color matching in the high-precision mode is instructed, the control unit 150A causes the control units 150A and 150B to execute the processes of steps S103 to S107 shown in FIG. As a result, the correction parameter 166A of the projector 100A and the correction parameter 166B of the projector 100B are generated. As a result, the correction parameters 166A and 166B corresponding to the correction parameters 166 in the high precision mode of the second embodiment are generated. In this case, in the projector 100B, the spectroscopic imaging unit 137B may perform only the processing related to the imaging corresponding to steps S103 to S105 in FIG. 12, and transmit the N1 spectral imaging data obtained by the imaging to the projector 100A. .. In this case, in the projector 100A, the correction parameter 166B of the projector 100B is generated by performing the processing of the remaining steps S106 to S107 using the spectral imaging data transmitted from the projector 100B.

The control unit 150A corrects the image data input to the projectors 100A and 100B by the correction parameters 166A and 166B, and displays the image corresponding to the corrected image data on the screen SC, respectively. As a result, the projected images of the projectors 100A and 100B are color-matched. Although these correction parameters 166A and 166B correct the absolute value and color unevenness of each color of the projectors 100A and 100B, other correction methods capable of aligning the colors of the projectors 100A and 100B may be used.

For example, the control unit 150A acquires N1 spectral imaging data obtained by the spectral imaging unit 137A of the projector 100A and N1 spectral imaging data obtained by the spectral imaging unit 137B of the projector 100B. Next, the control unit 150A measures the color difference between the projectors 100A and 100B based on the acquired spectroscopic imaging data. Subsequently, the control unit 150A may generate a correction parameter for correcting the color displayed by the projector 100B to the color of the projector 100A based on the measurement result.

When the color matching in the high-speed mode is instructed, the control unit 150A causes the control units 150A and 150B to sequentially execute the processes of steps S109 to S113, S107 and S108 shown in FIG. As a result, the correction parameter 166A of the projector 100A and the correction parameter 166B of the projector 100B are generated. As a result, the parameters 166A and 166B corresponding to the correction parameters 166 in the high-speed mode of the second embodiment are generated. In this case, in the projector 100B, the spectroscopic imaging unit 137B may perform only the processing related to the imaging corresponding to steps S109 to S111 in FIG. 12, and transmit the N2 spectral imaging data obtained by the imaging to the projector 100A. .. In this case, in the projector 100A, the correction parameter 166B of the projector 100B is generated by performing the processing of the remaining steps S112 to S113 and S107 using the spectral imaging data transmitted from the projector 100B.

Even in the high-speed mode, the control unit 150A measures the color difference between the projectors 100A and 100B, and based on the measurement result, a correction parameter for correcting the color displayed by the projector 100B to the color of the projector 100A may be generated. good. In this case, the time required to generate the correction parameters can be reduced as compared with the case of adopting the high-speed mode for generating the correction parameters 166A and 166B for correcting the absolute value and the color unevenness of each color of the projectors 100A and 100B. It is possible to shorten the time.

That is, in the display system 1 of the third embodiment, in the high-precision mode, the spectroscopic image pickup units 137A and 137B capture the N1 spectroscopic images at different spectral wavelengths for each of the projected images of the projectors 100A and 100B. Generate imaging data respectively. Next, either the projectors 100A or 100B generate corrected image data obtained by correcting the input image data based on the N1 spectral imaging data. The N1 spectral imaging data is an example of the "N1 first imaging data" of the present invention, and the input image data is an example of the "first image data" of the present invention. The corrected image data is an example of the "first corrected image data" of the present invention.
Next, the projectors 100A and 100B project the projected image based on the corrected image data onto the screen SC which is the projection surface, respectively. The projected image is an example of the "first projected image" of the present invention.

On the other hand, when the display system 1 is in the high-speed mode, the spectroscopic imaging units 137A and 137B generate N2 spectral imaging data imaged at different spectral wavelengths for each projected image of the projectors 100A and 100B. Next, either the projectors 100A or 100B generate corrected image data obtained by correcting the input image data based on the N2 spectral imaging data. Next, the projectors 100A and 100B project the projected image based on the corrected image data onto the screen SC which is the projection surface, respectively. The N2 spectral imaging data is an example of the "N2 second imaging data" of the present invention, and the corrected image data is an example of the "second corrected image data" of the present invention. The image is an example of the "second projection image" of the present invention.

In this way, the high-precision mode can be used when the projectors 100A and 100B are strictly color-matched, or when the image quality is important for theater applications and the like. On the other hand, if you want to perform a simple check, you can check the image quality in a shorter time than in the high-precision mode by setting the high-speed mode. Therefore, the convenience of the user is improved in adjusting and checking the image quality when a plurality of projectors 100 are used. The configuration corresponding to the spectroscopic imaging device may not be limited to the configuration provided for each of the projectors 100A and 100B. For example, only one of the projectors 100A and 100B may be provided with a configuration corresponding to the spectroscopic imaging device. May be good. Further, as shown in FIG. 15, the display system 1B may include a spectroscopic image pickup device 400 separate from the projectors 100A and 100B, and the spectroscopic image pickup device 400 may capture the projected images of the projectors 100A and 100B. ..

The present invention is not limited to the configuration of each of the above embodiments, and can be implemented in various embodiments without departing from the gist thereof.
In each of the above embodiments, the case where the present invention is applied to the display systems 1, 1A, the projectors 100, 100A, 100B shown in FIG. 9 and the like, and the control method thereof has been described, but the present invention is not limited thereto. For example, the case of correcting at least one of the absolute value of each color of RGB, the color unevenness in the projection surface, and the color between a plurality of projectors 100 has been illustrated, but the measurement target and the correction target may be changed as appropriate. good. Further, although the case where the light to be measured is visible light is illustrated, it may be other than visible light, and may be infrared rays or far infrared rays, for example. Further, although the case where the wavelength tunable interference filter is used for the spectroscopic element 302 has been described, another wavelength scanning type filter may be used.

Further, either a transmissive liquid crystal panel or a reflective liquid crystal panel may be applied to the three liquid crystal panels 115 included in the optical modulation device 113. Further, instead of the three liquid crystal panels 115, a configuration in which one liquid crystal panel and a color wheel are combined may be applied. Further, various known methods such as a method using three digital mirror devices (DMD) and a DMD method combining one digital mirror device and a color wheel may be applied to the optical modulation device 113.

Further, the configuration of each part shown in FIG. 9 or the like may be realized by hardware or may be realized by the cooperation of hardware and software, and independent hardware resources may be used as shown in the figure. It is not limited to the configuration to be arranged.

Further, the processing unit of the flowchart shown in FIG. 12 is a division of the processing by the control unit 150 according to the main processing contents. The embodiment is not limited by the method and name of dividing the processing unit of each flowchart. Further, the processing order of the above flowchart is not limited to the illustrated example.

Further, the control program 161 may be stored in an external device or device and acquired via the communication unit 139 or the like. It is also possible to record on a recording medium that is readable by a computer. As the recording medium, a magnetic or optical recording medium or a semiconductor memory device can be used. Specific examples thereof include portable or fixed recording media such as flexible disks, various optical disks, magneto-optical disks, flash memories, and card-type recording media. Further, the recording medium may be a non-volatile storage device such as RAM, ROM, or HDD, which is an internal storage device included in the image display device.

1 ... Display system, 3 ... Cable, 4 ... Cable, 5 ... Cable, 10 ... Imaging device, 11 ... Incident optical system, 12 ... Spectral element, 13 ... Imaging element, 14 ... Light source unit, 14a ... Substrate, 15 ... Reflection Film, 16 ... Reflective film, 17 ... Gap changing part, 20A ... Projection area, 20B ... Projection area, 30 ... Display device, 40 ... Storage device, 50 ... Processing device, 60 ... Control unit, 61 ... Imaging control unit, 70 ... Processing unit, 71 ... Measurement band indicator, 80 ... Storage unit, 81 ... Measurement band data storage unit, 82 ... Setting data storage unit, 83 ... Conversion data storage unit, 100 ... Projector, 100A ... Projector, 100B ... Projector, 101 ... 1st substrate, 102 ... 2nd substrate, 103 ... 3rd substrate, 104 ... Reflective film, 105 ... Electrostatic actuator, 106 ... Bonding layer, 110 ... Projector, 111 ... Light source, 112 ... Spectral element, 113 ... Optical modulator, 115 ... LCD panel, 117 ... Optical unit, 120 ... Drive unit, 121 ... Light source drive circuit, 123 ... Optical modulator drive circuit, 131 ... Operation panel, 133 ... Remote control light receiving unit, 135 ... Input interface, 137 ... Spectral imaging unit, 137A ... Spectral imaging unit, 137B ... Spectral imaging unit, 139 ... Communication unit, 141 ... Image interface, 143 ... Image processing unit, 145 ... Frame memory, 150 ... Control unit, 150A ... Control unit, 150B ... Control unit, 160 ... Storage unit, 161 ... Control program, 162 ... Adjustment image data, 163 ... Pattern image data, 164 ... Setting data, 165 ... Parameters, 166 ... Correction parameters, 166A ... Correction parameters, 166B ... Correction parameters, 167 ... Calibration data, 167a ... Correction data, 167b ... Projection conversion matrix, 167c ... Conversion data, 170 ... Processor, 171 ... Projection control unit, 171A ... Projection control unit, 173 ... Imaging control unit, 175 ... Calculation unit, 180 ... Bus, 200 ... image supply device, 301 ... incident optical system, 302 ... spectroscopic element, 303 ... image pickup element, 304 ... reflective film, 305 ... reflective film, 306 ... gap changing part, 400 ... spectral image pickup device, 400A ... spectral imaging Device, M1 ... High precision mode, M2 ... High speed mode, M3 ... Optimal mode.

Claims (15)

It is a control method of a spectroscopic image pickup device including an image pickup element and a spectroscopic element.
When the spectroscopic image pickup device is in the first mode,
The spectroscopic image pickup device generates a first measurement spectrum consisting of N1 wavelengths in which an object is imaged by different output wavelengths of the spectroscopic element.
When the spectroscopic image pickup device is in the second mode,
The spectroscopic image pickup device generates a second measurement spectrum consisting of N2 wavelengths in which the object is imaged by different output wavelengths of the spectroscopic element.
A control method for a spectroscopic image pickup apparatus, wherein N1 is an integer of 2 or more and N2 is an integer smaller than N1. The method for controlling a spectroscopic image pickup apparatus according to claim 1.
A method for controlling a spectroscopic image pickup apparatus, which derives a spectrum of an object based on the first measurement spectrum generated in the first mode or the second measurement spectrum generated in the second mode. The method for controlling a spectroscopic image pickup apparatus according to claim 2.
The spectrum of the object is obtained by converting the second measurement spectrum obtained in the second mode by using the conversion data for estimating the spectrum consisting of the N1 wavelength from the spectrum consisting of the N2 wavelengths. A method of controlling a spectroscopic image pickup device to estimate. The method for controlling a spectroscopic image pickup apparatus according to claim 2 or 3.
The spectrum of the object has the first measurement spectrum obtained in the first mode and the second measurement spectrum obtained in the second mode.
A method for controlling a spectroscopic imaging device in which the intervals between output wavelengths are different based on at least one of the first measurement spectrum and the second measurement spectrum. The method for controlling a spectroscopic image pickup apparatus according to any one of claims 1 to 4.
The spectroscopic element has a pair of reflective films and a gap changing portion capable of changing the gap size of the pair of reflective films, and is a wavelength variable interference filter arranged on an optical path of light incident on the image pickup element. There is a control method for a spectroscopic image sensor. A spectroscopic image pickup device including an image pickup element and a spectroscopic element.
When the spectroscopic image pickup device is in the first mode,
The spectroscopic image pickup device generates a first measurement spectrum consisting of N1 wavelengths in which an object is imaged by different output wavelengths of the spectroscopic element.
When the spectroscopic image pickup device is in the second mode,
The spectroscopic image pickup device generates N2 second measurement spectra in which the object is imaged by making the output wavelengths of the spectroscopic elements different.
A spectroscopic image pickup device in which N1 is an integer of 2 or more and N2 is an integer smaller than N1. A computer program for identifying an object based on image pickup data obtained by a spectroscopic image pickup device equipped with an image pickup element and a spectroscopic element.
In the first mode of the spectroscopic image pickup apparatus, a process of imaging the object with different output wavelengths and generating a first measurement spectrum consisting of N1 wavelengths which are integers of 2 or more.
In the second mode of the spectroscopic image pickup apparatus, at least one of a process of imaging the object with different output wavelengths and generating a second measurement spectrum consisting of two wavelengths of N2, which is an integer smaller than N1. A computer program that causes a computer to run. A method for controlling a display system including a spectroscopic image pickup device including an image pickup element and a spectroscopic element, and a projector that projects a projection image based on image data onto a projection surface.
When the display system is in the first mode
The projected image was imaged by the spectroscopic imaging device with different spectral wavelengths of the spectroscopic elements to generate N1 first imaging data.
The projector generates the first corrected image data obtained by correcting the first image data based on the N1 first imaging data.
The projector projects a first projection image based on the first corrected image data onto the projection surface.
When the display system is in the second mode
The projected image was imaged by the spectroscopic imaging device with different spectral wavelengths of the spectroscopic elements to generate N2 second imaging data.
The projector generates a second corrected image data obtained by correcting the first image data based on the N2 second imaging data.
The projector includes projecting a second projected image based on the second corrected image data onto the projection surface.
A display system control method in which N1 is an integer of 2 or more and N2 is an integer smaller than N1. The display system according to claim 8, wherein predetermined information regarding color is measured based on the N1 first image pickup data, and the data obtained by correcting the first image data based on the measurement result is the first corrected image data. Control method. A predetermined measurement result of a color obtained from the N1 first imaging data in the first mode and a predetermined measurement result of a color obtained from the N2 second imaging data in the second mode. Obtain the conversion data that associates with and in advance,
In the second mode, the predetermined measurement result obtained from the N2 second imaging data corresponds to the predetermined measurement result obtained from the N1 first imaging data based on the conversion data. The control method of the display system according to claim 9, wherein the data obtained by converting the data into the information to be used and correcting the first image data based on the converted information is used as the second corrected image data. The control method for a display system according to any one of claims 8 to 10, wherein the projector projects a projected image including an OSD menu including the options of the first mode and the second mode. The spectroscopic element has a pair of reflective films and a gap changing portion capable of changing the gap size of the pair of reflective films, and is a wavelength variable interference filter arranged on an optical path of light incident on the image pickup element. A control method for a display system according to any one of claims 8 to 11. It is a control method of a projector that projects a projected image based on image data on a projection surface.
When the projector is in the first mode
The first image pickup data of N1 pieces of the projected image imaged by the spectroscopic image pickup apparatus with different spectral wavelengths of the spectroscopic elements of the spectroscopic image pickup apparatus were acquired.
Based on the N1 first imaging data, the first corrected image data obtained by correcting the first image data is generated.
A first projection image based on the first corrected image data is projected onto the projection surface.
When the projector is in the second mode
The second imaging data of N2 pieces captured by the spectroscopic imaging device with the projected image at different spectral wavelengths of the spectroscopic elements of the spectroscopic imaging device was acquired.
Based on the N2 second imaging data, the second corrected image data obtained by correcting the first image data is generated.
Including projecting a second projection image based on the second corrected image data onto the projection surface.
A projector control method in which N1 is an integer of 2 or more and N2 is an integer smaller than N1. A display system including a spectroscopic image pickup device including an image pickup element and a spectroscopic element, and a projector that projects a projection image based on image data onto a projection surface.
When the display system is in the first mode
The spectroscopic imaging device generates N1 first imaging data obtained by imaging the projected image at different spectral wavelengths of the spectroscopic element.
The projector generates a first corrected image data corrected with the first image data based on the N1 first imaging data, and projects a first projected image based on the first corrected image data on the projection surface. death,
When the display system is in the second mode
The spectroscopic imaging device generates N2 second imaging data obtained by imaging the projected image at different spectral wavelengths of the spectroscopic element.
The projector generates a second corrected image data obtained by correcting the first image data based on the N2 second imaging data, and a second projected image based on the second corrected image data is displayed on the projection surface. Including projecting
A display system in which N1 is an integer greater than or equal to 2 and N2 is an integer smaller than N1. A projector that projects a projected image based on image data onto a projection surface.
When the projector is in the first mode
The first image pickup data of N1 pieces of the projected image imaged by the spectroscopic image pickup apparatus with different spectral wavelengths of the spectroscopic elements of the spectroscopic image pickup apparatus were acquired.
Based on the N1 first imaging data, the first corrected image data obtained by correcting the first image data is generated.
A first projection image based on the first corrected image data is projected onto the projection surface.
When the projector is in the second mode
The second imaging data of N2 pieces captured by different spectral wavelengths of the spectral elements of the spectral imaging device was acquired by the spectral imaging device for the projected image.
Based on the N2 second imaging data, the second corrected image data obtained by correcting the first image data is generated.
Including projecting a second projection image based on the second corrected image data onto the projection surface.
A projector in which N1 is an integer greater than or equal to 2 and N2 is an integer smaller than N1.

Images