JP2001165775A - Spectrum image sensor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To create a correct differential spectrum image without being affected by the movements of a subject, illumination, etc. SOLUTION: Real images divided into two onto an image pickup element 4 such as a CCD are simultaneously formed by an optical system 2. An optical filter 3 with regions 3a and 3b of different transmission spectral characteristics different from each other is arranged in the optical system 2 to form spectrum images A and B on the image pickup element 4. Differential values between pixels at the corresponding locations of the spectrum images A and B are extracted to create the differential spectrum image of a subject.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a spectral image sensor for generating a differential spectrum image to be monitored by calculating a difference between positions of a plurality of spectral images and determining an abnormality or the like.

[0002]

2. Description of the Related Art In recent years, it has been noted that various kinds of spectral imaging are very effective for advanced information recognition. In particular, in various fields such as medicine, biology, remote sending, food quality control, and fire monitoring, there is a demand for a spectral image processing technology that can easily perform advanced spectral analysis as a two-dimensional image.

The spectral image processing techniques developed so far include a high spatial resolution color camera using a liquid crystal filter, a multispectral camera integrated with a spectroscope, and a Fourier transform type multispectral imaging.
These have the problems that ultra-precision control of the optical system is required, it takes time to match the spectrum, the spatial resolution is poor, the device is too large, and the like. Etc. have proposed.

This differential spectrum image sensor comprises a wavelength variable interference filter (Japanese Patent Laid-Open No. 8-285688) capable of changing the spectral transmittance, a color CCD camera and a computer, and a plurality of narrow wavelength filters obtained by the wavelength variable interference filter. A differential spectrum image can be obtained easily and quickly by changing the band spectrum characteristic periodically and performing an inter-image operation that takes the difference between the narrow band spectrum image before and after the change.

[0005]

However, in this differential spectrum image sensor, when the object moves at a high speed or the illumination changes at a high frequency, the characteristics of the wavelength variable interference filter are periodically changed. In the meantime, the position and brightness of the subject change, and there is a problem that a correct spectral image cannot be obtained.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a spectrum image sensor capable of obtaining a correct differential spectrum image without being affected by movement of a subject or illumination.

[0007]

In order to achieve this object, the present invention is configured as follows. The spectral image sensor of the present invention is arranged on one image sensor, an optical system for forming a plurality of real images on the image sensor, and a real image forming system split by the optical system. A difference image generation unit that generates a differential spectrum image of a subject by taking a difference value between pixels at opposing positions from an optical filter having different transmission spectrum characteristics and a plurality of spectrum images simultaneously imaged by an image sensor. And characterized in that:

According to such an image spectrum sensor of the present invention, a plurality of images having different spectral characteristics are formed on the image sensor at the same time. There is no time lag,
Even if the subject moves at a high speed or the illumination changes, a correct differential spectrum image can always be obtained. At the same time, a correct differential spectrum image can be obtained even if the operation speed of the image processing system is low.

Here, the optical system of the spectrum image sensor is composed of a lens system having a plurality of optical axes for forming a plurality of divided real images on an image sensor.

The optical system of the spectrum image sensor may be provided with a beam splitter which is disposed in front of the image sensor and forms a plurality of divided real images on the image sensor.

Further, as an optical system and an optical filter of the spectrum image sensor, minute optical filters having different spectral characteristics corresponding to minute regions obtained by dividing the image sensor into a plurality of parts are dispersedly arranged, and the same object to be imaged is picked up on the image sensor. May have different spectral characteristics. In this case, as the optical filter, minute optical filters having different spectral characteristics are arranged in a staggered or linear manner.

As the image pickup device of the spectrum image sensor, a CCD or a light receiving device having two or more devices can be used.

[0013]

FIG. 1 is a block diagram showing an embodiment of a spectral image sensor according to the present invention.

In FIG. 1, the spectrum image sensor of the present invention comprises a monitoring camera 1, a signal processing device 5, and a monitor 10. The monitoring camera 1 is provided with an optical system 2, an optical filter 3, and an image sensor 4 such as a CCD. The optical system 2 forms, for example, a real image of the object to be divided into two on the image sensor 4.

The optical filter 3 has a filter section having different transmission spectrum characteristics arranged in an imaging system of two real images divided by the optical system 2, and is divided into two with different transmission spectrum characteristics on an image pickup device 4. A real image is formed.

The signal processing device 5 comprises an MPU 6, an AD converter 7, an image memory 8, and an output interface 9. The AD converter 7 converts an image signal of a real image divided into two with different transmission spectrum characteristics simultaneously formed on the image sensor 4 of the monitoring camera 1 into digital data, and stores the digital data in the image memory 8 as image data.

For this reason, in the image data of the image memory 8, the spectral images A and B of the same object having different transmission spectral characteristics acquired at the same time are stored. The MPU 6 differentiates the object by taking the difference between the pixels at the opposite positions of the two spectral images A and B of the same object having different transmission spectrum characteristics stored in the image memory 8. A spectrum image C is generated, output via the output interface 9, and displayed on the monitor 10. Further, the MPU 6 performs an abnormality determination such as a fire determination based on the generated differential spectrum image C.

FIG. 2 is an explanatory diagram of the first embodiment of the monitoring camera 1 of FIG. In FIG. 2, a surveillance camera 1 used in the present invention has two image forming lenses 2a,
2b, areas 4a, 4 divided above and below the image sensor 4.
b, the imaging lenses 2a, 2a,
2b, the real images A and B of the object 11 are independently imaged.

The imaging lenses 2a, 2 having these two optical axes
The optical filter 3 is disposed between the imaging lenses 2a and 2b and the image pickup device 4 with respect to the optical system 2 based on b. The optical filter 3 is divided into a first filter section 3a that transmits light from the imaging lens 2a and a second filter section 3b that transmits light from the imaging lens 2b.

The first filter section 3a has a narrow band transmission spectrum characteristic of a center wavelength λ1, while the second filter section 3b has a narrow band transmission spectrum characteristic of a different center wavelength λ2.

For this reason, the divided area 4 of the image sensor 4
a, a real image A of the object 11 having a narrow band transmission spectrum characteristic of the center wavelength λ1 of the first filter unit 3a is formed, and the area 4b is narrowed at the center wavelength λ2 of the second filter unit 3b. A real image B having a band transmission spectrum characteristic is formed.

As the first filter section 3a and the second filter section 3b having the narrow-band transmission spectrum characteristic shown in FIG. 2, a tunable filter disclosed in Japanese Patent Application Laid-Open No. 8-285688 can be used. By using the wavelength tunable filter, the band of the transmission spectrum can be arbitrarily varied.

FIG. 3 is an explanatory view of the structure of the wavelength variable interference filter used for the two divided filter sections 3a and 3b of the optical filter 3 of the present invention. Wavelength variable interference filter 13
Are arranged to face each other via a piezoelectric element 18 in which a pair of glass substrates 14 and 16 on which metal films 15 and 17 of Au or the like are vapor-deposited are vertically arranged, and a gap d1 is formed therebetween. The piezoelectric element 18 can change the gap d1 by receiving a DC voltage from the drive voltage source 19.

The wavelength tunable interference filter 13 causes light having a plurality of types of spectrums to pass through due to multiple interference caused by transmission of the light from the glass substrate 16 through the metal films 15 and 17. The light is selectively transmitted corresponding to D1.

FIG. 4 shows the tunable interference filter 13 of FIG.
For example, a wavelength band having a center wavelength λ1 = 4.3 μm of resonance radiation in CO ₂ peculiar to a flame at the time of fire and a center wavelength λ2 not including the center wavelength λ1 = 4.3 μm.
= Spectrum characteristics when a wavelength band of 3.8 μm is set.

That is, FIG. 4A is a wavelength spectrum peculiar to a flame caused by a fire, and has a peak due to resonance radiation of CO ₂ peculiar to the flame near a center wavelength λ1 = 4.3 μm.
FIG. 4B shows bandpass characteristics of filters used in the first filter unit 3a and the second filter unit 3b in FIG. 2 by the wavelength variable interference filter 13 in FIG.

For example, by setting a narrow-band transmission spectrum characteristic due to multiple interference having a wavelength band of a center wavelength of 4.3 μm as shown by a solid line at a driving voltage of 0 V, the first band in FIG.
The function of the filter unit 3a is realized. On the other hand, a change in the drive voltage of the piezoelectric element 18 by the drive voltage source 19 causes
By setting a plurality of narrow-band transmission spectrum characteristics due to multiple interference shifted on the wavelength axis so as to have a transmission characteristic in a wavelength band of center wavelength λ2 = 3.8 μm as shown by a broken line in FIG. The function of the second filter unit 3b in FIG. 1 can be realized.

If the wavelength spectrum of the object to be photographed is known in advance, as in the flame of FIG. 4A, the center wavelengths λ1 and λ2 of the filter sections 3a and 3b are determined so that the wavelength tunable interference filter can be obtained. 3, the gap d1 is uniquely determined. In this case, a member for providing the gap d1 corresponding to the center wavelengths λ1 and λ2 is interposed instead of the piezoelectric element 18 by the drive voltage source 19, so that the gap d1 is fixed. What is necessary is just to determine the narrow band transmission spectrum characteristic.

A wavelength variable interference filter as shown in FIG. 3 is provided for one of the filter sections, and the gap d1 is changed by the drive voltage source 19 so that the wavelength spectrum characteristic of the real image by the one filter section is changed. Is also good.

FIG. 5 is a block diagram of the signal processing device 5 shown in FIG.
It is a block diagram of the processing function realized by program control of PU6. 5, in the image memory 8, two images A and B having different transmission spectrum characteristics formed on the divided areas 4a and 4b on the image sensor 4 by the surveillance camera 1 having the structure of FIG. The image signal of the image sensor 4 is fetched by the image memory 8 by the AD converter 7, so that two image data A and B having different transmission spectrum characteristics can be obtained.

The difference image generator 12a calculates the absolute value of the difference between the pixels of the image data A and B stored in the image memory 8 to generate the difference image data S. The difference image data S generated in this manner is transmitted to the fire determination processing unit 12.
b, the presence or absence of a flame due to a fire is determined from the differential image data S, that is, the differential spectrum image data S.

FIG. 6 shows a specific example of a processed image in the signal processing of FIG. FIG. 6A is a visible image taken without passing through the optical filter 3 of the surveillance camera. The visible image includes, for example, a light bulb 40 as an example of a high-temperature object, a flame 41 due to a fire, and furniture as an example of an object at the same room temperature. 42 are included.

With respect to such a visible image, the first image shown in FIG.
The output image A obtained from the image sensor 4 by passing infrared rays in the wavelength band of the center wavelength λ1 = 4.3 μm of the CO ₂ resonance radiation peculiar to the flame of the filter unit 3a as shown in FIG. Become. In the output image A, an image of the light bulb 40 having a wavelength spectrum in a wavelength band around λ1 = 4.3 μm and a flame 41 caused by a fire are clearly obtained, and the furniture 42 is hardly obtained as an image.

On the other hand, the image obtained by the infrared ray which has passed through the adjacent center wavelength λ2 = 3.8 μm which does not include the center wavelength λ1 = 4.3 μm peculiar to the flame of the second filter portion 3b of FIG. The output image B has a center wavelength λ2 = 3.
An image of the bulb 40 distributed over a wide wavelength band including 8 μm can be obtained clearly. On the other hand, since the intensity of infrared rays due to the flame is smaller than the center wavelength λ1 inherent in the flame,
An unclear image of the flame 41 is obtained. Furthermore, the furniture 42 is hardly obtained as an image.

By calculating the absolute value of the difference of each pixel for the output image A and the output image B by the difference image generation unit 12a in FIG. 5, the difference image S shown in FIG.
Generate In the difference image S, output images A,
B, the images of the light bulb 40 and the furniture 42 that have similar outputs are removed, and the flame 41 having a large absolute value due to the difference between the output image A and the output image B is slightly inferior to the output image A but the difference image S Can be obtained clearly inside.

The difference image data S generated in this manner is supplied to the fire determination processing unit 12b shown in FIG. 5, and the determination processing as to whether or not the flame is caused by a fire is performed.

FIG. 7 is a flowchart of the fire monitoring process by the signal processing device 5 shown in FIG. First, image data is transferred from the image sensor 4 to the image memory 8 in step S1. At this time, real images A and B having different transmission spectral characteristics of the subject 11 are formed on the two divided areas 4a and 4b in FIG. Are stored as image data A and B.

Next, in step S2, the area of the center wavelength λ1 (area 4) is selected from the image data stored in the image memory 8.
a) is extracted as image data A. Then step S
In step 3, the area (area 4b) of the center wavelength λ2 is extracted as image data B from the image data in the image memory 8. Subsequently, in step S4, the difference image data S is generated by obtaining the absolute value of the difference between the image data A and B for each pixel.

Subsequently, in step S5, an abnormality determination process based on the difference image data S, for example, a fire determination process is performed. In step S6, the presence or absence of an abnormality is checked. If there is an abnormality, an alarm is output in step S7.

The difference image data S generated here is generated from the image data A and B taken simultaneously.
The correct differential spectrum image can be represented without being affected by the flicker of the flame.

As a specific embodiment using the differential spectrum image sensor of the present invention, an automatic fruit sorting apparatus will be described.

Conventionally, near-infrared spectroscopy has been used as an effective method for non-destructively measuring the sugar content of fruits (Literature: "Sifting of peach by light sensor", Mikio Kimura, Fruit Japan, 46 (2) '91) Near-infrared spectroscopy is to analyze the components of a substance by measuring near-infrared absorption at a specific wavelength by a specific molecule in the substance. In the case of sugar, 1200 nm,
Absorption is observed at 1750 nm and 2270 nm (Reference: “Near-infrared spectroscopy”, FIG. 4.2.38 (P155), 1998)
Published by IPC Co., Ltd. on March 20.

Light such as a peach is illuminated by a lamp or the like, and the scattered reflected light at this time is collected by an optical system such as a lens.
By analyzing light with a spectrometer and extracting light of the required wavelength, and measuring the reflection intensity (absorbance) at each wavelength,
The sugar content of the tested fruit can be measured. Kimura measures this by the reflection method or the transmission method.

However, according to Kimura's method, in order to prevent the difference in the sugar content between the peach parts, it was necessary to arrange the peach positions beforehand before measurement. In addition, the size and shape of the peach are important items for quality inspection, but a separate measurement principle must be used for this.

By applying the differential spectrum image sensor of the present invention to quality inspection of fruits such as peach, the sugar content,
It is possible to inspect all inspection items such as size, shape, discoloration, and stain at once.

The measurement of the sugar content was performed in the original peach spectrum.
It is measured by taking a differential spectrum image around 200 nm. The part showing the average sugar content of peach is
Since the portion is on the equator at 90 degrees lateral to the suture line, this portion is first specified by the differential spectrum image sensor. This can be identified from the measured images by the topological features of the peach apex and the suture line.

The transmission wavelength of the optical filter of the differential spectrum image sensor is determined by setting the filter unit λ1 shown in FIG.
Set λ2 to 1100 nm and 200 nm. When a differential spectrum image for this part is obtained, a differential spectrum image with brightness according to the sugar content appears.
At the same time, the size and shape of the fruit are inspected based on the obtained images.

Further, by using a wavelength variable filter as an optical filter to capture an image in the visible light range, it is possible to inspect for discoloration, dirt, and the like. That is, in FIG. 2, after measuring the sugar content by setting the wavelength λ1 of the wavelength tunable filter 3a to 1200 nm and setting the wavelength λ2 of the wavelength tunable filter 3b to 1100 nm, the wavelength λ1 of the filter 3a is set to 600 nm, for example. Wavelength λ2 is 5
By setting it to 00 nm, inspection for discoloration, dirt, etc. in the visible light range is performed.

FIG. 8 is a flowchart of the peach sugar content measuring process by the image sensor of the present invention. First, in step S1, peach image data is transferred from the image sensor 4 to the image memory 8. At this time, the wavelength λ1 of the filter 3a is set to 120
The wavelength λ2 of the filter 3b is set to 1100.
nm, and the two divided areas 4a, 4a in FIG.
A peach image having transmission spectrum characteristics different between 1200 nm and 1100 nm is formed on each of 4b, and this image is stored in the image memory 8 as image data. In the measurement, light is projected by a lamp or the like onto the peach to be imaged, and the scattered reflected light at this time is reflected by the imaging lens 2.
The images are formed on the image pickup devices 4a and 4b by a and 2b.

In the next step S2, a site for measuring the sugar content at 90 ° beside the peach suture line is specified from the image. Next, step S3
From the image data stored in the image memory 8 with the wavelength λ1
Extracts an area (area 4a) of 1200 nm as image data A.

Next, in step S4, a region (area 4) where the wavelength λ2 is 1100 nm is selected from the image data in the image memory 8.
b) is extracted as image data B. Then step S
In step 5, the absolute value of the difference is determined for each pixel for the image data A and B to generate difference image data S. In step S6, the sugar content of the peach is determined from the difference image data S of the sugar content measurement site.

FIG. 9 is a flowchart of a peach visible light inspection process by the image sensor of the present invention. This visible light inspection processing is performed immediately after the sugar content measurement processing in FIG.

First, at step S1, the wavelength λ1 of the filter 3a of FIG. 2 is set to 600 nm, then at step S2, the wavelength λ2 of the filter 3b of FIG. 2 is set to 500 nm, and at step S3, the image memory 8 Peach image data. Therefore, in the image memory, peach image data whose transmission spectrum characteristics are different between 600 nm and 500 nm is stored in the image memory 8.

In the next step S4, an area (area 4a) having a wavelength λ1 of 600 nm is extracted as image data C from the image data stored in the image memory 8 in step S5.
And the wavelength λ2 is 500 from the image data in the image memory 8.
The area of nm (area 4b) is extracted as image data D. Subsequently, in step S6, the size and shape of the peach are selected based on the image data C.

Subsequently, in step S7, the absolute value of the difference is determined for each pixel of the image data C and D to generate difference image data S2. In step S8, the color is changed by the difference image data S.
Identify dirt. Then, in step S9, the peach with discoloration or dirt is removed, and in step S10, the peach is sorted according to the sugar content and the size and shipped.

The measurement of the sugar content shown in FIG. 8 and the measurement of the visible light shown in FIG. 9 are performed by the optical filters 3a and 3a in the image sensor shown in FIG.
The switching may be performed by one unit by switching the setting wavelength of 3b, or two image sensors of the present invention may be installed separately so as to be at the same shooting position.

As described above, according to the automatic fruit sorting apparatus using the image sensor of the present invention, fruits such as peach flowing at high speed on a belt conveyor can be inspected for sugar content, shape, etc. without being confused by the speed. , Sorting can be performed.

Further, it is possible to measure the ripeness of fruits in a cultivation state before the fruits are collected. As a result, only fruits having an appropriate ripeness can be collected and shipped at an appropriate time, which can be an effective technique for realizing so-called precision agriculture (precision farming).

FIG. 10 is an explanatory diagram of a second embodiment of the surveillance camera 1 of FIG. 1. In this embodiment, a subject is divided into two images by using a beam splitter to form an image. It is characterized by doing.

In FIG. 10, first, the optical system 2 uses an imaging lens 14 constituting a lens system having only one optical axis with respect to the lens system having two optical axes shown in FIG. The image of the image lens 14 is divided into two by the beam splitter 20, and real images A and B are formed on the areas 4a and 4b of the image sensor 4, respectively.

The optical filter 3 is located immediately before the image pickup device 4.
Are arranged, and the optical filter 3 is divided into filter sections 3a and 3b. The filter section 3a has a narrow band transmission spectrum characteristic of a center wavelength λ1, and the filter section 3b has a narrow band transmission spectrum characteristic of a different center wavelength λ2. For example, wavelengths as shown in FIG. 4 are set as λ1 and λ2.

As the filter sections 3a and 3b, FIG.
Or an interference filter having a gap d1 corresponding to each of the wavelengths λ1 and λ2 without having the drive voltage source 19 is used.

FIG. 11 shows the beam splitter 20 of FIG.
Has been taken out. The beam splitter 20 is composed of a linear filter similar to a Fresnel lens, and splits the incident light 21 to the beam splitter 20 into an outgoing light 22 having a downward deflection direction and an outgoing light 23 having an upward deflection direction. And emit.

As a result, as shown in FIG. 10, the light from the imaging lens 14 is divided into two parts,
4b, real images A and B having different transmission spectrum characteristics of the filter portions 3a and 3b of the optical filter 3 arranged immediately before can be simultaneously formed.

FIG. 12 shows a third embodiment of the surveillance camera shown in FIG. 1. In this embodiment, an optical system 2 and an optical filter which divide an object into two and give different wavelength spectrum characteristics to the divided images. The third feature is realized by one filter unit.

In FIG. 12, the optical system 2 of the monitoring camera 1
Is realized by an imaging lens 2c constituting a lens system having only one optical axis, as in the second embodiment of FIG. 10, and a micro optical filter 24 is disposed immediately before the image sensor 4. . In the micro optical filter 24, minute narrow band filter elements corresponding to the image pickup device 4 such as a CCD in a one-to-one correspondence are arranged in the vertical and horizontal directions.

The minute narrow band filter elements have, for example, narrow band transmission spectrum characteristics of center wavelengths λ1, λ2, λ3, and are arranged in a staggered manner.

FIG. 13A shows an RGB filter 25 used in a color camera. This RGB filter 25
Corresponds to the light receiving pixels of the CCD as an image sensor,
R filter element 25a, G filter element 25b, and B
The filter elements 25c are evenly arranged in a staggered manner.

Therefore, when the filter structure of the micro optical filter 24 of the present invention is applied to such an RGB filter 25, for example, the result is as shown in FIG. FIG.
The micro optical filter 24 shown in FIG.
R, G, B filter elements 25a to 25 of B filter 25
corresponding to the center wavelength λ of each filter element.
Micro narrow bandpass filter elements 24a, 24b and 24c having different characteristics of 1, λ2 and λ3 are arranged.

By using such a micro optical filter 24, when an image is read from the image pickup device 4, for example, the center wavelengths λ1, λ2, λ3 correspond to the micro narrow band filter elements 24a, 24b, 24c in this order. When the pixels are read out, image data A having a narrow band transmission spectrum characteristic of the center wavelength λ1, image data B having a narrow band transmission spectrum characteristic of the center wavelength λ2, and a narrow band transmission spectrum characteristic of the center wavelength λ3 are stored in the image memory. Image data C can be obtained.

In this case, when the center wavelengths λ1 and λ2 for flame determination are used as shown in FIG. 4, the image sensor 4 is read out for each of the minute narrow band filter elements 24a and 24b. The image data A and B are stored in the image memory 8 as shown in FIG. 5, from which the difference image data S can be generated by the difference image generation unit 12a.

FIG. 14 shows a fourth embodiment of the surveillance camera 1 of FIG. 1, which is characterized in that an optical line filter is used in place of the micro optical filter 24 of FIG.

In FIG. 14, the optical system 2 comprises an imaging lens 2c constituting a lens system having only one optical axis, similarly to the embodiment of FIG. And an optical line filter 26 for giving different transmission spectrum characteristics to each image.

The optical line filter 26 has a narrow band line filter element 26a having a narrow band transmission spectrum characteristic of a center wavelength λ1 and a narrow band line filter element 26b having a narrow band transmission spectrum characteristic of a center wavelength λ2 in the vertical direction. They are arranged alternately.

FIG. 15A is a side view of a part of the optical line filter of FIG.
Has narrow band line filter elements 26a and 26b having different center wavelengths λ1 and λ2, which are alternately arranged vertically.

When the incident light 27 enters such an optical line filter 26, the narrow band line filter element 26a
Then, the outgoing light 28 according to the narrow band transmission spectrum characteristic of the center wavelength λ1 is emitted, while the outgoing light 29 having the narrow band transmission spectrum characteristic of the center wavelength λ2 is output from the narrow band line filter element 26b. You.

FIG. 15B shows an optical line filter 26.
Is viewed from the side, and an interference filter is used. The narrow band line filter element 26a has a gap 33 formed inside the substrate 30, and the reflection layer 3 is formed inside the gap 33.
1, 32 are provided.

Here, the gap d1 of the gap 33 of the narrow band line filter element 26a gives an interference distance, and this interference distance causes multiple interference when incident light passes through the reflection layers 31, 32, as shown in FIG. A plurality of types of spectra including a part of the narrow band transmission spectrum characteristic having the center wavelength at the specific wavelength λ1 are obtained.

On the other hand, the narrow band line filter element 26b
Is that the interference distance determined by the gap d2 of the gap 33 is the gap d1
This is a characteristic having a plurality of spectra including a narrow band transmission spectrum characteristic of the center wavelength λ2 by the interference distance d2.

The structure of the filter element using the interference filter shown in FIG. 15B is similar to the structure of the minute narrow band filter element 24 a in the minute optical filter 24 shown in FIG.
-24c may be applied.

In the case where the optical line filter 26 shown in FIG. 14 is used, the readout of the image signal from the image sensor 4 is performed separately for the narrow band line filter elements 26a and 26b, so that different transmission spectrum characteristics can be obtained. The difference image data S can be generated by obtaining the real images of the same subject 11 as image data A and B.

Further, in the above-described embodiment, the case where the present invention is applied to a fire monitoring device and an automatic fruit sorting device is taken as an example. However, the present invention is not limited to this, and is applied to medicine, biology, remote sensing, food management and the like. The present invention can be applied as it is as a sensor for differential spectrum imaging performed in various fields, and can generate an accurate differential spectrum image particularly when the subject changes with time or moves.

[0083]

As described above, according to the present invention, a plurality of images having completely different spectral characteristics can be formed on one image sensor at the same time, whereby a differential image is generated from the difference between the images. At this time, there is no time lag in the original image, and a correct differential spectrum image can always be obtained even if the subject moves at high speed or the illumination changes. At the same time, even if the operation speed of the image processing system is low, a correct differential spectrum image can always be generated if image data is stored in the image memory from the image sensor regardless of the time delay of the processing system.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a first embodiment of the surveillance camera in FIG. 1 using two lens systems.

FIG. 3 is a structural explanatory view of a wavelength variable interference filter used in the present invention.

FIG. 4 is an explanatory diagram of a wavelength spectrum of a fire flame and a filter transmission wavelength band.

FIG. 5 is a functional block diagram of the signal processing device of FIG. 1;

FIG. 6 is an explanatory diagram showing two images having different wavelength bands obtained by the signal processing device of FIG. 1 and a difference image together with a visible image;

FIG. 7 is a flowchart of a processing operation of the signal processing device of FIG. 1;

FIG. 8 is a flowchart of a peach sugar content measurement process performed by the image sensor according to the present invention.

FIG. 9 is a flowchart of peach visible light inspection processing by the image sensor of the present invention.

FIG. 10 is an explanatory diagram of a second embodiment of the monitoring camera in FIG. 1 using a beam splitter.

11 is an explanatory diagram of the beam splitter in FIG.

FIG. 12 is an explanatory diagram of a third embodiment of the surveillance camera in FIG. 1 in which micro optical filters are arranged in a zigzag on an image sensor.

FIG. 13 is an explanatory diagram of the micro optical filter of FIG.

FIG. 14 is an explanatory diagram of a fourth embodiment of the monitoring camera in FIG. 1 in which an optical line filter is arranged on an image sensor.

FIG. 15 is an explanatory diagram of the optical line filter of FIG. 14;

[Explanation of symbols]

 1: Surveillance camera 2: Optical system 2a to 2c, 14: Imaging lens 3: Optical filter 3a: First filter 3b: Second filter 4: Image sensor (CCD) 5: Signal processing device 6: MPU 7: AD conversion 8: Image memory 9: Output interface (output IF) 10: Monitor 11: Object to be imaged 12a: Difference image generation unit 12b: Fire determination processing unit 13: Wavelength variable interference filter 14, 16: Glass substrate 15, 17: Metal Film 18: Piezoelectric element 19: Driving voltage 20: Beam splitter 21: Incident light 22, 23: Deflected light 24: Micro optical filter unit 24a to 24c: Micro narrow band filter element 26: Optical line filter 26a, 26b: Narrow band Line filter element 30: Glass substrate 31, 32: Reflective layer 33: Void (interference distance)

Claims (6)

[Claims] 1. An image sensor, an optical system for forming a plurality of real images on the image sensor, and different optical systems arranged on an image system for real images divided by the optical system An optical filter having a transmission spectrum characteristic, and a difference image generation unit that generates a differential spectrum image of a subject by taking a difference value between pixels at opposite positions from a plurality of spectrum images simultaneously captured by the image sensor. And a spectral image sensor comprising: 2. The spectral image sensor according to claim 1, wherein said optical system is constituted by a lens system having a plurality of optical axes for forming a plurality of divided real images on said image pickup device. A spectral image sensor, comprising: 3. The spectral image sensor according to claim 1, wherein the optical system is provided in front of the image sensor, and a beam splitter that forms a plurality of divided real images on the image sensor is provided. A spectral image sensor. 4. The spectral image sensor according to claim 1, wherein the optical system and the optical filter are arranged by dispersing minute optical filters having different spectral characteristics corresponding to a plurality of minute regions on the image sensor. A spectrum image sensor for causing the image sensor to capture images of the same object having different spectral characteristics. 5. The spectral image sensor according to claim 4, wherein said optical filters include micro optical filters having different spectral characteristics arranged in a staggered or linear manner. 6. A spectral image sensor according to claim 1, wherein a CCD is used as said image pickup device.