JP2020193859A - Image processing device, image capturing system, image processing method and program - Google Patents

Abstract

To provide an image processing device which is inexpensively configured but offers high measurement accuracy.SOLUTION: An image processing device (120) is provided, comprising an image acquisition unit (122) configured to acquire a spectroscopic image (e.g., RGB image) from an image sensor (113) adapted to receive light passing through a filter (112) with three transmission wavelength bands, and a color separation processing unit (121) for reducing color mixing in the spectroscopic image.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing device that performs image measurement.

In recent years, a method for measuring a leaf color index using an image has been proposed for the purpose of improving the efficiency and labor saving of agricultural work. For example, Patent Document 1 discloses a plant growth index measuring device that measures a SPAD (Soil & Plant Analyzer Development) value using an image.

International Publication No. 2016/181734

However, the plant growth index measuring device disclosed in Patent Document 1 uses a wavelength near 650 nm as the first wavelength and an infrared light wavelength of 750 nm or more as the second wavelength, and thus is an expensive infrared imaging device. Needs.

On the other hand, as an inexpensive imaging device, an RGB imaging device that generates an RGB image is known. However, when image measurement is performed using an RGB imaging device, the measurement accuracy is low because the measured values vary widely with respect to changes in ambient light.

Therefore, an object of the present invention is to provide an image processing apparatus, an imaging system, an image processing method, and a program having an inexpensive configuration and high measurement accuracy.

The image processing apparatus as one aspect of the present invention has an image acquisition unit that acquires three spectroscopic images from an image sensor that receives light that has passed through a filter having three transmission wavelength bands, and reduces color mixing from the spectroscopic images. It has a color separation processing unit.

An imaging system as another aspect of the present invention includes an imaging optical system, a filter having three transmission wavelength bands, an imaging optical system, an imaging element that receives light that has passed through the filters, and the image processing apparatus. Has.

The image processing method as another aspect of the present invention includes a step of acquiring three spectroscopic images from an image sensor that receives light that has passed through a filter having three transmission wavelength bands, and a step of reducing color mixing from the spectroscopic images. And have.

A program as another aspect of the present invention causes a computer to execute the image processing method.

Other objects and features of the present invention will be described in the following examples.

According to the present invention, it is possible to provide an image processing apparatus, an imaging system, an image processing method, and a program having an inexpensive configuration and high measurement accuracy.

It is a block diagram of the image processing system in Example 1. It is a figure which shows the spectral characteristic of an image pickup optical system, a triple bandpass filter, and an image pickup element in Example 1. FIG. It is a figure which shows the characteristic which multiplied the spectral characteristic of the image pickup optical system, the triple bandpass filter, and the image pickup element in Example 1. FIG. It is explanatory drawing of the camera imaging model in Example 1. FIG. It is a figure which shows the spectral characteristic of the ambient light and the subject by the simulation in Example 1. FIG. It is a flowchart which shows the calculation by the simulation in Example 1. FIG. It is a figure which shows the simulation result of the comparative example and Example 1. It is a figure which shows the simulation result of the half-value width dependence of the image measurement accuracy in Example 1. FIG. It is a figure which shows the simulation result of the center wavelength dependence of the image measurement accuracy in Example 1. FIG. It is a block diagram of the imaging unit in Example 2. It is a figure which shows the spectral characteristic of an image pickup optical system, a triple bandpass filter, and an image pickup element in Example 3. FIG. It is a block diagram of the imaging unit in Example 4.

Hereinafter, examples of the present invention will be described in detail with reference to the drawings. In each figure, the same members are designated by the same reference numerals, and duplicate description will be omitted.

First, the configuration of the image processing system (imaging system) 100 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of the image processing system 100. The image processing system 100 includes an image pickup unit 110, an image processing unit (image processing device) 120, a control unit 130, a storage unit 140, a communication unit 150, and a display unit 160. The image pickup unit 110 includes an image pickup optical system 111, a triple bandpass filter (filter) 112, and an image pickup element 113. The image sensor 113 photoelectrically converts an optical image (subject image) formed via the image pickup optical system 111, and outputs an image (image data) to the image processing unit 120. In this embodiment, the image pickup device 113 is an RGB image pickup device, and the color image acquired from the image pickup device 113 has a three-band (R, G, B) image, but is not limited thereto. Of the image processing system 100, at least the imaging unit 110 is provided in the camera 200 of FIG.

The image processing unit 120 includes a color separation processing unit 121, an image acquisition unit 122, and a selection unit 123. The color separation processing unit 121 acquires a spectroscopic image (for example, a color image such as an RGB image) from the image pickup unit 110 (image sensor 113). The color separation processing unit 121 reduces color mixing from the spectroscopic image. Details of each of these functions will be described later. The image taken through the triple bandpass filter 112 is not limited to an RGB image based on visible light, and widely includes a spectroscopic image such as an image based on near infrared light.

The image processing system 100 can be provided inside the image pickup apparatus (camera). Alternatively, some functions such as the image processing unit 120 of the image processing system 100 may be configured to be realized on a computer (user PC) or cloud computing away from the image pickup apparatus. In this case, the image pickup apparatus has the function of only a part of the image processing system 100 including the image pickup unit 110.

Next, with reference to FIG. 2, the spectral characteristics of the image pickup optical system 111, the triple bandpass filter 112, and the image pickup element 113 will be described. Figure 2 (a) ~ (c) the spectral transmittance of each imaging optical system 111 L (lambda), the spectral transmittance of the triple band-pass filter 112 F (lambda), and spectral sensitivity characteristics _{S i} of the imaging device 113 It is a figure which shows (λ) (i = {1,2,3}). In FIGS. 2A and 2B, the horizontal axis represents the wavelength (nm) and the vertical axis represents the transmittance (%). In FIG. 2C, the horizontal axis represents the wavelength (nm) and the vertical axis represents the normalization sensitivity.

As shown in FIG. 2C, the image sensor 113 of this embodiment is a so-called RGB color image sensor having three sensitivity characteristics of RGB in the visible wavelength range of about 400 nm to 700 nm. Generally, an RGB color image sensor is configured to include an image sensor, an RGB color filter, and an infrared / ultraviolet cut filter, and FIG. 2C shows a characteristic in which these are multiplied. In FIG. 2 (c), S ₁ (λ) shown by the solid line is the spectral sensitivity characteristic of the B image, S ₂ (λ) shown by the dotted line is the spectral sensitivity characteristic of the G image, and S ₃ (λ) shown by the alternate long and short dash line. ) Represent the spectral sensitivity characteristics of the R image.

As shown in FIG. 2B, the triple bandpass filter 112 of this embodiment has three transmission wavelength bands in the wavelength band in which the image sensor 113 has sensitivity. In the example of FIG. 2B, the central wavelength of the transmission wavelength band is λ _c1 , λ _c2 , λ _c3 from the short wavelength side, and the half width of the transmission wavelength band is FWHM ₁ = FWHM ₂ = FWHM ₃ from the short wavelength side. .. At this time, the triple bandpass filter 112 has a transmission wavelength band of λ _c1 = 490 nm, λ _c2 = 550 nm, λ _c3 = 610 nm, and FWHM ₁ = FWHM ₂ = FWHM ₃ = 25 nm. The central wavelength of the transmission wavelength band of this embodiment is defined as either a wavelength located at the center of the transmission wavelength band or a wavelength calculated by a weighted average weighted by the spectral transmittance of the transmission wavelength band. Refers to, but is not limited to, wavelength.

The triple bandpass filter 112 of this embodiment is manufactured by laminating optical thin films having different film thicknesses and materials on a transparent substrate. As shown in FIG. 2B, it is preferable to design and manufacture a triple bandpass filter so that the transmittance in the wavelength band other than the transmission wavelength band is 1% or less. It is preferable that the transmittance of the wavelength band other than the transmission wavelength band is 1% or less because the measurement variation at the time of image measurement can be reduced. Further, in order to prevent a decrease in the amount of light incident on the image sensor 113, the transmittance in the transmission wavelength band is preferably close to 100%, but even if the transmittance in the transmission wavelength band is 80% or 90%, this embodiment. The effect of can be obtained. In the triple bandpass filter 112 of this embodiment, FWHM ₁ = FWHM ₂ = FWHM ₃ is set, but the effect of this embodiment can be obtained even if the full width at half maximum of each transmission wavelength band is not the same. .. Therefore, in order to adjust the difference in exposure amount due to the difference in the spectral sensitivity characteristics S ₁ (λ), S ₂ (λ), and S ₃ (λ) of the image sensor 113, FWHM ₁ , FWHM ₂ , and FWHM ₃ are individually adjusted. You may. Further, for the same purpose, the transmittance of each transmission wavelength band may be adjusted individually.

FIG. 3 is a diagram showing characteristics obtained by multiplying the spectral characteristics of the imaging optical system 111, the triple bandpass filter 112, and the imaging element 113 shown in FIG. In FIGS. 3A to 3C, the horizontal axis represents the wavelength (nm) and the vertical axis represents the normalized intensity. Figure 3 (a) ~ (c), respectively, spectral transmittance L (lambda), the spectral transmittance F (lambda), multiplies characteristics of the spectral sensitivity characteristic _{S i} (λ) (i.e., L (λ) F (λ ) S ₁ (λ), L (λ) F (λ) S ₂ (λ), L (λ) F (λ) S ₃ (λ)) are shown. The image processing unit 120 holds in advance _mij calculated by the following equation (8) using the characteristics of FIG.

Next, a case where a transmitted light image of a subject is imaged under outdoor ambient light by using an image pickup apparatus (imaging unit 110) will be described with reference to FIG. FIG. 4 is an explanatory diagram of the camera shooting model in this embodiment, and shows a case where the image pickup unit 110 is used to capture a transmitted light image of a subject in an outdoor environment. In the model of FIG. 4, assuming that the spectral irradiance of the ambient light is E (λ) and the spectral transmittance of the subject is T (λ), the pixel value I _{i of the} image acquired by the imaging unit 110 is expressed by the following equation (1). It can be represented by 1).

In the formula _{(1), λ Si1, λ} Si2 , respectively, is the shortest wavelength and the longest wavelength in a wavelength band image pickup element 113 has sensitivity of the spectral sensitivity characteristics _{S i (λ)} as shown in Figure 2 (c) ..

From the transmitted light image of the object taken in outdoor environment light for measuring the spectral transmittance T _i of the object, for example, performs a calculation according to equation (2) below.

In the formula (2), I _{0 and i} are pixel values when the ambient light incident on the subject is acquired by the imaging unit 110, and I _{0 and i} can be expressed by the following formula (3).

As a comparative example, the formula for calculating the transmittance when there is no triple bandpass filter 112 is expressed as the following formula (4).

It is known that when the half width of S _i (λ) is narrow and the central wavelengths of S _i (λ) are λ _{C and Si} , the following equation (5) holds for the equation (4). There is.

Therefore, when the full width at half maximum of S _i (λ) is narrow, the transmittance of the subject can be calculated by the equation (5). However, as shown in FIG. 2C, the spectral sensitivity characteristic S _i (λ) of a normal RGB color image sensor has a wide half-value width, and the approximation of the equation (5) does not hold, so that the subject is transmitted from the image. It is difficult to measure the rate with high accuracy. In particular, the spectral irradiance E of the ambient light (lambda) is variation is large spectral transmittance T _i at the time of change, it is difficult to accurately measure the spectral transmittance T _i.

Therefore, in this embodiment, a triple bandpass filter 112 is used in order to narrow the half width of the spectral sensitivity characteristic _Si (λ). However, if the triple bandpass filter 112 is simply used, as shown in FIG. 3, light in an unnecessary wavelength band is also mixed (color mixing). Here, the light of unnecessary wavelength band, Figure 3 wavelength λ _c1 ± FWHM _1/2 of the wavelength bands other than the light in (a), the wavelength band of 3 wavelengths in _{(b) λ c2 ± FWHM 2} /2 refers to non-light, light other than the wavelength band of 3 wavelengths in _{(c) λ c3 ± FWHM 3} /2. The generation of such unnecessary light (color mixing) is due to the fact that the spectral sensitivity characteristic _Si (λ) of the image sensor 113 has sensitivity in a wide wavelength band.

Therefore, in this embodiment, there is further a color separation processing unit 121 that performs a separation processing for separating light in these unnecessary wavelength bands. Next, the details of the separation process by the color separation process unit 121 will be described.

First, assuming that the pixel value acquired only by the light in the wavelength band of the wavelength λ _ci ± FWHM _i / 2 is I _Si , I _Si can be expressed by the following equation (6).

Next, in this embodiment, I _i is expressed by the following equation (7) using I _Si .

In the formula (7), _mij is a weighting coefficient defined by the following formula (8).

In the formula (8), λ _Fj1 and λ _Fj2 are the rising wavelength and the falling wavelength of the transmission wavelength band of the triple bandpass filter 112 shown in FIG. _2B , respectively. _mij is a value pre-calculated based on the equation (8) using known L (λ), F (λ), and S _i (λ), and the value is held in the color separation processing unit 121. ing. In this embodiment, the color separation processing unit 121 executes the color separation processing by the calculation represented by the following equation (9).

Therefore, according to this embodiment, it is possible to obtain an image equivalent to an image taken with a filter having a narrow half-value width with less color mixing from an image taken with an RGB color image sensor. Note that in order to measure the spectral transmittance T _i of the object from the transmitted light image of the subject captured by the imaging unit 110 of this embodiment, it is preferable to perform the calculation according to formula (10) below.

In the formula (10), I _{0 and Si} are pixel values when the color separation processing is performed on the I _{0 and i} , and are calculated by the following formula (11).

The value on the right side of the equation (10) can be further approximated as in the following equation (12).

Therefore, the image pickup unit 110 of this embodiment can measure the transmittance of the subject with high accuracy.

Next, the effect of this embodiment will be described using simulation results. Here, a case where the chlorophyll content of the leaf of the plant is measured from the pixel value of the transmitted light image of the leaf of the plant captured by the imaging unit 110 will be described as an example. The camera imaging model of FIG. 4 is used for the simulation. FIG. 4 shows a model in which the image pickup unit 110 captures a transmitted light image of a plant leaf 401 under outdoor ambient light.

FIG. 5 is a diagram showing the spectral characteristics of the ambient light and the subject by the simulation, and shows the spectral irradiance E (λ) of the ambient light used in the simulation and the spectral transmittance T (λ) of the subject. FIG. 5A is an actually measured value of the spectral irradiance measured under various weather conditions (sunny weather, cloudy weather, etc.). FIG. 5B is an actually measured value of the spectral transmittance of leaves of plants having different nitrogen contents, and the color of each line corresponds to the value of the SPAD value shown on the color bar. Here, the SPAD value is an index corresponding to the chlorophyll content, and the difference in the SPAD value represents the difference in the chlorophyll content. Hereinafter, in the simulation, the accuracy of measuring the SPAD value from the pixel value of the transmitted light image of the leaf of the plant will be described.

Next, the calculation by simulation will be described with reference to FIG. FIG. 6 is a flowchart showing the calculation by simulation. In this simulation, the calculation of FIG. 6 is performed for all combinations of the spectral irradiance E (λ) and the spectral transmittance T (λ).

First, in step S001, the pixel value I _i is calculated based on the equation (1) (actually, the pixel value I _i is acquired by photographing the subject with the imaging unit 110). Subsequently, in step S002, the image processing unit 120 calculates the pixel value _Isi based on the equation (9). Further, steps S003 and S004 are executed in parallel with steps S001 and S002. First, in step S003, the pixel values I _{0 and i} are calculated based on the equation (3) (actually, the pixel values I _i are acquired by photographing the ambient light with the imaging unit 110). After that, in step S004, the image processing unit 120 calculates the pixel values I _{0, si} based on the equation (11).

Subsequently, in step S005, the image processing unit 120, the calculated pixel value _{I si,} using _{I 0, si,} it calculates the spectral transmittance _{T i} of the object on the basis of the equation (10). Finally, in step S006, the image processing unit 120 calculates NGRDI (Normalized Green Red Difference Index) as an index having a correlation with the SPAD value. Here, NGRDI is a value calculated by the following formula (13).

Next, the simulation results will be described with reference to FIG. 7. FIG. 7 is a diagram showing a simulation result. In FIGS. 7 (a) to 7 (c), the horizontal axis represents the NGRDI and the vertical axis represents the SPAD value. Each point is the value of NGRDI calculated for the combination of the spectral irradiance E (λ) and the spectral transmittance T (λ), and the SPAD value of the spectral transmittance T (λ) used when calculating the NGRDI. The relationship is shown.

FIG. 7A shows a comparative example when the triple bandpass filter 112 is not used and the color separation process is not performed. FIG. 7B shows a comparative example in which the triple bandpass filter 112 is used and the color separation process is not performed. FIG. 7C shows this embodiment in which the triple bandpass filter 112 is used and the color separation process is performed. Here, not using the triple bandpass filter 112 means executing steps S001 and S003 in FIG. 6 with F (λ) = 100 [%]. Also, not to implement the color separation processing, without executing steps S002, _S004, refers to performing step S005 as _{I si = I i, I 0} , si = I 0, i.

Further, RMSE and ^{R 2} are described in the upper left of FIG. 7 (a) ~ (c) has an average in the case of performing linear regression by the equation (14) below explanatory variables NGRDI, the SPAD value as the dependent variable a quadratic square error RMSE and the coefficient of determination ^{R 2.}

In the formula (14), _{d i} is a constant calculated by linear regression.

As shown in FIG. 7, when the image processing system 100 of this embodiment is used, the SPAD value can be measured from NGRDI with high accuracy. Therefore, the image processing system 100 of this embodiment makes it possible to perform image measurement with high accuracy.

Next, with reference to FIG. 8, the simulation result of the half-value width dependence of the image measurement accuracy will be described. FIG. 8 is a diagram showing a simulation result of the half-value width dependence of the image measurement accuracy. In FIG. 8, the horizontal axis represents the full width at half maximum FWHM (nm), and the vertical axis represents RMSE. FIG. 8 shows the simulation results when the full width at half maximum was changed in the range of 5 nm to 55 nm with λ _c1 = 490 nm, λ _c2 = 550 nm, and λ _c3 = 610 nm fixed and FWHM ₁ = FWHM ₂ = FWHM _3. .. Round point in FIG. 8, mean square square error RMSE, square points represents the coefficient of determination R ^2.

As shown in FIG. 8, mean square square error RMSE as the half width is narrow decreases, the coefficient of determination R ² increases. Therefore, from the viewpoint of measurement accuracy, it is preferable that the half width is narrow. On the other hand, when the full width at half maximum is narrowed, the amount of light incident on the image sensor decreases, which causes problems such as time required for imaging and a decrease in the signal-to-noise ratio. Therefore, it is preferable that the full width at half maximum FWHM _i of the transmission wavelength band of the triple bandpass filter 112 of this embodiment satisfies the following conditional expression (15).

More preferably, the following conditional expression (16) is satisfied.

Next, with reference to FIG. 9, the simulation result of the center wavelength dependence of the image measurement accuracy will be described. FIG. 9 is a diagram showing a simulation result of the center wavelength dependence of the image measurement accuracy. In FIGS. 9A to 9D, the horizontal axis represents the wavelength λ _c2 (nm) and the vertical axis represents the wavelength λ _c3 (nm).

FIG. 9 shows the simulation results when λ _c1 , λ _c2 , and λ _c3 are changed with FWHM ₁ = FWHM ₂ = FWHM ₃ = 25 nm. 9 (a) and 9 (b) are simulation results when λ _c1 = 490 nm, 530 nm ≤ λ _c2 ≤ 590 nm, 560 nm ≤ λ _c3 ≤ 650 nm. FIG. 9A is a contour diagram showing the distribution of the mean square error RMSE. FIG. 9 (b) represents the distribution of the coefficient of determination R ² in contour plot. In both figures, the shaded area indicates a region where the second transmission wavelength band and the third transmission wavelength band counting from the short wavelength side overlap and do not form the triple bandpass filter 112. 9 (c) and 9 (d) are simulation results when λ _c1 = 430 nm, 530 nm ≤ λ _c2 ≤ 590 nm, 560 nm ≤ λ _c3 ≤ 650 nm. FIG. 9 (c) is a contour diagram showing the distribution of the mean square error RMSE. FIG. 9 (d) represents the distribution of the coefficient of determination R ² in contour plot.

From the simulation results of FIG. 9, in order to measure the SPAD value with high accuracy, it is preferable that the ranges of λ _c2 and λ _c3 satisfy the following conditional expressions (17) and (18) regardless of λ _c1 .

More preferably, the following conditional expressions (19) and (20) are satisfied.

In the image processing system 100 of this embodiment, the image pickup unit 110 takes an image of the subject and acquires the captured image. Subsequently, the image processing unit 120 (color separation processing unit 121), using the pixel values I _i of each pixel of the captured image, performs color separation process of the formula (9). The color-separated image composed of the pixel value _ISi calculated by the color-separation process is appropriately stored in the storage unit 140, transferred to the external device (not shown) by the communication unit 150, or transferred to the display unit 160. Is displayed. Each of the above operations is executed by a command from the control unit 130.

Next, the image processing system according to the second embodiment of the present invention will be described with reference to FIG. FIG. 10 is a configuration diagram of the image processing system (imaging unit 110a) in this embodiment.

The image processing system of this embodiment has the same configuration as the image processing system 100 of Example 1 described with reference to FIG. In this embodiment, the image processing system has an image measurement mode (measurement mode) and an RGB image shooting mode (shooting mode). The selection unit 123 of the image processing unit 120 selects an image measurement mode or an RGB image capturing mode. FIG. 10A shows the arrangement of the image pickup unit 110a of the image processing system in the image measurement mode. When the image measurement mode is selected, the triple bandpass filter 112 is inserted between the image pickup optical system 111 and the image pickup element 113 by using a drive mechanism (not shown). FIG. 10B shows the arrangement of the imaging unit 110a in the RGB image capturing mode. When the RGB image capturing mode is selected, the triple bandpass filter 112 is retracted from between the image pickup optical system 111 and the image pickup element 113 by using a drive mechanism (not shown).

According to this embodiment, one imaging unit 110a can capture both a normal RGB image and an image suitable for image measurement.

Next, with reference to FIG. 11, the spectral characteristics of the imaging optical system 111, the triple bandpass filter 112, and the imaging element 113 according to the third embodiment of the present invention will be described. Figure 11 (a) ~ (c) the spectral transmittance of each of the imaging optical system 111 L (lambda), the spectral transmittance of the triple band-pass filter 112 F (lambda), and spectral sensitivity characteristics _{S i} of the imaging device 113 It is a figure which shows (λ) (i = {1,2,3}). In FIGS. 11A and 11B, the horizontal axis represents the wavelength (nm) and the vertical axis represents the transmittance (%). In FIG. 11C, the horizontal axis represents the wavelength (nm) and the vertical axis represents the normalization sensitivity.

The image processing system of this embodiment has the same configuration as the image processing system 100 of Example 1 described with reference to FIG. However, the configuration of the image sensor 113 of this embodiment is different from that of the image sensor 113 of Example 1. The image sensor 113 of the first embodiment is composed of an image sensor, an RGB color filter, and an infrared / ultraviolet cut filter, but the image sensor of the present embodiment is composed of only an image sensor and an RGB color filter. There is.

Silicon-based image sensors used to acquire RGB images (color images) typically have sensitivity in the wavelength range of 300 nm to 1100 nm. Therefore, although it depends on the characteristics of the RGB color filter, by eliminating the infrared / ultraviolet cut filter, light having a wavelength other than visible light can be used as shown in FIG. 11 (c). As shown in FIG. 11B, the triple bandpass filter 112 of this embodiment has a transmission wavelength of λ _c1 = 440 nm, λ _c2 = 560 nm, λ _c3 = 785 nm, and FWHM ₁ = FWHM ₂ = FWHM ₃ = 30 nm. Has a band.

In this embodiment, near-infrared light is used as light having a wavelength other than visible light. By using near-infrared light, the diagnostic accuracy of the SPAD value can be improved. The SPAD value is measured using visible light having a wavelength of around 650 nm and near-infrared light having a wavelength of around 940 nm. Therefore, it is preferable to use near-infrared light as well when performing image measurement. As shown in FIG. 5 (b), the transmittance of plant leaves hardly changes in the range of 750 nm to 1300 nm, so that any wavelength of light can be used as near-infrared light within this wavelength range. Good. Moreover, it is preferable to use light of 550 nm to 700 nm as visible light. Further, as the third transmission wavelength of the triple bandpass filter 112, it is preferable to select a wavelength such that the 3 × 3 determinant on the right side of the equation (7) becomes regular. The color separation process of this example is carried out in the same manner as in Example 1.

Next, the image processing system according to the fourth embodiment of the present invention will be described with reference to FIG. FIG. 12 is a configuration diagram of the image processing system (imaging unit 110b) in this embodiment. The image pickup unit 110b of this embodiment includes an image pickup optical system 111, a triple bandpass filter 112, an image pickup element 113, and an infrared / ultraviolet cut filter 114. The image sensor 113 is composed of only an image sensor and an RGB color filter.

The image processing system of this embodiment has an image measurement mode (measurement mode) and an RGB image shooting mode (shooting mode). The selection unit 123 of the image processing unit 120 selects an image measurement mode or an RGB image capturing mode. FIG. 12A shows the arrangement of the imaging unit 110b in the image measurement mode. When the image measurement mode is selected, the triple bandpass filter 112 is inserted between the image pickup optical system 111 and the image pickup element 113 by using a drive mechanism (not shown). FIG. 12B shows the arrangement of the imaging unit 110b in the RGB image capturing mode. When the RGB image capturing mode is selected, an infrared / ultraviolet cut filter 114 is inserted between the image pickup optical system 111 and the image pickup element 113 by using a drive mechanism (not shown).

According to this embodiment, one imaging unit 110b can capture both a normal RGB image and an image suitable for image measurement.

As described above, in each embodiment, the image processing apparatus (image processing unit 120) has an image acquisition unit 122 and a color separation processing unit 121. The image acquisition unit 122 acquires three spectral images (for example, a color image) from an image sensor (for example, an RGB image sensor) 113 that receives light that has passed through a filter (triple bandpass filter 112) having three transmission wavelength bands. .. The color separation processing unit 121 reduces color mixing from the spectroscopic image.

Preferably, the color separation processing unit reduces the color mixing based on the spectral transmittance of the image pickup optical system 111, the spectral transmittance of the filter, and the spectral sensitivity characteristics of the image pickup element. Further, preferably, the half width of the three transmission wavelength bands is 45 nm or less. Further, preferably, the central wavelengths of two transmission wavelength bands out of the three transmission wavelength bands are within 560 nm ± 20 nm and within 610 nm ± 30 nm, respectively.

Preferably, the image processing apparatus has a selection unit 123 for selecting a measurement mode (image measurement mode) or a shooting mode (RGB image shooting mode), and an image acquisition unit has a filter when the measurement mode is selected by the selection unit. Acquires a spectroscopic image captured with the Further, the image acquisition unit acquires a spectroscopic image captured with the filter retracted when the shooting mode is selected by the selection unit. Further, preferably, when the photographing mode is selected by the selection unit, the image acquisition unit acquires a spectroscopic image captured with the infrared / ultraviolet cut filter 114 inserted. Further, when the measurement mode is selected by the selection unit, the image acquisition unit acquires a spectroscopic image captured with the infrared / ultraviolet cut filter retracted.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

According to each embodiment, it is possible to provide an image processing apparatus, an imaging system, an image processing method, and a program having high measurement accuracy in an inexpensive configuration.

Although preferable examples of the present invention have been described above, the present invention is not limited to these examples, and various modifications and modifications can be made within the scope of the gist thereof.

For example, the transmittance measurement is given as an example in the explanation of the effect of each embodiment, but the present invention is not limited to this, and the effect of each embodiment is also obtained in the reflectance measurement and the white balance correction of the image. be able to. Further, each embodiment is not limited to the measurement of the chlorophyll content of the leaves of plants, and can be applied to other measurements.

120 Image processing unit (image processing device)
121 Color separation processing unit 122 Image acquisition unit

Claims (9)

An image acquisition unit that acquires three spectral images from an image sensor that receives light that has passed through a filter having three transmission wavelength bands.
An image processing apparatus comprising: a color separation processing unit for reducing color mixing from the spectroscopic image. The first aspect of claim 1, wherein the color separation processing unit reduces the color mixing based on the spectral transmittance of the image pickup optical system, the spectral transmittance of the filter, and the spectral sensitivity characteristics of the image pickup element. Image processing device. The image processing apparatus according to claim 1 or 2, wherein the half width of the three transmission wavelength bands is 45 nm or less. The invention according to any one of claims 1 to 3, wherein the central wavelengths of the two transmission wavelength bands out of the three transmission wavelength bands are within 560 nm ± 20 nm and within 610 nm ± 30 nm, respectively. Image processing device. Further having a selection unit for selecting a measurement mode or a shooting mode,
The image acquisition unit
When the measurement mode is selected by the selection unit, the spectroscopic image captured with the filter inserted is acquired.
The image processing according to any one of claims 1 to 4, wherein when the photographing mode is selected by the selection unit, the spectroscopic image captured with the filter retracted is acquired. apparatus. The image acquisition unit
When the imaging mode is selected by the selection unit, the spectroscopic image captured with the infrared / ultraviolet cut filter inserted is acquired.
The image processing apparatus according to claim 5, wherein when the measurement mode is selected by the selection unit, the spectroscopic image captured with the infrared / ultraviolet cut filter retracted is acquired. Imaging optics and
A filter with three transmission wavelength bands and
An image sensor that receives light that has passed through the imaging optical system and the filter, and
An image pickup system comprising the image processing apparatus according to any one of claims 1 to 6. A step of acquiring three spectroscopic images from an image sensor that receives light that has passed through a filter having three transmission wavelength bands.
An image processing method comprising: a step of reducing color mixing from the spectroscopic image. A program comprising causing a computer to execute the image processing method according to claim 8.