JP2023163061A - Imaging apparatus, control method for imaging apparatus, and program - Google Patents

Abstract

To provide an imaging apparatus capable of obtaining appropriate exposure for the rays of light in a plurality of wavelength ranges with low power consumption.SOLUTION: An imaging apparatus (10) has an imaging unit (11) that images a subject and a control unit (12) that controls exposure during imaging. The imaging unit is capable of first imaging using light in a first wavelength range and second imaging using light in a second wavelength range different from the first wavelength range. The control unit determines a second exposure for the second imaging using a first exposure in the first imaging and reflectance of the subject.SELECTED DRAWING: Figure 4

Description

The present invention relates to an imaging device, a method of controlling the imaging device, and a program.

BACKGROUND ART Conventionally, systems are known that diagnose growth indicators from images of plants and ascertain changes in growth conditions. For example, a growth index such as a Normalized Difference Vegetation Index (NDVI) or a leaf area index is calculated using the brightness values of near-infrared light and red light images.

Patent Document 1 discloses that an image of a predetermined area is photographed by changing the exposure time for each of near-infrared light and red light to multiple types, and images with appropriate exposure times for each of near-infrared light and red light are captured. A method for selecting is disclosed.

Japanese Patent Application Publication No. 2007-171033

In the method disclosed in Patent Document 1, since it is necessary to take pictures with a plurality of exposure times for each of light in a plurality of wavelength ranges, the number of shots is increased. As a result, power consumption increases, and it is necessary to increase the size of power supplies such as solar cells and batteries.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an imaging device that can obtain appropriate exposure for each of light in a plurality of wavelength ranges with low power consumption.

An imaging device according to one aspect of the present invention includes an imaging unit that images a subject, and a control unit that controls exposure during imaging, and the imaging unit includes a first light source using light in a first wavelength range. imaging and a second imaging using light in a second wavelength range different from the first wavelength range, and the control unit controls the first exposure and the reflectance of the subject during the first imaging. is used to determine the second exposure during the second imaging.

Other objects and features of the invention are explained in the following embodiments.

According to the present invention, it is possible to provide an imaging device that can obtain appropriate exposure for each of light in a plurality of wavelength ranges with low power consumption.

It is an example of the spectral reflectance of the leaf of the plant in each embodiment. 1 is a block diagram showing the hardware configuration of an imaging device in a first embodiment. FIG. FIG. 2 is a block diagram showing the software configuration of the imaging device in the first embodiment. It is a flowchart which shows the control method in 1st Embodiment. It is a flowchart which shows the control method in 2nd Embodiment. It is a flowchart which shows the control method in 3rd Embodiment.

Embodiments of the present invention will be described in detail below with reference to the drawings. In each figure, the same reference numerals are given to the same members, and overlapping explanations will be omitted.

First, before giving a specific description of each embodiment, an overview of the present invention will be described. FIG. 1 is an example of the spectral reflectance of leaves of a plant to be photographed in the visible light wavelength range (first wavelength range) and the near-infrared light wavelength range (second wavelength range) in this embodiment. In FIG. 1, the horizontal axis represents wavelength (nm), and the vertical axis represents reflectance (%). In the leaves of the plant of this embodiment, the pigments in the leaves absorb visible light and have a low reflectance, but the near-infrared light has little absorption and a relatively high reflectance. In this case, for example, if near-infrared light is photographed with an exposure (exposure time) determined in accordance with visible light, overexposure occurs and an appropriate image cannot be obtained. On the other hand, if an image is taken using visible light in accordance with the exposure determined using near-infrared light, which has a high reflectance, the image will be underexposed and an appropriate image will not be obtained. When measuring the growth status of plants based on color information, color accuracy is important, so it is necessary to obtain images with appropriate exposure. Therefore, it is necessary to control exposure in accordance with the wavelength range of light and taking into account the reflectance of the subject. If the subject to be photographed is determined and the spectral reflectance of the subject is known, such as when placing an imaging device at a fixed point and continuously photographing plants to observe their growth status, the reflectance of the subject should be taken into account. It is possible to control the exposure by

The reflectance characteristics differ depending on the subject, and it is preferable to change the exposure depending on the subject to be photographed (leaves, flowers, fruits, panicles, etc.), for example. In addition, when placing an imaging device at a fixed point and continuously photographing plants, depending on the growth stage and growth condition, young leaves, mature leaves, leaves and panicles, or healthy leaves and withered leaves may be displayed on the screen. There are different subjects such as For example, the solid line in FIG. 1 indicates the spectral reflectance of young leaves, and the broken line indicates the spectral reflectance of grown leaves. In this way, the reflectance varies depending on the growth stage or growth condition. In this case as well, it is preferable to detect the growth stage and growth condition of the subject from the image and determine the exposure based on the correction value according to the target subject. Furthermore, in order to obtain appropriate exposure, it is preferable to determine the exposure by taking into account the spectral sensitivity of the imaging device, or, if white balance (WB) processing is not performed, the spectral characteristics of the light source. Each embodiment will be described in detail below.

(First embodiment)
First, a first embodiment will be described. FIG. 2 is a block diagram showing an example of the hardware configuration of the imaging device 10 in this embodiment. The imaging device 10 includes an imaging section 11 , a CPU (control section) 12 , a memory (storage section) 13 , an input section 14 , a display section 15 , and a communication section 16 . The imaging unit 11 images a subject. The CPU 12 controls each section in the imaging device 10. The memory 13 stores data such as programs, images captured by the imaging unit 11, and setting values. Note that the memory 13 includes a storage medium such as ROM or nonvolatile memory, and a work memory such as RAM or system memory.

FIG. 3 is a diagram showing an example of the software configuration of the imaging device 10. The imaging device 10 (CPU 12) has functions as an imaging control section 101, an exposure control section 102, an image processing section 103, a detection section 104, and an exposure determination section 105.

The imaging control unit 101 controls the imaging unit 11 and transmits the brightness signal and color signal obtained from the imaging unit 11 to the exposure control unit 102. The exposure control unit 102 controls exposure during imaging. The exposure control unit 102 controls exposure when photographing a subject, such as the exposure time set by the aperture of the optical system (imaging optical system) constituting the imaging unit 11 or the imaging element, and the imaging sensitivity. The image processing unit 103 performs various general image processing on image data of a subject imaged at an exposure determined by the exposure control unit 102. Image processing includes, for example, gamma correction, WB processing, color balance processing, sharpness processing, noise reduction processing, aberration correction, shading correction, and the like.

The exposure determining unit 105 performs imaging for light in a second wavelength range according to the subject detected by the detection unit 104 using a predetermined image or a captured image for light in the first wavelength range. Determine the actual exposure. When a plurality of subjects are detected, the exposure determining unit 105 can determine the exposure according to each of the plurality of subjects, or can determine the exposure by adopting the maximum reflectance among the reflectances of the plurality of subjects. You may decide. Although data regarding the spectral reflectance of the subject is used to determine the exposure, data stored in advance in the memory 13 or data acquired from an external device via the communication unit 16 may also be used. Exposure determining section 105 transmits the determined exposure to exposure controlling section 102.

In this embodiment, the configuration of the imaging device 10 shown in FIGS. 2 and 3 is used when imaging is performed using one device for light in multiple wavelength ranges, or when multiple devices are used for each wavelength range. It is applicable to any case where imaging is performed. Furthermore, in this embodiment, the imaging device 10 is installed in a rice field, and in order to obtain color information of leaves of plants as subjects, visible light (light in the first wavelength range) and near-infrared light (light in the second wavelength range) are used. An imaging device 10 that performs imaging using light) will be described. Note that in this embodiment, the plants as subjects include at least one of rice and wheat, but are not limited thereto, and may be other plants.

Next, with reference to FIG. 4, a procedure (control method) for acquiring leaf color information of a subject (target plant) in this embodiment will be described. FIG. 4 is a flowchart showing the control method in this embodiment.

First, in step S110, the CPU 12 (imaging control unit 101) uses the imaging unit 11 to perform imaging using visible light (R, G, B) in a first exposure. Subsequently, in step S111, the CPU 12 (exposure control unit 102) calculates the near-infrared light reflectance and the visible light reflectance (reflectance Ri of the subject in the wavelength range Wi) of a known plant (for example, rice). is obtained from the memory 13 (data reading). Subsequently, in step S112, the CPU 12 (exposure determining unit 105) uses the exposure (first exposure) during imaging using visible light and the reflectance acquired from the memory 13 to determine whether near-infrared light is used. The exposure (exposure time, second exposure) during image capture is determined. Here, the exposure time (first exposure) to visible light is Tg, the reflectance (first reflectance) of the leaves of the target plant (rice) for G (green) of visible light is Rg, and near-infrared light Let Ri be the reflectance (second reflectance) of the leaves of the target plant (rice). At this time, the exposure time Ti (second exposure) during imaging with near-infrared light is expressed as in the following equation (1).

Ti=Tg×Ri/Rg…(1)
Note that in this embodiment, in addition to the reflectance of the leaves of the target plant (ratio of the first reflectance to the second reflectance), exposure is performed using the spectral sensitivity or light source spectral characteristics of the imaging device 10 acquired in advance. A time Ti may also be determined.

Subsequently, in step S113, the CPU 12 (exposure control unit 102, imaging control unit 101) performs imaging using near-infrared light with the exposure (exposure time Ti) determined in step S112. Subsequently, in step S114, the CPU 12 (image processing unit 103) calculates the near-infrared light spectra of the target plant (rice) and the water in the paddy field from the image captured in the near-infrared light acquired in step S113. The leaf area of the target plant in the image is extracted using differences in reflectance. The CPU 12 (image processing unit 103) then acquires color information in the leaf area from the image captured using visible light.

As described above, the imaging unit 11 is capable of imaging using light in the first wavelength range and imaging using light in the second wavelength range different from the first wavelength range. The control unit (CPU 12) uses the first exposure during imaging using light within a first wavelength range and the reflectance of the subject to determine the second exposure during imaging using light within a second wavelength range. decide. In this embodiment, the light in the first wavelength range is visible light, and the light in the second wavelength range is near-infrared light, but the present invention is not limited to this. Infrared light or light in the second wavelength range may be used as visible light.

(Second embodiment)
Next, with reference to FIG. 5, a procedure (control method) for acquiring leaf color information of a subject (target plant) in the second embodiment will be described. FIG. 5 is a flowchart showing the control method in this embodiment. In this embodiment, an imaging device 10 that captures visible light images and near-infrared light images in order to determine the growth state of plants will be described. The imaging device 10 of this embodiment includes a detection unit 104 and detects a subject (detection target) in an image. The detection unit 104 detects various subjects such as plants, animals, or people in the image that has been subjected to various image processing by the image processing unit 103 (determines the type and part of the subject). Note that the detection unit 104 notifies the exposure determination unit 105 of the detection result of the subject.

First, in step S120, the CPU 12 (imaging control section 101) uses the imaging section 11 to perform imaging using near-infrared light. Subsequently, in step S121, the CPU 12 (detection unit 104) determines whether the detected leaf of the plant is a healthy leaf. That is, the CPU 12 detects healthy leaves and withered leaves based on the image obtained by imaging with near-infrared light (near-infrared light image), using the reflectance of leaves that differ depending on the moisture content, etc. . If it is determined that the detected leaf is a healthy leaf, the process advances to step S122. On the other hand, if it is determined that the detected leaf is not a healthy leaf, the process advances to step S126.

In steps S122 and S126, the CPU 12 determines the reflectance of near-infrared light and the reflectance of visible light (R, G, B) according to the growth state of the leaves (the reflectance for the subject type i and the wavelength range Wi). Ri) from the memory 13 (data reading). Subsequently, in steps S123 and S127, the CPU 12 (exposure determining unit 105) performs imaging in visible light (R, G, B) using near-infrared light exposure and leaf reflectance. Determine the actual exposure (exposure time). As in the first embodiment, in addition to the reflectance, the spectral sensitivity or light source spectral characteristics of the imaging device 10 acquired in advance may be used. Subsequently, in steps S124 and S128, the CPU 12 (imaging control unit 101) performs imaging using visible light (R, G, B) with the exposure determined in steps S123 and S127. Subsequently, in steps S125 and S129, the CPU 12 acquires color information in the leaf area from the visible light image for each subject.

In this embodiment, it is determined whether the leaves of the subject plant are healthy leaves or not as the type of subject; however, it is not limited to this, and the type of plant such as rice or wheat, or , processing may be based on determination of plant parts such as leaves, ears, flowers, and fruits.

As described above, in this embodiment, the control unit (CPU 12) determines the type or region of the subject based on the captured image using light in the first wavelength range. Then, the control unit determines exposure during imaging using light in the second wavelength range, using reflectance depending on the type or region of the subject. Further, in this embodiment, the imaging unit 11 performs imaging using light in the second wavelength range for each type or part of the subject. Note that in this embodiment, the light in the first wavelength range is near-infrared light, and the light in the second wavelength range is visible light; however, the light in the first wavelength range is visible light. The light in the second wavelength range may be near-infrared light.

(Third embodiment)
Next, with reference to FIG. 6, a procedure (control method) for acquiring leaf color information of a subject (target plant) in the third embodiment will be described. FIG. 6 is a flowchart showing the control method in this embodiment. In this embodiment, an imaging device 10 will be described in which the imaging device 10 is installed directly above a target plant and captures and acquires a visible light image and a near-infrared light image in order to calculate the height of the plant's leaves. . Note that in this embodiment, the imaging device 10 has a plurality of imaging units 11 for each wavelength range in order to perform imaging in a plurality of wavelength ranges. Specifically, the imaging unit 11 includes an imaging unit for visible light imaging and an imaging unit for near-infrared light in a stereo camera.

First, in step S130, the CPU 12 (imaging control unit 101) uses the imaging unit 11 to capture an image using visible light (R, G, B) in a first exposure (imaging using light in a first wavelength range). I do. In step S131, the CPU 12 calculates the reflectance of a known plant (for example, rice) with near-infrared light and the reflectance with visible light (R, G, B) (reflectance Ri of the subject in the wavelength range Wi). Each is acquired from the memory 13 (data reading). Subsequently, in step S132, the CPU 12 (exposure determining unit 105) determines exposure (second exposure) during imaging using near-infrared light (light in the second wavelength range). As in the first embodiment, in addition to the reflectance, the spectral sensitivity or light source spectral characteristics of the imaging device 10 acquired in advance may be used.

Subsequently, in step S133, the CPU 12 (imaging control unit 101) uses the imaging unit 11 to perform near-infrared light imaging at the exposure determined in step S132 (second exposure). Next, in step S134, the CPU 12 first uses the difference in reflectance between the leaves of the target plant and other subjects (fruit, ears, water, soil, etc.) from the visible light image acquired in step S130. Then, the leaf area of the target plant in the image is extracted. Then, from the near-infrared stereo image, height information of the subject in the image is acquired, and among them, height information in the leaf area is acquired.

In this embodiment, the light in the first wavelength range is visible light, and the light in the second wavelength range is near-infrared light, but the present invention is not limited to this. Infrared light or light in the second wavelength range may be used as visible light.

(Other embodiments)
The present invention provides a system or device with a program that implements one or more functions of the embodiments described above via a network or a storage medium, and one or more processors in a computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

According to each embodiment, it is possible to provide an imaging device, a control method for the imaging device, and a program that can obtain appropriate exposure for each of light in a plurality of wavelength ranges with low power consumption.

The disclosure of each embodiment includes the following configurations and methods.
(Configuration 1)
an imaging unit that captures an image of a subject;
A control unit that controls exposure during imaging;
The imaging unit is capable of first imaging using light in a first wavelength range and second imaging using light in a second wavelength range different from the first wavelength range,
The imaging device is characterized in that the control unit determines the second exposure during the second imaging using the first exposure during the first imaging and the reflectance of the subject.
(Configuration 2)
The imaging device according to configuration 1, further comprising a storage unit that stores data regarding the reflectance.
(Configuration 3)
The data regarding the reflectance includes a first reflectance of the object for light in the first wavelength range and a second reflectance of the object for light in the second wavelength range,
The imaging device according to configuration 2, wherein the control unit determines the second exposure based on a ratio between the first reflectance and the second reflectance.
(Configuration 4)
The light in the first wavelength range is one of visible light or near-infrared light,
4. The imaging device according to any one of configurations 1 to 3, wherein the light in the second wavelength range is the other of the visible light and the near-infrared light.
(Configuration 5)
The control unit includes:
Detecting the type or part of the subject based on the image obtained by the first imaging,
5. The imaging device according to any one of configurations 1 to 4, wherein the second exposure is determined using the reflectance according to the type or the part of the subject.
(Configuration 6)
The imaging device according to configuration 5, wherein the imaging unit performs the second imaging for each type or part of the subject.
(Configuration 7)
7. The imaging device according to any one of configurations 1 to 6, wherein the subject is a plant.
(Configuration 8)
The control unit includes:
Detecting the growth state of the plant based on the image obtained by the first imaging,
8. The imaging device according to configuration 7, wherein the second exposure is determined using the reflectance depending on the growth state.
(Configuration 9)
9. The imaging device according to configuration 8, wherein the growth state is a state regarding whether or not the plant has healthy leaves.
(Configuration 10)
10. The imaging device according to any one of configurations 71 to 9, wherein the plant includes at least one of rice and wheat.
(Configuration 11)
11. The imaging device according to any one of configurations 1 to 10, wherein the control unit determines the second exposure by further using spectral sensitivity of the imaging device.
(Method 1)
imaging a subject with a first exposure using light in a first wavelength range;
determining a second exposure for imaging using light in a second wavelength range different from the first wavelength range, using the first exposure and the reflectance of the subject;
A method for controlling an imaging device, comprising the step of imaging the subject at the second exposure using light in the second wavelength range.
(Configuration 12)
A program that causes a computer to execute the control method described in Method 1.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the invention.

10 Imaging device 11 Imaging unit 12 CPU (control unit)

Claims (13)

an imaging unit that captures an image of a subject;
A control unit that controls exposure during imaging,
The imaging unit is capable of first imaging using light in a first wavelength range and second imaging using light in a second wavelength range different from the first wavelength range,
The imaging device is characterized in that the control unit determines the second exposure during the second imaging using the first exposure during the first imaging and the reflectance of the subject. The imaging device according to claim 1, further comprising a storage unit that stores data regarding the reflectance. The data regarding the reflectance includes a first reflectance of the object for light in the first wavelength range and a second reflectance of the object for light in the second wavelength range,
The imaging device according to claim 2, wherein the control unit determines the second exposure based on a ratio between the first reflectance and the second reflectance. The light in the first wavelength range is one of visible light or near-infrared light,
The imaging device according to any one of claims 1 to 3, wherein the light in the second wavelength range is the other of the visible light and the near-infrared light. The control unit includes:
Detecting the type or part of the subject based on the image obtained by the first imaging,
The imaging device according to any one of claims 1 to 3, wherein the second exposure is determined using the reflectance according to the type or the part of the subject. The imaging device according to claim 5, wherein the imaging unit performs the second imaging for each type or part of the subject. The imaging device according to any one of claims 1 to 3, wherein the subject is a plant. The control unit includes:
Detecting the growth state of the plant based on the image obtained by the first imaging,
The imaging device according to claim 7, wherein the second exposure is determined using the reflectance according to the growth state. The imaging device according to claim 8, wherein the growth condition is a condition related to whether the plant has healthy leaves. The imaging device according to claim 7, wherein the plant includes at least one of rice and wheat. The imaging device according to any one of claims 1 to 3, wherein the control unit determines the second exposure further using spectral sensitivity of the imaging device. imaging a subject with a first exposure using light in a first wavelength range;
using the first exposure and the reflectance of the subject to determine a second exposure for imaging using light in a second wavelength range different from the first wavelength range;
A method for controlling an imaging device, comprising the step of imaging the subject in the second exposure using light in the second wavelength range. A program that causes a computer to execute the control method according to claim 12.