WO2023047825A1

WO2023047825A1 - Imaging device and imaging method

Info

Publication number: WO2023047825A1
Application number: PCT/JP2022/030411
Authority: WO
Inventors: 臣一下津; 智行河合; 哲也藤川; 智大島田; 敏浩青井
Original assignee: 富士フイルム株式会社
Priority date: 2021-09-27
Filing date: 2022-08-09
Publication date: 2023-03-30

Abstract

Provided are an imaging device and an imaging method with which it is possible to obtain an image that allows good visual recognition of subjects, said subjects being a heat source that radiates infrared light and the surroundings of the heat source which reflect visible light. An imaging device (1) captures a near-infrared image and a visible image, and comprises: an imaging optical system (100) that has transmissive characteristics in the near-infrared light wavelength band and the visible light wavelength band; an image sensor (132) that receives light transmitted through the imaging optical system (100) and acquires an image; and a light amount adjustment optical element that attenuates the light amount of near-infrared light and/or the light amount of visible light being transmitted through the imaging optical system (100). The light amount adjustment optical element adjusts the light amount balance between the light amount of near-infrared light and the light amount of visible light received by the image sensor (132).

Description

Imaging device and imaging method

The present invention relates to an imaging device and an imaging method, and more particularly to technology of an imaging device and an imaging method for capturing a near-infrared image and a visible image.

Conventionally, techniques for capturing images of light having wavelength bands other than visible light and visible light have been proposed.

Patent Document 1 describes an imaging device capable of imaging with visible light and imaging with near-infrared light.

International Publication No. 2020/095513

One embodiment according to the technology of the present disclosure is an imaging device that can capture an image in which a heat source that radiates near-infrared light and its surroundings that reflect visible light are used as subjects, and the subject can be visually recognized well. and an imaging method.

An imaging device according to one aspect of the present invention is an imaging device that captures a near-infrared image and a visible image, and is imaging optics having transmission characteristics in the wavelength band of near-infrared light and the wavelength band of visible light. a system, an image sensor that acquires an image by receiving light that has passed through the imaging optical system, a light amount adjustment optical element that attenuates the amount of near-infrared light and/or the amount of visible light that passes through the imaging optical system, and the light amount adjusting optical element adjusts the light amount balance between the light amount of near-infrared light and the light amount of visible light received by the image sensor.

Preferably, the light amount adjusting optical element constitutes a first filter that attenuates near-infrared light and/or a second filter that attenuates visible light.

Preferably, the first filter is composed of a plurality of light amount adjusting optical elements.

Preferably, the second filter is composed of a plurality of light amount adjusting optical elements.

Preferably, the first filter and/or the second filter are held by a filter holding mechanism that holds the light amount adjusting optical element removably into the optical path of the imaging optical system.

Preferably, the filter holding mechanism is configured with a turret having a first filter and/or a second filter.

Preferably, the second filter can attenuate visible light by 30 dB or more.

Preferably, the image sensor acquires a superimposed image in which the near-infrared image and the visible image are superimposed.

Preferably, in the imaging optical system, the diameter of the near-infrared light source image of the point light source image when the visible image is in focus is 0.2 mm or more and 0.9 mm or less.

Preferably, the imaging device includes a control section, and the control section causes the light amount adjustment optical element to adjust the balance of the light amount based on the image acquired by the image sensor.

An imaging method, which is another aspect of the present invention, is an imaging apparatus for imaging a near-infrared image and a visible image, and is imaging optics having transmission characteristics in the wavelength band of near-infrared light and the wavelength band of visible light. and an image sensor that acquires an image by receiving light that has passed through the imaging optical system, wherein near-infrared light that passes through the imaging optical system is detected by a light amount adjusting optical element. and/or an attenuation step of attenuating the amount of light and/or the amount of visible light, wherein the attenuation step adjusts the light amount balance between the amount of near-infrared light and the amount of visible light received by the image sensor.

FIG. 1 is a schematic configuration diagram of an imaging device. FIG. 2 is a schematic diagram of the optical filter switching section viewed from the AA direction of FIG. FIG. 3 is a diagram showing transmission spectrum data of the first filter. FIG. 4 is a diagram showing transmission spectrum data of the second filter. FIG. 5 is a schematic block diagram of the imaging device. FIG. 6 is a schematic configuration diagram of a computer. FIG. 7 shows a profile of light transmittance of the imaging optical system. FIG. 8 is a diagram showing a thermal radiation spectrum generated from a heat source. FIG. 9 is a diagram showing light intensity at each wavelength of a 400° C. heat source and a 200° C. heat source. FIG. 10 is a diagram showing a superimposed image of a visible image and a near-infrared image of a soldering iron whose tip is heated to 400° C. and its periphery as an object. FIG. 11 is a diagram showing a superimposed image of a visible image and a near-infrared image of a soldering iron whose tip is heated to 400° C. and its periphery as an object. FIG. 12 is a diagram showing the relationship between the focal length and the diameter of the blurred image. FIG. 13 is a diagram showing focal length and focal position difference. FIG. 14 is a diagram showing a simulation of defocus and the diameter of a blurred image. FIG. 15 is a flow chart showing an imaging method of the imaging device. FIG. 16 is a schematic diagram of an optical filter switching unit. FIG. 17 is a flow chart showing an imaging method of the imaging device. FIG. 18 is a schematic configuration diagram of an imaging device.

Preferred embodiments of an imaging device and an imaging method according to the present invention will be described below with reference to the accompanying drawings.

<Imaging device>
FIG. 1 is a schematic configuration diagram of an imaging apparatus of the present invention.

As shown in FIG. 1 , the imaging device 1 includes an imaging optical system 100 and an imaging section 130 . The imaging unit 130 has an image sensor 132 . The image sensor 132 receives light transmitted through the imaging optical system 100, converts an optical image of an imaging target into an electrical signal, and obtains an image. The imaging device 1 is a thermal camera capable of imaging a near-infrared light image (near-infrared image) and a visible light image (visible image), for example, with a heat source and its surroundings as subjects.

The imaging optical system 100 has a plurality of lenses. The imaging optical system 100 includes a focus lens 12, a zoom lens 14, an aperture 30, an optical filter switching unit 50, and an adjustment lens 16 from the object side to the imaging side. The object side is the side on which the object to be imaged is located, and the imaging side is the side on which an optical image of the object to be imaged is formed, that is, the side on which the image sensor 132 is located. An “imaging optical system” as used in the present disclosure means, for example, an optical system for forming an optical image of an imaging target on the imaging surface 132A of the image sensor 132 using a plurality of lenses. The “imaging optical system” may include not only lenses but also optical elements such as diaphragms, optical filters, half mirrors, and/or deflection elements.

The focus lens 12 is an optical system that adjusts the focus position of the image to be captured. The zoom lens 14 is an optical system that adjusts zoom magnification. The focus lens 12 and the zoom lens 14 move back and forth along the optical axis OP of the imaging optical system 100 in conjunction with each other by a cam mechanism (not shown). As a result, the magnification is changed and the focal position is adjusted so that the imaging surface 132A of the image sensor 132 is in focus. Note that the optical axis OP is also called an optical path OP. The focus lens 12 and the zoom lens 14 are driven by rotating a zoom cam (not shown) by a zoom lens driving mechanism 20 . The zoom lens driving mechanism 20 is controlled by the control section 110 according to instructions given to the imaging device 1 by the user. An objective lens for condensing light from an imaging target may be provided on the objective side of the zoom lens 14 .

The diaphragm 30 is an optical element that blocks unnecessary light such as stray light and narrows down the luminous flux. In FIG. 1, the diaphragm 30 is arranged between the zoom lens 14 and the optical filter switching unit 50, but the position of the diaphragm 30 is not limited to this. You may arrange|position so that a movement is possible.

The optical filter switching unit 50 includes a filter holding mechanism that holds a first filter composed of a light amount adjustment optical element that attenuates near-infrared light and a second filter composed of a light amount adjustment optical element that attenuates visible light. .

FIG. 2 is a schematic diagram of the optical filter switching section 50 viewed from the AA direction in FIG.

The optical filter switching unit 50 is a turret switching device in which four

optical filters

52, 54, 56 and 58 are arranged on a disc. This switching device rotates a disk by a turret drive mechanism 22 such as a motor, and holds each optical filter so that it can be extracted onto the optical path OP. Note that the rotation of the disk may be clockwise or counterclockwise.

The optical filter switching unit 50 has a sensor (not shown) for detecting the filter arranged on the optical path OP. The sensor may be installed at the turret driving mechanism 22 instead of the optical filter switching section 50 . The turret driving mechanism 22 is controlled by the control section 110 in accordance with instructions given to the imaging apparatus 1 by the user.

Although the optical filter switching section 50 is arranged between the zoom lens 14 and the adjustment lens 16 in FIG. 1, the position of the optical filter switching section 50 is not limited to this. The optical filter switching unit 50 can be arranged between the object side of the focus lens 12 and the imaging side of the adjustment lens 16 . For example, the optical filter switching section 50 may be arranged between the adjustment lens 16 and the image sensor 132 .

Further, the imaging device 1 may have a configuration in which the housing 90 that houses the imaging optical system 100 and the imaging unit 130 can be separated. For example, the imaging device 1 may be configured such that the housing 90 is an interchangeable lens unit, the imaging unit 130 is a camera unit, and any one of a plurality of types of lens units can be attached to one camera unit. good. In this case, the optical filter switching section 50 may be arranged in the imaging section 130, that is, in the camera section.

The optical filter 52 is a first filter composed of one light amount adjusting optical element that attenuates the light amount of near-infrared light. The optical filter 52 is a bandpass filter that reduces the light transmittance of at least part of the wavelength band of near-infrared light. At least a part of the wavelength band of the near-infrared light means the wavelength band of the near-infrared light that passes through the imaging optical system 100 . Here, at least a part of the wavelength band of the near-infrared light refers to, for example, a wavelength band of 1100 nm or more in the wavelength band of the near-infrared light. A part of the wavelength band of near-infrared light refers to, for example, a near-infrared light peak wavelength band described later. The optical filter 52 is an example of a first filter according to the technology of the present disclosure. The optical filter 52 is used, for example, to attenuate near-infrared light emitted from a high-temperature heat source (for example, 1500° C.) when the amount of light is extremely high.

FIG. 3 is a diagram showing transmission spectrum data of a first filter composed of one light amount adjusting optical element. The vertical axis indicates transmittance, and the horizontal axis indicates wavelength. FIG. 3A shows the transmittance in the range of 0 to 100%, and FIG. 3B is an enlarged view showing the transmittance in the range of 0 to 10%.

The first filter is composed of, for example, a multilayer filter. The first filter has a transmittance of approximately 80% or more in the visible light wavelength band, whereas the transmittance of the near-infrared light wavelength band (for example, wavelengths of 1500±100 nm) is approximately 1% or less. Therefore, the first filter can selectively attenuate only the amount of near-infrared light, and can balance the amount of visible light and the amount of near-infrared light. The first filter can further attenuate the amount of near-infrared light by providing multiple stages. When the first filter is composed of one light amount adjusting optical element, the light amount of near-infrared light can be attenuated by 20 dB. When the first filter is composed of two light amount adjusting optical elements, it can attenuate the light amount of near-infrared light by 40 dB. It should be noted that the illustrated transmission spectrum data of the first filter is an example, and the present invention is not limited to this.

The optical filter 54 is a second filter composed of one light amount adjusting optical element that attenuates the amount of visible light. The optical filter 54 is a bandpass filter that reduces the light transmittance of at least part of the wavelength band of visible light. The wavelength band of at least part of visible light means the wavelength band of visible light that passes through the imaging optical system 100 . Here, at least a part of the wavelength band of visible light refers to, for example, a wavelength band of 800 nm or less. Further, the partial wavelength band of the visible light wavelength band refers to, for example, the visible light peak wavelength band described later. The optical filter 54 is an example of a second filter according to the technology of the present disclosure. For example, the optical filter 54 has a very strong visible light such as illumination, and when the heat source is relatively low temperature (for example, 200° C. to 300° C.), the amount of near-infrared light radiated from the heat source is relatively low. get weaker. In such a case, the second filter is used to attenuate the amount of visible light.

FIG. 4 is a diagram showing transmission spectrum data of a second filter composed of one light amount adjusting optical element. The vertical axis indicates transmittance, and the horizontal axis indicates wavelength. FIG. 4A shows the transmittance in the range of 0 to 100%, and FIG. 4B is an enlarged view showing the transmittance in the range of 0 to 1%.

The second filter is composed of, for example, a multilayer filter. The second filter has a transmittance of 90% or more for light in the wavelength band of near-infrared light (for example, a wavelength of 1500±100 nm), whereas it transmits light in the wavelength band of visible light (for example, a wavelength of 400 to 600 nm). The transmittance is approximately 1% or less. Therefore, the second filter can selectively attenuate only the amount of visible light, and can balance the amount of visible light and the amount of near-infrared light. In addition, the second filter can attenuate the light amount of the near-infrared light more by making it multistage. When the second filter is composed of one light amount adjusting optical element, it can attenuate the light amount of visible light by 30 dB. The second filter can attenuate the amount of visible light by 60 dB when it is composed of two light amount adjusting optical elements. The illustrated transmission spectrum data of the second filter is an example, and the present invention is not limited to this.

The optical filter 56 is a transparent glass plate having a refractive index close to that of the other

optical filters

52, 54 and 58. The optical filter 56 is an optical path length adjustment filter for not changing the optical path length as much as possible from the optical path length when the

optical filters

52, 54 and 58 are used when the other

optical filters

52, 54 and 58 are not used. is.

The optical filter 58 is an ND (Neutral Density) filter for adjusting the amount of light.

Returning to FIG. 1, the adjustment lens 16 is a lens for adjusting the difference between the focal length for visible light and the focal length for near-infrared light. The focal length of near-infrared light, which has a longer wavelength than that of visible light, is longer than that of visible light. The focus lens 12 and the zoom lens 14 are configured to move in conjunction with each other so that the focal position during zooming with visible light is aligned with the imaging surface 132A of the image sensor 132 . Therefore, it is impossible to adjust the focus position with near-infrared light. Therefore, the near-infrared image can be focused by moving the adjusting lens 16 along the optical path OP.

The adjustment lens 16 is driven by an adjustment lens driving mechanism 24. The adjustment lens drive mechanism 24 is controlled by the control section 110 according to instructions from the user. Specifically, the control unit 110 controls the adjusting lens drive mechanism 24 so that the position of the adjusting lens 16 is adjusted to the in-focus position according to the imaging conditions instructed by the user. Here, the imaging condition refers to, for example, selection of focusing on a visible image or a near-infrared image and selection of zoom magnification according to a user's instruction. Note that the focus position of the adjustment lens 16 refers to the position of the adjustment lens 16 for forming an image of light (visible light or near-infrared light) on the imaging surface 132A of the image sensor 132 in a focused state.

The control unit 110 adjusts the position of the adjustment lens 16 based on instructions from the user. Specifically, when a command to focus on the visible image is input, the adjustment lens 16 is moved to the in-focus position in the visible image, and when a command to focus on the near-infrared image is input, the adjustment lens 16 is moved. to the in-focus position in the near-infrared image.

Although illustrated in FIG. 1 in a simplified manner, each of the focus lens 12, the zoom lens 14, and the adjustment lens 16 is composed of one or more lens groups. The imaging optical system 100 is composed of several to several tens of lenses as a whole. Each lens of the imaging optical system 100 is coated so as to have high light transmittance in a specific wavelength band of visible light and near-infrared light. Coatings may be applied to only some of the total lenses. However, it is more preferable to apply the coating to all lenses. Note that the configuration of the imaging lens 100 is not limited to the above aspect. For example, various types of lens configurations are known for the zoom lens 14 and the focus lens 12, and configurations for zoom control and focus control (extend the entire lens group, extend a part of the lens group, etc.). ) are also known, and their constructions can also be employed in the present invention.

FIG. 5 is a schematic block configuration diagram of the imaging device 1. FIG. 6 is a schematic block diagram of the computer shown in FIG.

The imaging device 1 is controlled by the control unit 110 . The control unit 110 has a computer 200 . As shown in FIG. 6 as an example, the computer 200 includes a CPU (Central Processing Unit) 202, a RAM (RAM: random access memory) 204, and a ROM (ROM: read only memory) 206, which are connected to each other via a bus line 112. have. The CPU 202 controls the imaging device 1 as a whole. A RAM 204 is, for example, a volatile memory that is used as a work area or the like when executing an imaging apparatus control program. The ROM 206 is, for example, a non-volatile memory that stores an imaging device control program 210 that controls the imaging device 1, focus position data 212, and the like. Although the CPU 202 is exemplified in this embodiment, it is also possible to use a plurality of CPUs instead of the CPU 202 .

The CPU 202 reads the imaging device control program 210 from the ROM 206 and develops the read imaging device control program 210 in the RAM 204 . Then, the CPU 202 executes the imaging apparatus control program 210 to control the zoom lens driving section 114, the turret driving section 116, and the adjustment lens driving section 118 shown in FIG. 5 as an example.

The focus position data 212 is data on the position of the adjusting lens 16 when focusing on a visible image and the position of the adjusting lens 16 when focusing on a near-infrared image. The focus position data 212 is stored as position data of the adjustment lens 16 for each magnification of visible light and near-infrared light, for example.

A known mechanism can be used for the zoom lens drive mechanism 20, the turret drive mechanism 22, and the adjustment lens drive mechanism 24. Although FIG. 1 shows the case where these mechanisms are inside the housing 90 , they may be arranged outside the housing 90 .

The image sensor 132 is composed of, for example, an InGaAs imaging element capable of imaging a subject with both visible light and near-infrared light wavelengths. An imaging surface 132A of the image sensor 132 is provided with a filter, specifically, a filter that selectively receives visible light and near-infrared light. Note that the image sensor 132 can acquire a visible image as a color image or as a monochrome image. An image obtained by the image sensor 132 is, for example, a superimposed image in which a near-infrared image and a visible image are superimposed.

An optical image formed by the imaging optical system 100 is converted into an electrical signal by the image sensor 132 of the imaging unit 130, and after various image processing is performed, it is displayed as an image on the image display unit 26, which will be described later. Also, an image that has undergone image processing may be transmitted to the outside by wire or wirelessly.

As shown in FIG. 5, the control unit 110 includes a zoom lens driving unit 114, a turret driving unit 116, an adjusting lens driving unit 118, an output I/F (I/F: interface) 120, an input I/F 122, an image processing unit 126, and computer 200. These are connected by a bus line 112 . Control unit 110 also includes an external I/F 124 .

The zoom lens driving section 114 is connected to the zoom lens driving mechanism 20 . The turret driving section 116 is connected to the turret driving mechanism 22 . The adjustment lens drive section 118 is connected to the adjustment lens drive mechanism 24 . The output I/F 120 is connected to the image display section 26 . The input I/F 122 is connected to the image sensor 132 and the input section 28 .

The image display unit 26 displays images based on image signals input via the output I/F 120 . The input unit 28 accepts instructions given by the user. The input I/F 122 is an interface for receiving an electric signal from the image sensor 132 and an instruction input by the user via the input unit and sending the same to the computer 200 . The external I/F 124 is an interface for receiving an instruction from a user, for example, through wireless communication, and transmitting an image that has undergone image processing through wireless communication. The image processing unit 126 processes the image acquired by the image sensor 132 .

The zoom lens drive unit 114 adjusts the positions of the focus lens 12 and the zoom lens 14 by controlling the zoom lens drive mechanism 20 according to instructions from the computer 200 . The turret driving section 116 switches the filters of the optical filter switching section 50 by controlling the turret driving mechanism 22 according to instructions from the control section 110 . The adjusting lens driving section 118 adjusts the position of the adjusting lens 16 by controlling the adjusting lens driving mechanism 24 according to the instruction from the control section 110 . The output I/F 120 is an interface for sending to the image display section 26 a captured image obtained by performing image processing by the image processing section 126 .

FIG. 7 is a diagram showing a profile of light transmittance of the imaging optical system 100. FIG. The horizontal axis of FIG. 7 is the wavelength, and the vertical axis is the light transmittance of the imaging optical system 100 . The imaging optical system 100 has transmission characteristics in the wavelength band of near-infrared light and the wavelength band of visible light. As shown in FIG. 7, the optical transmittance profile of the imaging optical system 100 has a first transmittance peak PK1 in the near-infrared light peak wavelength band NIR (NIR: Near InfraRed) from 1450 nm to 1650 nm. That is, the light transmittance on the short wavelength side of the near-infrared light peak wavelength band NIR decreases as the wavelength becomes shorter from the light transmittance at the short wavelength end (1450 nm) of the near-infrared light peak wavelength band NIR. . In addition, the light transmittance on the longer wavelength side than the near-infrared light peak wavelength band NIR decreases as the wavelength increases from the light transmittance at the long wavelength end (1650 nm) of the near-infrared light peak wavelength band NIR. .

As can be seen from FIG. 7, the light transmittance of the first transmittance peak PK1 is about 92% at a wavelength of 1520 nm. Moreover, the light transmittance in the wavelength range from 1490 nm to 1560 nm is 90% or more. The first filter described above reduces the light transmittance in the wavelength band of the near-infrared light peak wavelength band NIR to attenuate the light amount of the near-infrared light.

In addition, the optical transmittance profile of the imaging optical system 100 has a second transmittance peak PK2 in the visible light peak wavelength band VIS (VIS: Visible) from 450 nm to 700 nm. That is, the light transmittance on the shorter wavelength side than the visible light peak wavelength band VIS decreases as the wavelength shortens from the light transmittance at the short wavelength end (450 nm) of the visible light peak wavelength band VIS. Further, the light transmittance on the longer wavelength side than the visible light peak wavelength band VIS decreases as the wavelength increases from the light transmittance at the long wavelength end (700 nm) of the visible light peak wavelength band VIS.

As can be seen from FIG. 7, the light transmittance of the second transmittance peak PK2 is about 96% at wavelengths from 570 nm to 580 nm. Moreover, the light transmittance in the wavelength range of 480 nm to 660 nm is 90% or more. The second filter described above reduces the light transmittance of the wavelength band of the visible light peak wavelength band VIS to attenuate the amount of visible light.

In addition, the light transmittance of the wavelength band on the short wavelength side of the blue wavelength band included in the wavelength band of visible light is lower than the light transmittance of the wavelength band on the long wavelength side of the wavelength band of blue light. Specifically, the light transmittance in the wavelength band of 450 nm or less in the blue wavelength band is smaller than the light transmittance in the wavelength band longer than 450 nm. Moreover, the light transmittance at wavelengths from 400 nm to 430 nm is 50% or less. If the light transmittance in the wavelength range of 400 nm to 430 nm is more than 50%, the light transmittance in the wavelength range of 1200 nm to 1290 nm, which is the peak third harmonic of the near-infrared wavelength band, also increases. This means that the peak of the wavelength band of near-infrared light is widened, and there is a possibility that the light transmittance in the vicinity of the wavelength of 1550 nm is lowered, or the characteristics are deteriorated such as residual ripples.

Furthermore, the imaging optical system 100 has a higher light transmittance than the near-infrared peak wavelength band and the visible peak wavelength band over a wavelength range of 900 nm to 1100 nm between the near-infrared peak wavelength band and the visible peak wavelength band. It has a small, low light transmittance region LOW. The light transmittance of the low light transmittance area LOW is preferably 5% or less. The low light transmittance region LOW forms a light transmittance peak in the near-infrared light region in the near-infrared light peak wavelength band NIR, and forms a light transmittance peak in the visible light region in the visible light peak wavelength band VIS. This is the area that arises as a result of doing so. However, since the wavelength of the low light transmittance region LOW is a wavelength band that does not contribute to imaging with visible light or near-infrared light, the light transmittance of the low light transmittance region LOW is low. is not a problem.

The light transmittance profile shown in FIG. 7 has one light transmittance peak PK1 in the near-infrared light peak wavelength band NIR and one light transmittance peak PK2 in the visible light peak wavelength band VIS. . However, the light transmittance profile of the present disclosure is not limited to this. It may have a waveform shape (ripple) due to a plurality of light transmittance peaks in the near-infrared light peak wavelength band NIR. Moreover, it may have a ripple in the visible light peak wavelength band VIS. Ripple is a shape that exhibits one characteristic of variations in light transmission. Thus, any profile having a light transmittance peak in the near-infrared light peak wavelength band NIR and a light transmittance peak in the visible light peak wavelength band VIS may be sufficient. The number is not limited.

The first transmittance peak PK1 formed in the near-infrared light peak wavelength band NIR should have a half width as narrow as possible. Near-infrared light, which has a longer wavelength than visible light, is more prone to chromatic aberration than visible light when the wavelength range is widened. Therefore, it is preferable that the imaging wavelength range be as narrow as possible.

The profile of light transmittance as shown in FIG. 7 shows that the light transmittance peak at one-third wavelength of the fundamental wave caused by the interference caused by the coating of the fundamental wave having the light transmittance peak in the near-infrared light peak wavelength band is visible. Obtained by coating to lie in the optical peak wavelength band. The fundamental wave preferably has a peak near 1550 nm. A light transmittance profile that satisfies the above conditions can be obtained by constructing the coating so that the light transmission peak at 1/3 wavelength of the fundamental wave is suppressed and the light transmission peak at 1/3 wavelength is increased. can get. It is possible by conventional techniques to design and form a coating that provides a light transmission profile that satisfies the above conditions.

<Light balance adjustment>
Next, the adjustment of the light amount balance between the light amount of near-infrared light and the light amount of visible light will be described. In the imaging apparatus 1, adjustment of the light amount balance is performed by a first filter (optical filter 52) or a second filter (optical filter 54) configured by a light amount adjusting optical element.

FIG. 8 is a diagram showing a thermal radiation spectrum generated from a heat source. The vertical axis indicates the light intensity (au), and the horizontal axis indicates the wavelength (μm).

FIG. 8 shows the light intensity at each wavelength of a 10°C heat source, a 200°C heat source, and a 1500°C heat source. Also shown is the limit value LD of the light intensity observable by the InGaAs imaging device.

Generally, the temperature of flames at fire sites reaches around 1500°C. As shown in FIG. 8, the light intensity of near-infrared light (wavelength = around 1.5 μm) radiated from a heat source at 1500° C. is 1.00E+11 or more, and visible light (wavelength = around 0.5 μm) 5 or more orders of magnitude (indicated by arrow W1) compared to the light intensity of . Then, when the flames and their surroundings at the fire site are taken as an object using near-infrared light and visible light, the near-infrared image part is blown out in the obtained image, and the object is cannot be seen well. For example, in the image acquired by the image sensor 132, the practical observation width is about 25 dB (indicated by arrow W2). Therefore, in such a case, the above-described first filter is inserted in the optical path OP to attenuate the light amount of the near-infrared light and adjust the light amount balance between the light amount of the near-infrared light and the light amount of the visible light. it becomes necessary to

A specific example of capturing a superimposed image of a near-infrared image and a visible image by heating the tip of a soldering iron to 400°C as a heat source will be described below.

FIG. 9 is a diagram showing the light intensity at each wavelength of a 400° C. heat source and a 200° C. heat source. The vertical axis indicates the light intensity (au), and the horizontal axis indicates the wavelength (μm).

In the state where the first filter is not inserted in the optical path OP, the light intensity of the near-infrared light radiated from the 400° C. heat source is point F1. On the other hand, by inserting the first filter into the optical path OP, the light intensity of the near-infrared light is attenuated to point F2. By attenuating the light intensity of the near-infrared light in this manner, the light intensity level becomes approximately the same as that of the visible light, as shown in the figure. As a result, a superimposed image in which the subject can be visually recognized well can be obtained without overexposure in the near-infrared image region.

FIG. 10 is a diagram showing a superimposed image of a visible image and a near-infrared image of a soldering iron whose tip is heated to 400° C. and its periphery as an object. FIG. 10A is a superimposed image when imaging without attenuating the amount of near-infrared light, and FIG. 10B is a superimposed image when imaging with the amount of near-infrared light attenuated by the first filter. FIG. 4 is a diagram showing an image;

In the case shown in FIG. 10(A), the tip of the soldering iron is a heat source of 400°C, and near-infrared light (wavelength 1550 nm) is radiated from the tip of the soldering iron. A near-infrared image corresponding to this near-infrared light has a larger amount of light than the surrounding visible image. Therefore, in the superimposed image, the portion of the near-infrared image is blown out, and the tip of the soldering iron and its peripheral portion cannot be visually recognized (see arrow E1). On the other hand, by inserting the first filter into the optical path OP, the amount of near-infrared light can be suppressed. Specifically, by inserting the first filter into the optical path OP, the near-infrared light (wavelength 1550 nm) is attenuated by 20 dB. In this way, by inserting the first filter in the optical path OP and adjusting the light amount of the near-infrared light and the light amount of the visible light, it is possible to suppress overexposure at the portion of the near-infrared image in the superimposed image. . As a result, it is possible to obtain a superimposed image in which the tip of the soldering iron and its peripheral portion can be visually recognized well.

Here, in the superimposed image shown in FIG. 10, imaging is performed so that the visible image is in focus. Therefore, the near-infrared image is out of focus and becomes a blurred image. As a result, the location of the blurred image (indicated by arrow E2) is an image corresponding to the near-infrared light of thermal radiation, and can be estimated to be the heat source or the vicinity of the heat source. On the other hand, when acquiring a superimposed image of the same subject, the imaging device 1 can obtain a superimposed image in which the near-infrared image is focused by moving the adjustment lens 16, for example.

FIG. 11 is a diagram showing a superimposed image of a soldering iron whose tip is heated to 400° C. and a visible image and a near-infrared image of its surroundings, as in FIG. Whereas in FIG. 10 the visible image was focused, in FIG. 11 the near-infrared image is focused.

FIG. 11A shows a superimposed image when an image is captured without attenuating the amount of near-infrared light, and FIG. 11B shows the amount of near-infrared light using the first filter. A superimposed image is shown when imaged with attenuation. In the case shown in FIG. 11(A), the tip of the soldering iron (near-infrared image) is overexposed because the amount of light is larger than that of the visible image around it. cannot be verified (see arrow G1). On the other hand, as shown in FIG. 11B, by inserting the first filter into the optical path OP, the near-infrared light (wavelength: 1550 nm) is attenuated by 20 dB, and the tip of the soldering iron and its peripheral portion can be checked. A possible superimposed image can be obtained (see arrow G2). Note that in the superimposed image shown in FIG. 11, the visible image is in a defocused state, but the object can be visually recognized roughly.

As described above, the imaging device 1 can focus on the near-infrared image or the visible image when acquiring a superimposed image of the near-infrared image and the visible image. When the superimposed image is focused on the visible image, it can be confirmed that the portion of the blurred image is a near-infrared image and has a high temperature. On the other hand, when the superimposed image is focused on the near-infrared image, the radiation image from the heat source can be clearly visually recognized.

<Focus and blurred image of visible image or near-infrared image>
Next, in the superimposed image acquired by the imaging device 1, the focused and blurred images of the visible image or the near-infrared image will be described.

FIG. 12 is a diagram showing the relationship between the focal length and the diameter of the blurred image, and FIG. 13 is a diagram showing the focal length and the focal position difference.

In FIG. 12, the vertical axis indicates the diameter (mm) of the blurred image, and the horizontal axis indicates the focal length (mm). The diameters of blurred images are shown for D-line (wavelength 587.56 nm) as visible light and wavelengths 1500 nm, 1550 nm and 1600 nm as near-infrared light. Since the visible image is focused, the diameter of the blurred image is 0 at the wavelength of 587.56 nm. Since the near-infrared image is focused on the visible image, it is a blurred image, and the diameter of the blurred image increases as the focal length increases.

In FIG. 13, the vertical axis indicates the focal position difference (mm), and the horizontal axis indicates the focal length (mm). It should be noted that the focal position difference shows the focal position difference with respect to the image (visible image) of the light of the D line (wavelength 587.56) with the image direction being positive. The focal position difference is shown for D line (wavelength 587.56 nm) as visible light and wavelengths 1500 nm, 1550 nm and 1600 nm as near-infrared light. The focal position difference between the visible image and the near-infrared image increases as the focal length increases.

As described above, when acquiring a superimposed image with the imaging device 1, the diameter of the blurred image of the near-infrared image increases as the focal length increases, and the focal position difference increases as the focal length increases.

FIG. 14 is a diagram showing a simulation of the defocus and the diameter of the blurred image.

Reference H1 indicates an image with a defocus of 0 mm and a blurred image with a diameter of 0 mm. Symbol H2 indicates an image with a defocus of 1 mm and a blurred image with a diameter of 0.2 mm. Symbol H3 indicates an image with a defocus of 3 mm and a blurred image diameter of 0.5 mm. Reference H4 indicates an image with a defocus of 6 mm and a blurred image with a diameter of 0.6 mm. Symbol H5 indicates an image with a defocus of 9 mm and a blurred image diameter of 0.9 mm. Reference H6 indicates an image with a defocus of 12 mm. Reference H7 indicates an image with a 15 mm defocus.

The image of the cross shown up to symbol H5 can be visually recognized well. On the other hand, in the images indicated by H6 and H7, the cross cannot be seen well. Therefore, for example, when the visible image is focused, the diameter of the blurred image of the near-infrared image is preferably 0.2 mm or more and 0.9 mm or less in order to visually recognize the subject with the blurred image. . The diameter of the blurred image is measured, for example, by the diameter of the near-infrared light image (bokeh image) of the point light source image in visible light.

<Imaging method>
Next, an imaging method using the imaging device 1 will be described.

FIG. 15 is a flowchart showing the imaging method of the imaging device 1. FIG. Note that the imaging method is performed by the CPU 202 executing an imaging device control program (imaging method execution program) stored in the computer of the imaging device 1 .

First, the first filter or the second filter is inserted into the optical path OP (attenuation step: step S10) to attenuate the amount of near-infrared light or the amount of visible light to adjust the light amount balance of the resulting superimposed image. For example, by rotating the optical filter switching unit 50 and inserting the first filter (optical filter 52) into the optical path OP, the light amount of the near-infrared light is attenuated, and the light amount of the near-infrared light and the visible light are attenuated. Adjust the light intensity balance with the light intensity. After that, the imaging device 1 moves the adjustment lens 16 to focus on the near-infrared image or the visible image (step S11). For example, based on a user's command, the imaging device 1 moves the adjusting lens 16 to focus on the visible image. After that, the imaging device 1 captures a superimposed image of the near-infrared image and the visible image (step S12).

As described above, according to the imaging device 1, when imaging a subject having a high-temperature heat source (for example, 1500° C.) using a near-infrared image and a visible image, the first filter is placed in the optical path OP. It is inserted to attenuate the light quantity of the near-infrared image to adjust the light quantity balance between the light quantity of the visible light and the light quantity of the near-infrared light. Thereby, the imaging device 1 can obtain a superimposed image in which the subject of the visible image and the near-infrared image can be visually recognized well.

In the above example, an example in which the first filter is used to attenuate the amount of near-infrared light is described, but the technology of the present disclosure is not limited to this. For example, when the amount of visible light is strong due to illumination or the like, the amount of near-infrared light radiated from a heat source at 200° C. to 300° C. is relatively low. In such a case, by inserting the second filter in the optical path OP to attenuate the amount of visible light, it is possible to acquire a superimposed image in which the subject can be visually recognized well, with overexposure suppressed.

<Modification 1>
Next, modification 1 will be described. In the imaging apparatus 1 of this example, the control unit 110 replaces the optical filters based on the image acquired by the image sensor 132 to adjust the light amount balance between the near-infrared light amount and the visible light amount.

FIG. 16 is a schematic diagram of the optical filter switching section 50 of this example. It should be noted that FIG. 16 shows the optical filter switching section 50 viewed from the direction AA in FIG. 1, as in FIG.

The optical filter switching unit 50 of this example is a turret switching device in which four

optical filters

62, 64, 66, 68 are arranged on a disc. It should be noted that the provided optical filters are different from those of the optical filter switching section 50 already explained with reference to FIG.

The optical filter 62 is a first filter composed of one light amount adjusting optical element. This first filter can attenuate the amount of near-infrared light by 20 dB.

The optical filter 64 is a first filter composed of two light amount adjusting optical elements. This first filter can attenuate the amount of near-infrared light by 40 dB.

The optical filter 66 is a first filter composed of three light quantity adjusting optical elements. This first filter can attenuate the amount of near-infrared light by 60 dB.

The optical filter 68 is a second filter composed of one light amount adjusting optical element. The second filter can attenuate the amount of visible light by 30 dB.

Thus, the optical filter switching unit 50 included in the imaging device 1 of this example is configured with optical filters that stepwisely attenuate the amount of near-infrared light. In the imaging apparatus 1 of this example, the optical filter switching section 50 is driven by the control section 110 to obtain a superimposed image with the light amount balance adjusted.

FIG. 17 is a flowchart showing the imaging method of the imaging device 1 of this example. Note that the imaging method is performed by the CPU 202 executing an imaging device control program (imaging method execution program) stored in the computer of the imaging device 1 .

First, the control unit 110 of the imaging device 1 inserts the optical filter 62 into the optical path OP (step S20). After that, the imaging device 1 captures a superimposed image of the near-infrared image and the visible image (step S21). After that, the captured superimposed image is input to the image processing unit 126 of the imaging apparatus 1 of this example, and the image processing unit 126 determines whether or not the overexposed area in the superimposed image is equal to or larger than a threshold value (step S22). When the overexposed area is equal to or greater than the threshold, the control unit 110 determines that the attenuation of the near-infrared light amount by the optical filter 62 is insufficient, drives the optical filter switching unit 50, The optical filter 62 is retracted and the optical filter 64 is inserted into the optical path OP. As described above, the optical filter 64 is a first filter configured by stacking two light amount adjusting optical elements, and can attenuate the amount of near-infrared light from the optical filter 62 . Thereafter, the imaging device 1 captures a superimposed image of the near-infrared image and the visible image, and the image processing unit 126 determines whether the overexposed area is equal to or greater than the threshold. In this manner, the imaging apparatus 1 drives the optical filter switching unit 50 to change the optical filter until the overexposed area becomes less than the threshold value.

On the other hand, if the whiteout area in the superimposed image is less than the threshold, the imaging device 1 records the superimposed image as the main captured image (step S23).

As described above, in the imaging device 1 of this example, the size of the overexposed area is determined based on the superimposed image captured by the control unit 110 . Then, based on the determination result, the optical filter switching unit 50 is operated to change the optical filter, thereby adjusting the light amount balance between the near-infrared light amount and the visible light amount in the superimposed image. As a result, it is possible to obtain a superimposed image in which the near-infrared image and the visible image can be satisfactorily visually recognized.

<Modification 2>
Next, modification 2 will be described. The optical filter switching unit 50 of the imaging apparatus 1 of this example is not the turret switching device described with reference to FIGS. filter holding mechanism).

FIG. 18 is a schematic configuration diagram of the imaging device 1 of this example. In addition, the same code|symbol is attached|subjected to the location which already demonstrated in FIG. 1, and description is abbreviate|omitted.

The imaging device 1 shown in FIG. 18 is composed of a filter insertion/extraction mechanism 72 and a filter insertion/extraction mechanism 74 . A first filter is held by the filter insertion/extraction mechanism 72 , and a second filter is held by the filter insertion/extraction mechanism 74 . The first filter held by the filter inserting/removing mechanism 72 is driven by the inserting/removing driving mechanism 70 to move in the direction indicated by the arrow in the figure, and is inserted onto or retracted from the optical path OP. The second filter held by the filter inserting/removing mechanism 74 is driven by the inserting/removing driving mechanism 70 to move in the direction indicated by the arrow in the drawing, and is inserted onto or retracted from the optical path OP. Note that in the imaging apparatus 1 of this example, the insertion/extraction driving mechanism 70 and the insertion/extraction driving section for driving the insertion/extraction driving mechanism 70 are replaced with the turret driving mechanism 22 and the turret driving section 116 described with reference to FIG.

According to the imaging apparatus 1 of this example described above, the first filter and the second filter are held by the filter insertion/removal mechanism 72 and the filter insertion/removal mechanism 74, respectively, so that the first filter and the second filter are held by the turret. The actuation mechanism can be simplified rather than

<Others>
In the above embodiment, the hardware structure of the processing unit (control unit 110) (processing unit) that executes various processes is various processors as shown below. For various processors, the circuit configuration can be changed after manufacturing such as CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, FPGA (Field Programmable Gate Array), etc. Programmable Logic Device (PLD), which is a processor, ASIC (Application Specific Integrated Circuit), etc. be

One processing unit may be composed of one of these various processors, or composed of two or more processors of the same type or different types (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA). may Also, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units in a single processor, first, as represented by a computer such as a client or server, a single processor is configured by combining one or more CPUs and software. There is a form in which a processor functions as multiple processing units. Second, as typified by System On Chip (SoC), etc., there is a form of using a processor that realizes the function of the entire system including multiple processing units with a single IC (Integrated Circuit) chip. be. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

Each configuration and function described above can be appropriately realized by arbitrary hardware, software, or a combination of both. For example, a program that causes a computer to execute the above-described processing steps (procedures), a computer-readable recording medium (non-temporary recording medium) recording such a program, or a computer capable of installing such a program However, it is possible to apply the present invention.

Although the examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and that various modifications are possible without departing from the gist of the present invention.

1: Imaging device 12: Focus lens 14: Zoom lens 16: Adjustment lens 20: Zoom lens drive mechanism 22: Turret drive mechanism 24: Adjustment lens drive mechanism 26: Image display unit 28: Input unit 30: Aperture 50: Optical filter switching Unit 90: housing 100: imaging optical system 110: control unit 112: bus line 114: zoom lens driving unit 116: turret driving unit 118: adjustment lens driving unit 120: output I/F
122: Input I/F
126: Image processing unit 130: Imaging unit 132: Image sensor 132A: Imaging surface 200: Computer 202: CPU
204: RAM
206: ROM
210: Imaging device control program 212: In-focus position data

Claims

An imaging device that captures a near-infrared image and a visible image,
an imaging optical system having transmission characteristics in the wavelength band of near-infrared light and the wavelength band of visible light;
an image sensor that acquires an image by receiving light that has passed through the imaging optical system;
a light amount adjusting optical element that attenuates the light amount of the near-infrared light and/or the light amount of the visible light that is transmitted through the imaging optical system;
with
An imaging apparatus that adjusts the light amount balance between the light amount of the near-infrared light and the light amount of the visible light received by the image sensor by the light amount adjusting optical element.
The imaging apparatus according to claim 1, wherein the light amount adjusting optical element constitutes a first filter that attenuates the near-infrared light and/or a second filter that attenuates the visible light.
The imaging device according to claim 2, wherein the first filter is composed of a plurality of the light amount adjusting optical elements.
The imaging device according to claim 2 or 3, wherein the second filter is composed of a plurality of the light quantity adjusting optical elements.
5. The first filter and/or the second filter according to any one of claims 2 to 4, wherein the light quantity adjusting optical element is held by a filter holding mechanism that holds the light quantity adjusting optical element in an insertable/removable manner in an optical path of the imaging optical system. imaging device.
The imaging device according to claim 5, wherein the filter holding mechanism is configured by a turret having the first filter and/or the second filter.
The imaging device according to any one of claims 4 to 6, wherein the second filter can attenuate the visible light by 30 dB or more.
The imaging device according to any one of claims 1 to 7, wherein the image sensor acquires a superimposed image in which the near-infrared image and the visible image are superimposed.
9. The imaging optical system according to any one of claims 1 to 8, wherein the diameter of the near-infrared light source image of the point light source image when the visible image is in focus is 0.2 mm or more and 0.9 mm or less. The imaging device according to any one of items 1 and 2.
Equipped with a control unit,
The imaging apparatus according to any one of claims 1 to 9, wherein the control section causes the light amount adjusting optical element to adjust the light amount balance based on the image acquired by the image sensor.
An imaging device for capturing a near-infrared image and a visible image, comprising: an imaging optical system having transmission characteristics in a wavelength band of near-infrared light and a wavelength band of visible light; and light transmitted through the imaging optical system. An imaging method for an imaging device comprising an image sensor that receives light and acquires an image,
an attenuation step of attenuating the light amount of the near-infrared light and/or the light amount of the visible light transmitted through the imaging optical system by a light amount adjusting optical element;
including
In the attenuating step, the imaging method adjusts the light amount balance between the light amount of the near-infrared light and the light amount of the visible light received by the image sensor.