WO2022181095A1

WO2022181095A1 - Control device, imaging device, control method, and program

Info

Publication number: WO2022181095A1
Application number: PCT/JP2022/000783
Authority: WO
Inventors: 哲也藤川; 智大島田; 臣一下津; 敏浩青井
Original assignee: 富士フイルム株式会社
Priority date: 2021-02-26
Filing date: 2022-01-12
Publication date: 2022-09-01
Also published as: JPWO2022181095A1

Abstract

This control device comprises: a processor; and a memory which is connected to or built into the processor. The processor is provided with: a first mode in which an image is captured on the basis of light received by an image sensor of an imaging device; and a second mode in which the temperature is derived on the basis of near-infrared light received by the image sensor. Different control factors are used in the first mode and in the second mode.

Description

Control device, imaging device, control method, and program

The technology of the present disclosure relates to a control device, an imaging device, a control method, and a program.

Japanese Patent Application Laid-Open No. 10-134272 describes a monitoring system for large-space disaster prevention, etc. for detecting fire outbreaks and intruders from thermal images of a large space, and for environmental control. an infrared camera for taking out, a temperature filter selection means having a plurality of temperature filters with different temperature ranges from room temperature to a temperature range necessary for fire monitoring, and selecting a temperature filter for the infrared camera from the plurality of temperature filters, and an infrared camera image data storage means for storing thermal image data extracted from and processed, and temperature filter selection means for selecting a temperature filter of the infrared camera in accordance with preset switching timings of a plurality of monitoring modes, and selecting a monitoring mode. A monitoring system for large space disaster prevention and the like is disclosed, which is characterized by comprising control processing means for taking out thermal image data from an infrared camera each time and executing processing in each monitoring mode.

International Publication No. 2005/71372 pamphlet describes a photographing optical system for forming an image of a subject, an image sensor section for photographing the subject image formed by the photographing optical system and outputting an image signal, and an image capturing operation unit for performing operations related to image capturing, and an image capturing unit configured to acquire a spectroscopic image, wherein the image processing system acquires An image processing system is disclosed, further comprising mode display means for displaying mode-related information corresponding to each of the plurality of modes to be obtained.

Japanese Patent Application Publication No. 2004-534941 discloses a handheld infrared camera including a lens assembly, the lens assembly being supported by a housing, the housing being a source of electrical energy and information received via the lens assembly. and wherein the housing is provided with user control means for manually and visually controlling the device, wherein the housing is in a substantially elongated configuration. with a lens assembly mounted at one end, a user handle at the opposite end, and a manual handle on one side of the housing intended to be operated by the user's thumb. A control assembly and a visual control are provided, the visual control being positioned between the manual control assembly and the lens assembly to position the infrared camera away from the user's eyes and body. Disclosed is a hand-held infrared camera adapted to be viewed when holding a camera and characterized in that the infrared camera is intended to be operated with one hand.

One embodiment according to the technology of the present disclosure provides a control device, an imaging device, a control method, and a program that are capable of differentiating the details of control on a controlled object between the first mode and the second mode.

A first aspect of the technology of the present disclosure includes a processor and a memory connected to or built into the processor, and the processor performs imaging based on light received by an image sensor of an imaging device in a first mode and , and a second mode for deriving temperature based on the near-infrared light received by the image sensor, the control factor being different between the first mode and the second mode.

A second aspect of the technology of the present disclosure is the control device according to the first aspect, wherein the control factor includes a display control factor to be displayed on the display.

A third aspect of the technology of the present disclosure is the control device according to the second aspect, wherein in the first mode, the processor, as the display control factor, is a captured image obtained by light being received by the image sensor. is displayed on the display, and in the second mode, as the display control factor, a temperature information display factor is set for displaying temperature information indicating temperature on the display.

A fourth aspect of the technology of the present disclosure is the control device according to any one of the first to third aspects, wherein the control factor includes a light projection control factor that operates the light projector, and the processor comprises: In the second mode, the controller sets a light projection suppression control factor for suppressing the light projection of the light projector as the light projection control factor.

A fifth aspect of the technology of the present disclosure is the control device according to any one of the first to fourth aspects, wherein the control factor includes an imaging setting factor related to imaging settings.

A sixth aspect of the technology of the present disclosure is the control device according to the fifth aspect, wherein the imaging settings include settings related to the projector, settings related to the shutter speed, settings related to the aperture, settings related to photometry, settings related to the sensitivity of the image sensor, The control device includes at least one setting of a setting related to high dynamic range and a setting related to anti-vibration control.

A seventh aspect of the technology of the present disclosure is the control device according to any one of the first to sixth aspects, wherein the control factor includes an image processing setting factor related to image processing setting. be.

An eighth aspect of the technology of the present disclosure is the control device according to the seventh aspect, wherein the image processing settings include at least one of noise reduction settings, sharpness settings, contrast settings, and tone settings. A controller containing settings.

A ninth aspect of the technology of the present disclosure includes a processor and a memory connected to or built into the processor, wherein the processor performs imaging based on light received by an image sensor of an imaging device in a first mode and and a second mode for deriving a temperature based on the near-infrared light received by the image sensor, setting the first mode when the light projector state is on and the light projector state is off. In the case, it is a control device for setting the second mode.

A tenth aspect of the technology of the present disclosure is the control device according to the ninth aspect, wherein the processor switches from the second mode to the first mode in accordance with the temperature in the second mode.

An eleventh aspect of the technology of the present disclosure is the control device according to the tenth aspect, wherein the light projector performs pulsed light emission, the processor sets the first mode during the light emission period of the pulsed light emission, and emits the pulsed light emission. The control device repeats the operation of setting the second mode during the stop period according to the light emission timing of the pulse light emission.

A twelfth aspect of the technology of the present disclosure is the control device according to any one of the first to eleventh aspects, wherein the processor comprises: It is a control device that outputs a synthesized image obtained by synthesizing an image and temperature information indicating temperature.

A thirteenth aspect of the technology of the present disclosure is an imaging device including the control device according to any one of the first to twelfth aspects, and an image sensor.

A fourteenth aspect of the technology of the present disclosure includes a first mode of imaging based on light received by an image sensor of an imaging device, and a first mode of deriving temperature based on near-infrared light received by the image sensor. The control method includes switching between two modes and different control factors between the first mode and the second mode.

A fifteenth aspect of the technology of the present disclosure includes a first mode of imaging based on light received by an image sensor of an imaging device, and a first mode of deriving temperature based on near-infrared light received by the image sensor. and setting the first mode when the operation of the light projector is on and setting the second mode when the operation of the light projector is off. be.

A sixteenth aspect of the technology of the present disclosure provides a computer with a first mode for capturing an image based on light received by an image sensor of an imaging device, and calculating a temperature based on near-infrared light received by the image sensor. It is a program for executing processing including switching between a second mode to be derived and different control factors between the first mode and the second mode.

A seventeenth aspect of the technology of the present disclosure provides a computer with a first mode for capturing an image based on light received by an image sensor of an image capturing device, and measuring a temperature based on near-infrared light received by the image sensor. and switching to and from a second mode to derive, and setting the first mode when operation of the light projector is on and setting the second mode when operation of the light projector is off. It is a program for executing processing.

1 is a perspective view showing an example of a camera according to a first embodiment; FIG. 2 is a block diagram showing an example of the internal configuration of the camera according to the first embodiment; FIG. 2 is a block diagram showing an example of the electrical configuration of the camera according to the first embodiment; FIG. FIG. 4 is an explanatory diagram showing an example of the configuration and operation of the turret filter according to the first embodiment; 3 is a block diagram showing an example of a functional configuration of a CPU according to the first embodiment; FIG. It is a block diagram showing an example of a configuration as a mode switching processing unit of the CPU according to the first embodiment. 3 is a block diagram showing an example of a configuration of a CPU as an imaging processing unit according to the first embodiment; FIG. FIG. 4 is a front view showing an example of a captured image obtained by imaging processing according to the first embodiment; It is a block diagram showing an example of a configuration of a CPU as a temperature measurement processing unit according to the first embodiment. FIG. 4 is an explanatory diagram showing an example of a function of a CPU as a wavelength selector according to the first embodiment; FIG. 4 is a front view showing a first example of a synthesized image obtained by temperature measurement processing according to the first embodiment; FIG. 11 is a front view showing a second example of a synthesized image obtained by temperature measurement processing according to the first embodiment; FIG. 11 is a front view showing a third example of a synthesized image obtained by temperature measurement processing according to the first embodiment; FIG. 11 is a front view showing a fourth example of a synthesized image obtained by temperature measurement processing according to the first embodiment; 6 is a flowchart showing an example of the flow of mode switching processing according to the first embodiment; 4 is a flowchart showing an example of the flow of imaging processing according to the first embodiment; 6 is a flowchart showing an example of the flow of temperature measurement processing according to the first embodiment; FIG. 11 is a front view showing a modified example of the captured image obtained by the imaging process according to the first embodiment; FIG. 11 is a front view showing a modified example of the synthesized image obtained by the temperature measurement process according to the first embodiment; FIG. 11 is a block diagram showing an example of a configuration of a CPU as a mode switching processing unit according to the second embodiment; 9 is a flowchart showing an example of the flow of mode switching processing according to the second embodiment; FIG. 12 is a block diagram showing an example of a configuration of a CPU as a parameter change processing unit according to the third embodiment; FIG. 12 is a block diagram showing an example of a configuration of a CPU as an imaging processing unit according to the third embodiment; FIG. 12 is a block diagram showing an example of a configuration of a CPU as a temperature measurement processing unit according to the third embodiment; FIG. 11 is a flow chart showing an example of the flow of parameter change processing according to the third embodiment; FIG. FIG. 14 is a block diagram showing an example of a configuration of a CPU as a mode switching processing unit according to the fourth embodiment; FIG. 16 is a flowchart showing an example of the flow of mode switching processing according to the fourth embodiment; FIG. FIG. 20 is a block diagram showing an example of a configuration of a CPU as an integrated display processing unit according to the fifth embodiment; FIG. 14 is a flowchart showing an example of the flow of integrated display processing according to the fifth embodiment; FIG. FIG. 11 is a block diagram showing an example of an electrical configuration of an imaging device according to a first modified example; FIG. 11 is a block diagram showing an example of an electrical configuration of an imaging device according to a second modified example;

An example of embodiments of a control device, an imaging device, a control method, and a program according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

First, the wording used in the following explanation will be explained.

　CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". NVM is an abbreviation for "Non-Volatile Memory". RAM is an abbreviation for "Random Access Memory". IC is an abbreviation for "Integrated Circuit". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip." SSD is an abbreviation for "Solid State Drive". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". SRAM is an abbreviation for "Static Random Access Memory". I/F is an abbreviation for "Interface". USB is an abbreviation for "Universal Serial Bus". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network". BPF is an abbreviation for "Band Pass Filter". Ir is an abbreviation for "Infrared Rays". EL is an abbreviation for "Electro Luminescence".

In the description of this specification, "perpendicular" means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to perfect verticality, and does not go against the spirit of the technology of the present disclosure. It refers to the vertical in the sense of including the error of In the description of this specification, "horizontal" means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to being completely horizontal, and is not contrary to the spirit of the technology of the present disclosure. It refers to the horizontal in the sense of including the error of In the description of this specification, "parallel" means, in addition to complete parallelism, an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, and does not go against the spirit of the technology of the present disclosure. It refers to parallel in the sense of including the error of In the description of this specification, "orthogonal" is an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to perfect orthogonality, and is not contrary to the spirit of the technology of the present disclosure. It refers to orthogonality in the sense of including the error of In the description of this specification, "match" means, in addition to perfect match, an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, and does not go against the spirit of the technology of the present disclosure. It refers to a match in terms of meaning including errors in In the description of this specification, the term “equidistant interval” means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to perfectly equal intervals, and is contrary to the spirit of the technology of the present disclosure. It refers to equal intervals in the sense of including errors to the extent that they do not occur.

[First embodiment]
First, the first embodiment will be described. As shown in FIG. 1 as an example, camera 1 includes camera body 10 and lens unit 20 . The camera 1 has a function of obtaining a visible light image by capturing visible light, a function of obtaining a near-infrared light image by capturing near-infrared light, and a function of capturing a subject based on electromagnetic waves emitted by thermal radiation from the subject. and a function to measure the temperature of The camera 1 is an example of an “imaging device” according to the technology of the present disclosure.

A camera-side mount 12 for attaching the lens unit 20 is provided on the front surface 11 of the camera body 10 . An illumination window 13 is provided on the front surface 11 of the camera body 10 for illuminating the subject with the illumination light IL.

The camera body 10 includes a projector 14 that generates illumination light IL. The light projector 14 is, for example, an LED that emits near-infrared light with a peak wavelength of 1550 nm as illumination light IL. The light projector 14 may be a halogen light. Illumination light IL generated by the light projector 14 is transmitted through the irradiation window 13 and emitted forward of the camera body 10 . The light projector 14 is an example of a "light projector" according to the technology of the present disclosure.

The camera body 10 also includes an image sensor 15 . The image sensor 15 captures an image of the light L incident from the subject through the lens unit 20 . The image sensor 15 has a light receiving surface 15A. The light L incident on the lens unit 20 is imaged on the light receiving surface 15A by the lens unit 20. As shown in FIG. An image is obtained by forming an image of the light L on the light receiving surface 15A. A plurality of photodiodes are arranged in a matrix on the light receiving surface 15A.

As an example, the plurality of photodiodes includes a plurality of silicon photodiodes sensitive to visible light and a plurality of indium-gallium-arsenide photodiodes sensitive to near-infrared light. Hereinafter, the silicon photodiode will be referred to as a Si diode, and the indium-gallium-arsenide photodiode will be referred to as an InGaAs diode. A plurality of Si diodes generate and output analog image data according to the received visible light. A plurality of InGaAs diodes generate and output analog image data corresponding to the received near-infrared light. Hereinafter, visible light and near-infrared light incident on the image sensor 15 will be referred to as light unless it is necessary to distinguish between them.

In the first embodiment, a CMOS image sensor is exemplified as the image sensor 15, but the technology of the present disclosure is not limited to this. The technology of the present disclosure is also established. The image sensor 15 is an example of an “imaging device image sensor” according to the technology of the present disclosure.

The lens unit 20 includes a lens barrel 21 and a lens side mount 22. The lens side mount 22 is provided at the rear end of the lens barrel 21 . The lens side mount 22 is configured to be connectable to the camera side mount 12 of the camera body 10 . The lens unit 20 is detachably attached to the camera body 10 by a lens side mount 22 . Note that the lens unit 20 may be fixed to the camera body 10 in a non-detachable manner.

As shown in FIG. 2 as an example, the lens unit 20 includes an objective lens 30, a focus lens 31, a zoom lens 32, an aperture 33, a blur correction lens 34, a turret filter 35, and an adjustment lens 37. An objective lens 30, a focus lens 31, a zoom lens 32, an aperture 33, a blur correction lens 34, a turret filter 35, and an adjustment lens 37 are arranged in order from the object side to the image side along the optical axis OA of the lens unit 20. there is

The objective lens 30 is fixed to the tip of the lens barrel 21 and is a lens that collects light. The focus lens 31 is a lens for adjusting the focus position of the image. The zoom lens 32 is a lens for adjusting zoom magnification.

The diaphragm 33 is an optical element for adjusting the amount of light. The diaphragm 33 has an aperture 33A. Light guided by zoom lens 32 passes through aperture 33A. The diaphragm 33 is a movable diaphragm in which the diameter of the aperture 33A is variable. The amount of light directed by zoom lens 32 is modified by aperture 33 . The blur correction lens 34 is a lens for correcting image blur.

The turret filter 35 has a plurality of optical filters. The turret filter 35 selects the optical filter inserted in the optical path of the light in the lens unit 20 among the plurality of optical filters, thereby filtering out light in a plurality of wavelength bands (eg, visible light, It is an optical element that selectively transmits (near-infrared light in different wavelength bands within the near-infrared wavelength band). The optical path of light within the lens unit 20 is positioned, for example, on the optical axis OA. Hereinafter, the optical path of light within the lens unit 20 is simply referred to as an optical path. The configuration of the turret filter 35 will be detailed later with reference to FIG.

The adjustment lens 37 is a lens for adjusting the difference in focal length when the plurality of optical filters included in the turret filter 35 are switched.

The order of arrangement of the focus lens 31, zoom lens 32, diaphragm 33, blur correction lens 34, turret filter 35, and adjustment lens 37 may be other than the above. Each of the objective lens 30, the focus lens 31, the zoom lens 32, the blur correction lens 34, and the adjusting lens 37 may be a single lens, or may be a lens group having a plurality of lenses. In addition to the focus lens 31, zoom lens 32, blur correction lens 34, and adjustment lens 37, the lens unit 20 may include other lenses. Also, the lens unit 20 may include an optical element such as a half mirror or a polarizing element.

As shown in FIG. 2 as an example, the lens unit 20 includes a zoom drive mechanism 42, an aperture drive mechanism 43, a blur correction drive mechanism 44, a turret drive mechanism 45, and an adjustment drive mechanism 47. The zoom drive mechanism 42, the aperture drive mechanism 43, the blur correction drive mechanism 44, the turret drive mechanism 45, and the adjustment drive mechanism 47 are electrically connected to an electrical contact 38 provided at the rear end of the lens barrel 21. .

The camera body 10 includes a control circuit 50. The control circuit 50 is electrically connected to electrical contacts 58 provided on the camera-side mount 12 . When the lens side mount 22 is connected to the camera side mount 12 and the lens unit 20 is attached to the camera body 10, the electrical contact 38 is connected to the electrical contact 58, and the control circuit 50 operates the zoom drive mechanism 42 and the aperture drive mechanism. 43 , the blur correction drive mechanism 44 , the turret drive mechanism 45 and the adjustment drive mechanism 47 are electrically connected.

The zoom drive mechanism 42, the diaphragm drive mechanism 43, the blur correction drive mechanism 44, the turret drive mechanism 45, and the adjustment drive mechanism 47 are all drive mechanisms including actuators such as motors.

As shown in FIG. 3 as an example, the control circuit 50 includes a computer 60, a zoom drive circuit 52, an aperture drive circuit 53, a blur correction drive circuit 54, a turret drive circuit 55, and an adjustment drive circuit 57. The zoom drive circuit 52 , the aperture drive circuit 53 , the blur correction drive circuit 54 , the turret drive circuit 55 and the adjustment drive circuit 57 are connected to the computer 60 via the input/output I/F 59 .

The computer 60 includes a CPU 61, NVM 62, and RAM 63. The CPU 61 , NVM 62 and RAM 63 are interconnected via a bus 64 , and the bus 64 is connected to the input/output I/F 59 .

The NVM 62 is a non-temporary storage medium and stores various parameters and various programs. For example, NVM 62 is an EEPROM. However, this is merely an example, and an HDD and/or an SSD may be applied as the NVM 62 instead of or together with the EEPROM. The RAM 63 temporarily stores various information and is used as a work memory. The CPU 61 reads necessary programs from the NVM 62 and executes the read programs in the RAM 63 . The CPU 61 controls the entire camera 1 according to programs executed on the RAM 63 .

The CPU 61 is an example of a "processor" according to the technology of the present disclosure, and the RAM 63 is an example of a "memory" according to the technology of the present disclosure. Also, the computer 60 is an example of a “control device” according to the technology of the present disclosure.

The zoom drive circuit 52 adjusts the positions of the focus lens 31 and the zoom lens 32 by driving the zoom drive mechanism 42 according to instructions from the computer 60 . The focus lens 31 and the zoom lens 32 move along the optical axis OA of the lens unit 20 by applying power from the zoom drive mechanism 42 .

The aperture drive circuit 53 changes the diameter of the aperture 33A (see FIG. 2) provided in the aperture 33 by driving the aperture drive mechanism 43 according to instructions from the computer 60.

The blur correction drive circuit 54 adjusts the position of the blur correction lens 34 by driving the blur correction drive mechanism 44 according to instructions from the computer 60 and a feedback signal output from a feedback circuit 75, which will be described later. The blur correction lens 34 moves along a plane perpendicular to the optical axis OA of the lens unit 20 by applying power from the blur correction driving mechanism 44 . Specifically, the blur correction lens 34 moves in a direction in which blurring of an image obtained by forming an image of light on the image sensor 15 is corrected.

The turret drive circuit 55 adjusts the position of the turret filter 35 in the rotational direction by driving the turret drive mechanism 45 according to instructions from the computer 60 . The turret filter 35 rotates along a plane perpendicular to the optical axis OA of the lens unit 20 by applying power from the turret driving mechanism 45 . The rotation operation of the turret filter 35 will be detailed later with reference to FIG.

The adjustment drive circuit 57 adjusts the position of the adjustment lens 37 by driving the adjustment drive mechanism 47 according to instructions from the computer 60 . The adjustment lens 37 is moved along the optical axis OA of the lens unit 20 by applying power from the adjustment drive mechanism 47 .

As shown in FIG. 3 as an example, the camera body 10 includes an image sensor driver 71, a signal processing circuit 72, a light projection control circuit 73, a vibration sensor 74, a feedback circuit 75, a display 76, a display control circuit 77, an input device 78, An input circuit 79 and an external I/F 80 are provided. Image sensor driver 71, signal processing circuit 72, light projection control circuit 73, feedback circuit 75, display control circuit 77, input circuit 79, and external I/F 80 are connected to computer 60 via input/output I/F 59. there is

The image sensor driver 71 causes the image sensor 15 to capture light according to instructions from the computer 60 . The signal processing circuit 72 performs various signal processing on the analog image data output from the image sensor 15 to generate and output digital image data.

The light projection control circuit 73 switches the light projector 14 on and off according to instructions from the computer 60 . The light projector 14 outputs illumination light when switched on, and stops outputting illumination light when switched off.

The vibration sensor 74 is, for example, a gyro sensor, and detects vibration of the camera 1. A gyro sensor included in the vibration sensor 74 detects vibrations of the camera 1 around the pitch axis and the yaw axis. The vibration sensor 74 converts vibrations about the pitch axis and the vibrations about the yaw axis detected by the gyro sensor into vibrations in a two-dimensional plane parallel to the pitch axis and the yaw axis. to detect vibration acting in the direction of the pitch axis and vibration acting in the direction of the yaw axis. The vibration sensor 74 outputs a vibration detection signal corresponding to the detected vibration.

Note that the vibration sensor 74 may be an acceleration sensor. Also, instead of the vibration sensor 74, for example, a motion vector obtained by comparing successive captured images stored in the NVM 62 and/or the RAM 63 may be used as vibration. Also, the final used vibration may be derived based on the vibration detected by the physical sensor and the motion vector obtained by image processing.

The feedback circuit 75 generates a feedback signal by performing various signal processing on the vibration detection signal output from the vibration sensor 74 . The feedback circuit 75 is connected to the blur correction drive circuit 54 via the input/output I/F 59 and outputs a feedback signal to the blur correction drive circuit 54 according to instructions from the computer 60 .

The display 76 is, for example, a liquid crystal display or an EL display, and displays images and/or character information. The display control circuit 77 causes the display 76 to display an image according to instructions from the computer 60 .

The input device 78 is, for example, a device such as a touch panel and/or a switch, and receives instructions given by the user. The input circuit 79 outputs an input signal according to an instruction given to the input device 78 by the user. The external I/F 80 is an interface communicably connected to an external device.

As shown in FIG. 4 as an example, the turret filter 35 has a disc 81 . The disc 81 is provided with an Ir cut filter 82, a first BPF 83A, a second BPF 83B, a third BPF 83C, and a fourth BPF 83D as a plurality of optical filters at equal intervals along the circumferential direction of the disc 81. Hereinafter, the Ir cut filter 82, the first BPF 83A, the second BPF 83B, the third BPF 83C, and the fourth BPF 83D are referred to as optical filters unless they need to be distinguished and described. Further, hereinafter, the first BPF 83A, the second BPF 83B, the third BPF 83C, and the fourth BPF 83D will be referred to as BPFs 83 unless they need to be distinguished and described.

The turret filter 35 selectively inserts and removes a plurality of optical filters with respect to the optical path in a turret system. Specifically, by rotating the turret filter 35 in the direction of the arc arrow R shown in FIG. (upper optical path). When the optical filter is inserted into the optical path, the optical axis OA passes through the center of the optical filter, and the center of the optical filter inserted into the optical path coincides with the center of the light receiving surface of the image sensor 15 .

The Ir cut filter 82 is an optical filter that cuts infrared rays and transmits only light other than infrared rays. The BPF 83 is an optical filter that transmits near-infrared light. The first BPF 83A, the second BPF 83B, the third BPF 83C, and the fourth BPF 83D transmit near-infrared light in different wavelength bands.

The first BPF 83A is an optical filter that corresponds to a wavelength band near 1000 nm (nanometers). As an example, the first BPF 83A transmits only near-infrared light in the wavelength band from 950 nm to 1100 nm. The near-infrared light transmitted through the first BPF 83A is hereinafter referred to as first near-infrared light.

The second BPF 83B is an optical filter corresponding to a wavelength band near 1250 nm. As an example, the second BPF 83B transmits only near-infrared light in the wavelength band from 1150 nm to 1350 nm. The near-infrared light transmitted through the second BPF 83B is hereinafter referred to as second near-infrared light.

The third BPF 83C is an optical filter that corresponds to a wavelength band near 1550 nm. As an example, the third BPF 83C transmits only near-infrared light in the wavelength band from 1500 nm to 1750 nm. The near-infrared light transmitted through the third BPF 83C is hereinafter referred to as third near-infrared light.

The fourth BPF 83D is an optical filter corresponding to a wavelength band near 2150 nm. As an example, the fourth BPF 83D transmits only near-infrared light in the wavelength band from 2000 nm to 2400 nm. The near-infrared light transmitted through the fourth BPF 83D is hereinafter referred to as fourth near-infrared light.

Hereinafter, the first near-infrared light, the second near-infrared light, the third near-infrared light, and the fourth near-infrared light are referred to as near-infrared light unless it is necessary to distinguish them. Note that each band mentioned here includes an error that is generally allowed in the technical field to which the technology of the present disclosure belongs and that does not deviate from the gist of the technology of the present disclosure. Further, each wavelength band mentioned here is merely an example, and different wavelength bands may be used.

When the Ir cut filter 82 is inserted into the optical path and the visible light transmitted through the Ir cut filter 82 is imaged on the light receiving surface of the image sensor 15, a plurality of Si diodes arranged on the light receiving surface cut the received visible light. Analog image data obtained by imaging is output. This realizes a function of obtaining a visible light image by imaging visible light. Further, when the BPF 83 is inserted into the optical path and the near-infrared light transmitted through the BPF 83 is imaged on the light receiving surface of the image sensor 15, a plurality of InGaAs diodes arranged on the light receiving surface captures the received near-infrared light. The analog image data obtained by doing is output. This realizes a function of obtaining a near-infrared light image by capturing near-infrared light.

As described above, the camera 1 has a function of obtaining a visible light image by capturing visible light and a function of obtaining a near-infrared light image by capturing near-infrared light. In addition to the function of obtaining a visible light image and the function of obtaining a near-infrared light image, the camera 1 also has a function of measuring the temperature of a subject based on electromagnetic waves emitted by thermal radiation from the subject. In the first embodiment, two-color thermometry is used to improve measurement accuracy. In order to realize temperature measurement by two-color thermometry, for example, it is required to capture light in two different wavelength bands. In the first embodiment, the turret filter 35 is used as one means for generating light in two different wavelength bands, and as an example, near-infrared light in two wavelength bands is used for temperature measurement.

As shown in FIG. 5 as an example, the imaging support processing is realized by executing the imaging support processing program 100 by the CPU 61 . The imaging support processing program 100 is an example of a “program” according to the technology of the present disclosure. In the example shown in FIG. 5 , the imaging support processing program 100 is stored in the NVM 62 , and the CPU 61 reads the imaging support processing program 100 from the NVM 62 and executes it on the RAM 63 .

The CPU 61 performs imaging support processing according to the imaging support processing program 100 executed on the RAM 63 . The CPU 61 functions as a mode switching processing section 110 , an imaging processing section 120 and a temperature measurement processing section 130 by executing the imaging support processing program 100 on the RAM 63 .

The CPU 61 has an imaging mode and a temperature measurement mode as operation modes, and switches between the imaging mode and the temperature measurement mode. The imaging processing unit 120 is a processing unit that operates when the operation mode of the CPU 61 is switched to the imaging mode. The imaging processing unit 120 displays a visible light image obtained by imaging visible light with the image sensor 15 or a near-infrared light image obtained by imaging near-infrared light with the image sensor 15. 76 is a processing unit that executes imaging processing to be displayed on 76 .

The temperature measurement processing unit 130 is a processing unit that operates when the operation mode of the CPU 61 is switched to the temperature measurement mode. The temperature measurement processing unit 130 calculates the temperature distribution of the subject based on the near-infrared light image obtained by capturing the near-infrared light by the image sensor 15, and calculates the temperature distribution generated based on the temperature distribution of the subject. It is a processing unit that executes temperature measurement processing for displaying information on the display 76 .

The mode switching processing unit 110 is a processing unit that executes mode switching processing for switching the operation mode of the CPU 61 between the imaging mode and the temperature measurement mode. The mode switching processing unit 110, the imaging processing unit 120, and the temperature measurement processing unit 130 will be described in order below.

The mode switching processing unit 110 has a mode selection information acquisition unit 111 , a mode determination unit 112 , a flag setting unit 113 , a light projection control unit 114 and a mode setting unit 115 .

As an example, as shown in FIG. 6, the mode selection information acquisition unit 111 obtains imaging mode selection information for selecting the imaging mode as the operation mode and temperature measurement mode for selecting the temperature measurement mode as the operation mode, for example, via the reception device. Selectively acquire mode selection information. The receiving device receives various information and outputs the received various information to the CPU 61 . Examples of receiving devices include the input circuit 79 and the external I/F 80 . For convenience of explanation, the imaging mode selection information and the temperature measurement mode selection information are hereinafter referred to as mode selection information when there is no need to distinguish between them.

Various instructions from the user are input to the input device 78 . The input circuit 79 outputs information to the CPU 61 according to the instruction input to the input device 78 . The input circuit 79 outputs mode selection information to the CPU 61 according to a mode selection instruction given to the input device 78 by the user. The mode selection information acquisition unit 111 acquires mode selection information input from the input circuit 79 . The external I/F 80 receives mode selection information output from an external device (not shown) and outputs the received mode selection information to the CPU 61 . Mode selection information acquisition unit 111 acquires mode selection information input from external I/F 80 .

The mode determination unit 112 determines whether the operation mode selected by the mode selection information acquired by the mode selection information acquisition unit 111 is the imaging mode or the temperature measurement mode. When the mode selection information acquired by the mode selection information acquisition unit 111 is the imaging mode selection information, the mode determination unit 112 determines that the operation mode selected by the mode selection information is the imaging mode. Further, when the mode selection information acquired by the mode selection information acquisition unit 111 is the temperature measurement mode selection information, the mode determination unit 112 determines that the operation mode selected by the mode selection information is the temperature measurement mode. do.

A display control flag storage area 141 and a light projection control flag storage area 142 are provided in the RAM 63 . The display control flag storage area 141 stores a display control flag 151 that designates an image to be displayed on the display 76 . The light projection control flag storage area 142 stores a light projection control flag 152 that designates the operation of the light projector 14 .

When the mode determination unit 112 determines that the operation mode is the imaging mode, the flag setting unit 113 sets the captured image display flag 151A as the display control flag 151 in the display control flag storage area 141 and A light projection ON control flag 152 A is set as the light projection control flag 152 in the light control flag storage area 142 . Further, when the mode determination unit 112 determines that the operation mode is the temperature measurement mode, the flag setting unit 113 sets the composite image display flag 151B as the display control flag 151 in the display control flag storage area 141, A light projection OFF control flag 152 B is set as the light projection control flag 152 in the light projection control flag storage area 142 . Hereinafter, the display control flag 151 and the light projection control flag 152 will be referred to as control flags unless they need to be distinguished and described.

The light projection control unit 114 outputs an ON command to the light projection control circuit 73 when the light projection ON control flag 152A is set by the flag setting unit 113 . The ON command is a command to switch the projector 14 ON. Further, when the flag setting unit 113 sets the light projection OFF control flag 152B, the light projection control unit 114 outputs an OFF command to the light projection control circuit 73 . The OFF command is a command to switch the projector 14 off. Note that "on" refers to a setting in which the light projector 14 projects light, and "off" refers to a setting in which the light projector 14 does not perform light projection.

The mode setting unit 115 sets the imaging mode as the operation mode of the CPU 61 when the captured image display flag 151A is set as the display control flag 151 . Further, when the composite image display flag 151B is set as the display control flag 151, the mode setting unit 115 sets the temperature measurement mode as the operation mode of the CPU 61. FIG.

In the first embodiment, the imaging mode is an example of the "first mode" according to the technology of the present disclosure, and the temperature measurement mode is an example of the "second mode" according to the technology of the present disclosure. Also, the display control flag 151 and the light projection control flag 152 are examples of the “control factor” according to the technique of the present disclosure. In addition, the display 76 and the light projector 14 are examples of the “controlled object” according to the technology of the present disclosure, and the image displayed on the display 76 and the operation of the light projector 14 are the same as the “controlled object” according to the technology of the present disclosure. This is an example of "content". Further, the display control flag 151 is an example of the "display control factor" according to the technology of the present disclosure, the captured image display flag 151A is an example of the "captured image display factor" according to the technology of the present disclosure, and the composite image The display flag 151B is an example of a "temperature information display factor" according to the technology of the present disclosure. Further, the light projection ON control flag 152A and the light projection OFF control flag 152B are examples of the “light projection control factor” according to the technology of the present disclosure, and the light projection ON control flag 152A is an example of the “projection control factor” according to the technology of the present disclosure. The light emission OFF control flag 152B is an example of the "light emission suppression control factor" according to the technology of the present disclosure.

As shown in FIG. 7 as an example, the imaging processing unit 120 has a wavelength selection unit 121, a turret control unit 122, an imaging control unit 123, and a display control unit .

The wavelength selection unit 121 selects one wavelength band used for imaging from a plurality of wavelength bands according to the wavelength selection instruction accepted by the input device 78 . As an example, the wavelength selection unit 121 selects the visible light wavelength band, the first near-infrared light wavelength band from 950 nm to 1100 nm, the second near-infrared light wavelength band from 1150 nm to 1350 nm, and the third near-infrared light wavelength band from 1500 nm to 1750 nm. One wavelength band is selected from the infrared wavelength band and the fourth near-infrared wavelength band from 2000 nm to 2400 nm. Here, an example is given in which the wavelength band is selected according to the wavelength selection instruction received by the input device 78, but the wavelength can be selected according to various conditions (for example, subject temperature and/or imaging conditions). A band may be selected.

The turret control unit 122 outputs to the turret drive circuit 55 a rotation command for inserting into the optical path an optical filter corresponding to the wavelength band selected by the wavelength selection unit 121 among the plurality of optical filters. Upon receiving the rotation command, the turret drive circuit 55 drives the turret drive mechanism 45 to rotate the turret filter 35 to a position where the optical filter corresponding to the rotation command is inserted into the optical path.

The imaging control unit 123 outputs an imaging command to the image sensor driver 71 . The imaging command is a command to cause the image sensor 15 to capture light. The image sensor 15 captures light emitted from a subject and outputs analog image data obtained by capturing the light. The signal processing circuit 72 performs various signal processing on the analog image data output from the image sensor 15 to generate and output digital image data.

The display control unit 124 causes the display 76 to display the captured image 161 A based on the digital image data generated by the signal processing circuit 72 . For example, the captured image 161A is displayed as a moving image, but may be displayed as a still image.

FIG. 8 shows an example of a captured image 161A displayed on the display 76 as a result of the imaging processing being executed by the imaging processing unit 120. As shown in FIG. In the example shown in FIG. 8, the display 76 displays a captured image 161A in which a curtain 163 provided on the indoor side of a window 162 of a building is on fire 164 .

As an example, as shown in FIG. 9, the temperature measurement processing unit 130 includes a wavelength selection unit 131, a first turret control unit 132, a first imaging control unit 133, a second turret control unit 134, a second imaging control unit 135, a temperature It has a derivation unit 136 and a display control unit 137 .

As an example, as shown in FIG. 10, the wavelength selection unit 131 selects two wavelength bands, that is, a first wavelength band and a second wavelength band, used for dichroic thermometry. As an example, the wavelength selection unit 131 uses a first near-infrared light wavelength band from 950 nm to 1100 nm, a second near-infrared light wavelength band from 1150 nm to 1350 nm, and a wavelength band from 1500 nm to Two wavelength bands are selected from the third near-infrared wavelength band of 1750 nm and the fourth near-infrared wavelength band of 2000 nm to 2400 nm. Further, as an example, the wavelength selection unit 131 uses the wavelength band of the first near-infrared light, the wavelength band of the second near-infrared light, and the wavelength of the third near-infrared light as the first wavelength band and the second wavelength band. and two adjacent wavelength bands from among the wavelength bands of the fourth near-infrared light.

Further, as an example, the wavelength selection unit 131 uses the wavelength band of the first near-infrared light, the wavelength band of the second near-infrared light, and the wavelength of the third near-infrared light as the first wavelength band and the second wavelength band. and the wavelength band of the fourth near-infrared light, two wavelength bands are selected based on the temperature of the subject. Further, as an example, the wavelength selection unit 131 uses the wavelength band of the first near-infrared light, the wavelength band of the second near-infrared light, and the wavelength of the third near-infrared light as the first wavelength band and the second wavelength band. A shorter wavelength band is selected from the band and the wavelength band of the fourth near-infrared light as the temperature of the object increases.

For example, in the case of a fire, the temperature of the subject is predicted based on information about the temperature expected from the fire situation and/or information input to the input device 78 by the user. Information about the temperature expected from the fire situation is obtained through the external I/F 80 shown in FIG. The information on the temperature expected from the fire situation may be information on the standard fire temperature with respect to the elapsed time from the occurrence of the fire. Standard fire temperature is defined in ISO834. Note that the wavelength selection unit 131 may switch the wavelength band according to the wavelength selection instruction accepted by the input device 78 shown in FIG.

As an example, as shown in FIG. 9, the first turret control unit 132 inserts the BPF 83 corresponding to the first wavelength band selected by the wavelength selection unit 131 from among the plurality of BPFs 83 (see FIG. 4) into the optical path. A rotation command is output to the turret drive circuit 55 . Upon receiving the first rotation command, the turret drive circuit 55 drives the turret drive mechanism 45 to rotate the turret filter 35 to a position where the BPF corresponding to the first rotation command is inserted into the optical path.

The first imaging control section 133 outputs a first imaging command to the image sensor driver 71 . The first imaging command is a command to cause the image sensor 15 to capture light. The image sensor 15 captures the first near-infrared light transmitted through the BPF 83 corresponding to the first wavelength band, and outputs first analog image data obtained by capturing the first near-infrared light. The signal processing circuit 72 performs various signal processing on the first analog image data output from the image sensor 15 to generate and output first digital image data.

The second turret control unit 134 outputs to the turret drive circuit 55 a second rotation command for inserting into the optical path the BPF 83 corresponding to the second wavelength band selected by the wavelength selection unit 131 from among the plurality of BPFs 83 . Upon receiving the second rotation command, the turret drive circuit 55 drives the turret drive mechanism 45 to rotate the turret filter 35 to a position where the BPF corresponding to the second rotation command is inserted into the optical path.

The second imaging control section 135 outputs a second imaging command to the image sensor driver 71 . The second imaging command is a command to cause the image sensor 15 to capture the light L. FIG. The image sensor 15 captures the second near-infrared light transmitted through the BPF corresponding to the second wavelength band, and outputs second analog image data obtained by capturing the second near-infrared light. The signal processing circuit 72 performs various signal processing on the second analog image data output from the image sensor 15 to generate and output second digital image data.

The temperature derivation unit 136 calculates the temperature distribution of the subject by two-color thermometry based on the first digital image data and the second digital image data. Specifically, the temperature derivation unit 136 extracts the value of the first signal output by the physical pixel from the first digital image data for each of the plurality of physical pixels of the image sensor 15, and extracts the value of the first signal output from the physical pixel from the second digital image data. A value of the second signal output by the physical pixel is extracted. Then, the temperature derivation unit 136 derives the temperature measured by the physical pixel based on the value of the first signal and the value of the second signal, based on the two-color thermometry method, for each of the plurality of physical pixels. A calculation formula based on a two-color thermometry method may be used to derive the temperature, or a data matching table may be used. The temperature derivation unit 136 then derives the temperature measured by each of the plurality of physical pixels to calculate the temperature distribution of the object.

The display control unit 137 generates temperature-related temperature information based on the temperature distribution of the subject obtained by the temperature derivation unit 136 . Then, the display control unit 137 outputs a synthesized image 161B obtained by synthesizing the captured image obtained based on the first digital image data or the second digital image data with the temperature information, and causes the display 76 to display the synthesized image 161B. Examples of temperature information include information indicating a region where the temperature is equal to or higher than a predetermined threshold, information indicating a specific temperature value, and a plurality of sections divided for each predetermined temperature range. For example, information indicating a specific numerical value of the temperature, or information indicating a temperature distribution with a color tone corresponding to the temperature.

FIG. 11 shows a first example of a composite image 161B displayed on the display 76 as a result of the temperature measurement processing being executed by the temperature measurement processing section 130. As shown in FIG. A synthesized image 161B according to the first example is an image obtained by synthesizing a captured image 165 with temperature information 166 indicating an area whose temperature is equal to or higher than a predetermined threshold. In the first example, the temperature information 166 is, for example, information indicating a rectangular frame, but may be information other than the frame. Also, the frame may have a shape other than a square shape. Also, the frame may be displayed in a color according to the temperature. Moreover, when the frame is displayed in a color corresponding to the temperature, a scale display indicating the temperature corresponding to the color may be displayed together with the frame. Also, the temperature information 166 may include a character string indicating a specific temperature value. Specific numerical values of temperature may be maximum and/or minimum values of temperature. Also, the temperature information 166 may be displayed still or blinking.

FIG. 12 shows a second example of the composite image 161B displayed on the display 76 as a result of the temperature measurement processing being executed by the temperature measurement processing section 130. As shown in FIG. A synthesized image 161B according to the second example is an image obtained by synthesizing a plurality of pieces of

temperature information

167A, 167B, and 167C indicating specific numerical values of temperatures with a captured image 165 . Specific numerical values of temperature may be maximum and/or minimum values of temperature. Also, the plurality of pieces of

temperature information

167A, 167B, and 167C may be displayed stationary or may be displayed blinking.

FIG. 13 shows a third example of the composite image 161B displayed on the display 76 as a result of the temperature measurement processing being executed by the temperature measurement processing section 130. FIG. A synthesized image 161B according to the third example is an image obtained by synthesizing temperature information 168 indicating a plurality of sections divided for each predetermined temperature range together with specific temperature values into a captured image 165. . Specific numerical values of temperature may be maximum and/or minimum values of temperature. Also, the temperature information 168 may be displayed still or blinking.

FIG. 14 shows a fourth example of the composite image 161B displayed on the display 76 as a result of the temperature measurement processing being executed by the temperature measurement processing section 130. FIG. A synthesized image 161B according to the fourth example is an image in which the captured image 165 is synthesized with the temperature information 169 indicating the temperature distribution with a color tone corresponding to the temperature. The temperature distribution of the fourth example shown in FIG. 14 is obtained by displaying the temperature of each section of the third example shown in FIG. 13 in color tones corresponding to the temperature. For example, the color tone is defined at a constant hue angle (eg, 0.01° per 1° C.) around a reference temperature (eg, 3000° C.). Also, the temperature distribution may be visualized with an achromatic shade instead of a color tone. Temperature information 169 may include an indicator of the temperature corresponding to the color tone. Also, the temperature information 169 may include a character string indicating a specific numerical value of the temperature. Specific numerical values of temperature may be maximum and/or minimum values of temperature. Also, the temperature information 169 may be displayed still or blinking.

The

temperature information

166, 167A, 167B, 167C, 168, and 169 are examples of "temperature information" according to the technology of the present disclosure. The captured image 161A obtained in the imaging mode is an example of the “captured image” and the “first captured image” according to the technology of the present disclosure. A composite image 161B obtained in the temperature measurement mode is an example of a “composite image” according to the technology of the present disclosure. In the following description, the captured image 161A obtained in the imaging mode and the composite image 161B obtained in the temperature measurement mode will be referred to as images unless otherwise distinguished.

Next, a method for controlling the camera 1 will be described as an operation of the first embodiment.

First, the mode switching processing executed by the mode switching processing unit 110 (see FIG. 6) among the imaging support processing executed by the CPU 61 will be described with reference to FIG.

At step S11, the mode selection information acquisition unit 111 acquires mode selection information.

In step S12, the mode determination unit 112 determines whether the operation mode selected by the mode selection information is the imaging mode or the temperature measurement mode, based on the mode selection information acquired by the mode selection information acquisition unit 111. judge. If it is determined in step S12 that the mode is the imaging mode, the process shown in FIG. 15 proceeds to step S13, and if it is determined that the mode is the temperature measurement mode, the process illustrated in FIG. 15 proceeds to step S16. .

In step S13, the flag setting unit 113 sets the captured image display flag 151A as the display control flag 151 in the display control flag storage area 141, and sets the light projection control flag 152 in the light projection control flag storage area 142 so that light projection is turned on. Set control flag 152A.

In step S14, the light projection control unit 114 turns on the light projector 14.

At step S15, the mode setting unit 115 sets the imaging mode as the operation mode of the CPU 61.

In step S16, the flag setting unit 113 sets the composite image display flag 151B as the display control flag 151 in the display control flag storage area 141, and sets the light projection OFF control flag as the light projection control flag 152 in the light projection control flag storage area 142. 152B.

In step S17, the light projection control unit 114 turns off the light projector 14.

At step S18, the mode setting unit 115 sets the temperature measurement mode as the operation mode of the CPU 61.

Next, the imaging process executed by the imaging processing unit 120 (see FIG. 7) among the imaging support processes executed by the CPU 61 will be described with reference to FIG.

In step S21, the wavelength selection unit 121 selects one wavelength band used for imaging from a plurality of wavelength bands according to the wavelength selection instruction received by the input device 78.

In step S22, the turret control unit 122 rotates the turret filter 35 to a position where the optical filter corresponding to the wavelength band selected by the wavelength selection unit 121 among the plurality of optical filters is inserted into the optical path.

At step S23, the imaging control unit 123 causes the image sensor 15 to capture light. The image sensor 15 outputs analog image data obtained by capturing light, and the signal processing circuit 72 generates and outputs digital image data by performing various signal processing on the analog image data.

In step S24, the display control unit 124 causes the display 76 to display the captured image 161A based on the digital image data generated by the signal processing circuit 72.

Subsequently, the temperature measurement processing executed by the temperature measurement processing unit 130 (see FIG. 9) among the imaging support processing executed by the CPU 61 will be described with reference to FIG.

In step S31, the wavelength selection unit 131 selects two wavelength bands, that is, the first wavelength band and the second wavelength band, to be used for the two-color thermometry method.

In step S32, the first turret control unit 132 sets the turret filter 35 to a position where the BPF 83 corresponding to the first wavelength band selected by the wavelength selection unit 131 among the plurality of BPFs 83 (see FIG. 4) is inserted into the optical path. rotate.

In step S33, the first imaging control unit 133 causes the image sensor 15 to image the first near-infrared light transmitted through the BPF 83 corresponding to the first wavelength band. The image sensor 15 outputs the first analog image data obtained by imaging the first near-infrared light, and the signal processing circuit 72 performs various signal processing on the first analog image data to obtain the first analog image data. 1 Generate and output digital image data.

In step S34, the second turret control unit 134 rotates the turret filter 35 to a position where the BPF 83 corresponding to the second wavelength band selected by the wavelength selection unit 131 among the plurality of BPFs 83 is inserted into the optical path.

In step S35, the second imaging control unit 135 causes the image sensor 15 to image the second near-infrared light transmitted through the BPF 83 corresponding to the second wavelength band. The image sensor 15 outputs second analog image data obtained by imaging the second near-infrared light, and the signal processing circuit 72 performs various signal processing on the second analog image data to obtain the second analog image data. 2. Generate and output digital image data.

In step S36, the temperature derivation unit 136 calculates the temperature distribution of the subject by two-color thermometry based on the first digital image data and the second digital image data.

In step S<b>37 , the display control unit 137 generates temperature information related to temperature based on the temperature distribution of the subject obtained by the temperature derivation unit 136 . Then, the display control unit 137 outputs a synthesized image 161B obtained by synthesizing the captured image obtained based on the first digital image data or the second digital image data with the temperature information, and causes the display 76 to display the synthesized image 161B.

Note that the CPU 61 controls the position of focus by moving the focus lens 31 along the optical axis OA and adjusts the zoom magnification by moving the zoom lens 32 in each of the imaging mode and the temperature measurement mode. Control for adjustment is performed on the zoom drive mechanism 42 . Further, the CPU 61 controls the blur correction drive mechanism 44 to correct image blur by moving the blur correction lens 34 in each of the imaging mode and the temperature measurement mode. In addition, the CPU 61 controls the diaphragm drive mechanism 43 to adjust the amount of light passing through the diaphragm 33 by changing the diameter of the aperture 33A provided in the diaphragm 33 in each of the imaging mode and the temperature measurement mode. do. The CPU 61 also controls the adjustment drive mechanism 47 to adjust the focus position by moving the adjustment lens 37 in each of the imaging mode and the temperature measurement mode.

The control method of the camera 1 according to the first embodiment is an example of the "control method" according to the technology of the present disclosure.

Next, the effects of the first embodiment will be described.

In the first embodiment, the CPU 61 has an imaging mode for causing the image sensor 15 to image light and a temperature measurement mode for deriving temperature based on the near-infrared light received by the image sensor 15 . The control flag differs between the imaging mode and the temperature measurement mode. Therefore, it is possible to change the control contents of the controlled object between the imaging mode and the temperature measurement mode.

For example, in the first embodiment, the control flag includes the display control flag 151 displayed on the display 76. Therefore, an image displayed on the display 76 can be made different as an example of differentiating the control details in the controlled object between the imaging mode and the temperature measurement mode.

Also, in the first embodiment, the control flag includes the light projection control flag 152 that operates the light projector 14 . Therefore, the operation of the light projector 14 can be made different as an example of differentiating the control details in the controlled object between the imaging mode and the temperature measurement mode.

As described above, in the first embodiment, the control contents of the controlled object can be made different between the imaging mode and the temperature measurement mode. Therefore, the operation of the imaging device can be controlled to be suitable for each mode.

In addition, in the imaging mode, the CPU 61 sets a captured image display flag 151A as the display control flag 151 to cause the display 76 to display a captured image 161A obtained by light being received by the image sensor 15 . As a result, the captured image 161A that does not contain the temperature information is displayed on the display 76. FIG. Therefore, in the imaging mode, the visibility of the captured image 161A can be improved as compared with the case where the captured image 161A includes temperature information, for example.

In addition, in the temperature measurement mode, the CPU 61 sets, as the display control flag 151, a composite image display flag 151B that causes the display 76 to display a composite image 161B including temperature information indicating temperature. As a result, a synthesized image 161B including temperature information is displayed on the display 76. FIG. Therefore, in the temperature measurement mode, the user can accurately grasp the temperature of the object, compared to the case where the temperature information is not displayed on the display 76, for example.

In addition, the CPU 61 sets a light projection ON control flag 152A for switching the light projector 14 ON as the light projection control flag 152 in the imaging mode. Thereby, in imaging mode, the light projector 14 is switched on. Therefore, in the imaging mode, it is possible to ensure the amount of light emitted from the subject. This allows the user to check the indoor environment through the captured image 161A, for example, even in a situation where the room is filled with smoke due to a fire.

Also, in the temperature measurement mode, the CPU 61 sets a light projection off control flag 152B for switching off the light projector 14 as the light projection control flag 152 . This switches off the light projector 14 in the temperature measurement mode. Therefore, it is possible to prevent the illumination light from the light projector 14 from being mixed with the near-infrared light emitted from the subject. In addition, the measurement accuracy of temperature measurement can be improved as compared with, for example, the case where illumination light from the projector 14 is mixed with near-infrared light emitted from the object.

In addition, the CPU 61 switches the light projector 14 on and off in response to switching between the imaging mode and the temperature measurement mode. Therefore, convenience can be improved compared to, for example, the case where the user has to switch the light projector 14 on and off.

Also, in the temperature measurement mode, the CPU 61 outputs a synthesized image 161B obtained by synthesizing a captured image obtained by receiving light from the image sensor 15 and temperature information indicating temperature. Therefore, by displaying the synthesized image 161B on the display 76, the user can easily grasp the situation and temperature of the subject.

Next, a modified example of the first embodiment will be described.

In the first embodiment, as an example, as shown in FIG. 8, a captured image 161A in which a curtain 163 provided on the indoor side of a window 162 of a building is lit with fire 164 is displayed on the display 76. However, the captured image 161A may be any image. For example, as shown in FIG. 18 as an example, the captured image 161A may be an image obtained by capturing the arm 170 of a person.

Further, in the first embodiment, as an example, as shown in FIGS. 11 to 14, a synthesized image 161B obtained by synthesizing temperature information with a captured image of a curtain provided on the indoor side of a window of a building is displayed. 76 is shown, the synthesized image 161B may be any image as long as it is an image obtained by synthesizing the captured image and the temperature information. For example, as shown in FIG. 19 as an example, the synthesized image 161B may be an image obtained by synthesizing temperature information 172 indicating the temperature distribution with an imaged image 171 obtained by imaging a person's arm 170 . Synthesis may be, for example, superimposition of a plurality of images by alpha blending, or embedding of temperature information in a captured image.

Further, in the first embodiment, instead of setting the light projection OFF control flag 152B by the flag setting unit 113 shown in FIG. Further, when the light emission suppression control flag is set by the flag setting unit 113, the light emission control unit 114 outputs a light emission suppression command to the light emission control circuit 73, and the light emission control circuit 73 outputs the light emission suppression command. Accordingly, the light projection of the light projector 14 may be suppressed (that is, the amount of light projected from the light projector 14 may be suppressed). Suppression refers to, for example, the operation of reducing the amount of light below a reference amount. The reference amount may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may

[Second embodiment]
Next, a second embodiment will be described.

In the second embodiment, the configuration of the camera 1 is changed as follows from the first embodiment. The points of the second embodiment that are different from the first embodiment will be described below.

As shown in FIG. 20 as an example, the mode switching processing unit 110 has a state signal acquisition unit 181, a state signal determination unit 182, a flag setting unit 113, a light projection control unit 114, and a mode setting unit 115.

The state signal acquisition unit 181 acquires the state signal output from the light projection control circuit 73 according to the operating state of the light projector 14 . The light projection control circuit 73 outputs an ON state signal representing the ON state of the light projector 14 as a state signal when the light projector 14 is ON, and an ON state signal representing the OFF state of the light projector 14 when the light projector 14 is OFF. Output the state signal as a state signal.

The state signal determination unit 182 determines whether the state signal acquired by the state signal acquisition unit 181 is an ON state signal indicating that the light projector 14 is ON.

When the state signal determination unit 182 determines that the state signal is the ON state signal, the flag setting unit 113 sets the captured image display flag 151A as the display control flag 151 in the display control flag storage area 141 . Further, when the state signal determination unit 182 determines that the state signal is not the ON state signal, the flag setting unit 113 sets the composite image display flag 151B as the display control flag 151 in the display control flag storage area 141. .

The mode setting unit 115 sets the imaging mode as the operation mode of the CPU 61 when the captured image display flag 151A is set as the display control flag 151 . Further, when the composite image display flag 151B is set as the display control flag 151, the mode setting unit 115 sets the temperature measurement mode as the mode of the CPU 61. FIG.

In the second embodiment, the imaging mode is an example of the "first mode" according to the technology of the present disclosure, and the temperature measurement mode is an example of the "second mode" according to the technology of the present disclosure. Also, the display control flag 151 is an example of a “control factor” according to the technology of the present disclosure. In addition, the display 76 is an example of the "controlled object" according to the technique of the present disclosure, and the image displayed on the display 76 is an example of "control details for the controlled object" according to the technique of the present disclosure. Further, the display control flag 151 is an example of the "display control factor" according to the technology of the present disclosure, the captured image display flag 151A is an example of the "captured image display factor" according to the technology of the present disclosure, and the composite image The display flag 151B is an example of a "temperature information display factor" according to the technology of the present disclosure.

Next, a method for controlling the camera 1 will be described as an operation of the second embodiment.

In the second embodiment, the imaging processing performed by the imaging processing unit 120 and the temperature measurement processing performed by the temperature measurement processing unit 130 are the same as in the first embodiment. In the second embodiment, mode switching processing executed by the mode switching processing unit 110 is different from that in the first embodiment. Mode switching processing executed by the mode switching processing unit 110 according to the second embodiment will be described below with reference to FIG. 21 .

In step S41, the state signal acquisition unit 181 acquires the state signal output from the light projection control circuit 73 according to the operating state of the light projector .

In step S42, the state signal determination unit 182 determines whether or not the state signal is the ON state signal. If it is determined in step S42 that it is an on-state signal, the processing shown in FIG. 21 proceeds to step S43, and if it is determined that it is not an on-state signal, the processing illustrated in FIG. 21 proceeds to step S45. do.

In step S<b>43 , the flag setting unit 113 sets the captured image display flag 151</b>A as the display control flag 151 in the display control flag storage area 141 .

At step S44, the mode setting unit 115 sets the imaging mode as the operation mode of the CPU 61.

In step S45, the flag setting unit 113 sets the composite image display flag 151B as the display control flag 151 in the display control flag storage area 141.

At step S46, the mode setting unit 115 sets the temperature measurement mode as the operation mode of the CPU 61.

The control method of the camera 1 according to the second embodiment is an example of the "control method" according to the technology of the present disclosure.

Next, regarding the effects of the second embodiment, the differences from the first embodiment will be described.

In the second embodiment, the CPU 61 sets the imaging mode when the light projector 14 is on, and sets the temperature measurement mode when the light projector 14 is off. Therefore, convenience can be improved as compared with the case where the imaging mode and the temperature measurement mode are not switched according to the ON/OFF operation of the light projector 14, for example.

[Third embodiment]
Next, a third embodiment will be described.

In the third embodiment, the configuration of the camera 1 is changed as follows from the first embodiment. The points of the third embodiment that are different from the first embodiment will be described below.

As an example, as shown in FIG. 22, the CPU 61 functions as a parameter change processing section 190 in addition to the mode switching processing section 110, imaging processing section 120, and temperature measurement processing section 130 described above. The parameter change processing unit 190 is a processing unit that operates when the mode of the CPU 61 is the imaging mode and the temperature measurement mode. The parameter change processing unit 190 is a processing unit that sets different mode parameters 211 for the imaging mode and the temperature measurement mode. The parameter change processing unit 190 has a mode determination unit 191 and a mode-specific parameter setting unit 192 .

The mode determination unit 191 determines whether the operation mode of the CPU 61 is the imaging mode or the temperature measurement mode.

The RAM 63 is provided with a parameter storage area 201 for storing mode-specific parameters 211 .

When the mode determination unit 191 determines that the operation mode of the CPU 61 is the imaging mode, the mode-specific parameter setting unit 192 performs various parameter setting processes based on the imaging conditions, etc., to correspond to the imaging mode. An imaging mode parameter 211A is derived. Then, the mode-specific parameter setting unit 192 sets the imaging mode parameter 211 A as the mode-specific parameter 211 in the parameter storage area 201 .

Further, when the mode determination unit 191 determines that the operation mode of the CPU 61 is the temperature measurement mode, the mode-specific parameter setting unit 192 performs various parameter setting processes based on the temperature measurement conditions and the like to determine the temperature. A temperature measurement mode parameter 211B corresponding to the measurement mode is derived. Then, the mode-specific parameter setting unit 192 sets the temperature measurement mode parameter 211 B as the mode-specific parameter 211 in the parameter storage area 201 .

The imaging mode parameters 211A include a first imaging setting parameter 212A set for imaging and a first image processing setting parameter 213A set for image processing of the captured image. Similarly, the temperature measurement mode parameters 211B include a second imaging setting parameter 212B set for imaging and a second image processing setting parameter 213B set for image processing of the captured image.

The first imaging setting parameter 212A and the second imaging setting parameter 212B include a parameter related to the projector 14, a parameter related to shutter speed, a parameter related to the diaphragm 33, a parameter related to photometry, a parameter related to the sensitivity of the image sensor 15, a parameter related to high dynamic range, and a parameter related to protection. Contains parameters related to vibration control. The parameters relating to the sensitivity of the image sensor 15 include parameters relating to the gain of the image sensor 15 and/or parameters relating to the conversion efficiency of the image sensor 15 .

The first image processing setting parameter 213A and the second image processing setting parameter 213B include noise reduction parameters, sharpness parameters, contrast parameters, and tone parameters.

As an example, the first imaging setting parameter 212A included in the imaging mode parameter 211A is set as follows.
The parameters related to the light projector 14 are set to the parameters for the light projector 14 to project light.
A parameter related to the shutter speed is set to a parameter that makes the shutter speed equal to or higher than the reference speed. Shutter speed is the time from when the front curtain starts to open until the rear curtain finishes closing in a mechanical shutter, the time from when an electronic front curtain operates until the rear curtain of a mechanical shutter finishes closing, or an electronic It is defined by the time from when the shutter starts operating to when it finishes operating. The reference speed may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (for example, subject temperature and/or imaging conditions). may
A parameter relating to the diaphragm 33 is set to a parameter that makes the diaphragm amount greater than or equal to the reference diaphragm amount. The diaphragm amount is proportional to the diameter of the aperture 33A provided in the diaphragm 33. FIG. The reference aperture amount may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (for example, subject temperature and/or imaging conditions). There may be.
The parameters related to photometry are set to parameters for performing photometry by the average photometry method or the multi-pattern photometry method. Photometry is the measurement of the brightness of a subject.
A parameter related to the gain of the image sensor 15 is set to a parameter that makes the gain of the image sensor 15 greater than or equal to the reference gain. The gain of the image sensor 15 refers to the analog gain of an A/D converter (not shown) connected to the photodiode of the image sensor 15, for example. The reference gain may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may
A parameter related to the conversion efficiency of the image sensor 15 is set to a parameter that makes the conversion efficiency of the image sensor 15 equal to or higher than the reference conversion efficiency. The conversion efficiency of the image sensor 15 refers to the efficiency of converting electric charges accumulated in a variable capacitor (not shown) connected to a photodiode included in the image sensor 15 into voltage. The reference conversion efficiency may be a fixed value, or may be a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (such as subject temperature and/or imaging conditions). There may be.
A parameter related to the high dynamic range is set to a parameter that turns on the high dynamic range. The high dynamic range is a display technology that improves the contrast (that is, the contrast ratio) between the bright and dark portions of the captured image 161A. Turning on the high dynamic range means expanding the dynamic range beyond the reference range. The reference range may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may
A parameter related to anti-vibration control is set to a parameter for turning on anti-vibration control. Anti-vibration control is control for moving the blur correction lens 34 in a direction in which image blur is corrected. Turning on anti-vibration control means performing control to move the blur correction lens 34 .

A parameter related to noise reduction is set to a parameter that makes the degree of strength of noise reduction larger than the first reference degree. Noise reduction is image processing for reducing noise that appears in a captured image, and the strength of noise reduction means increasing or decreasing the rate at which noise that appears in a captured image is reduced. The first reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be
A parameter related to sharpness is set to a parameter that makes the degree of strength of sharpness larger than the second reference degree. Sharpness refers to enhancement of the contour of the captured image, and strength of sharpness refers to increasing or decreasing the rate at which the contour of the captured image is enhanced. The second reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be
The parameter related to contrast is set to a parameter that makes the degree of strength of contrast larger than the third reference degree. Contrast refers to the difference in brightness and/or color of a captured image, and strength of contrast refers to increasing or decreasing the difference in brightness and/or color of a captured image. The third reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be
A parameter relating to tone is set to a parameter that makes the degree of intensity of the tone larger than the fourth reference degree. The tone is the color tone of the captured image, and the intensity of the tone is to increase or decrease the color tone of the captured image. The fourth reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be

Also, the second imaging setting parameter 212B included in the temperature measurement mode parameter 211B is set as follows, as an example.
The parameters relating to the light projector 14 are set to parameters for which the light projector 14 does not project light.
A parameter related to the shutter speed is set to a parameter that makes the shutter speed less than the reference speed. The reference speed may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (for example, subject temperature and/or imaging conditions). may
A parameter related to the diaphragm 33 is set to a parameter that makes the diaphragm amount smaller than the reference amount. The reference aperture amount may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (for example, subject temperature and/or imaging conditions). There may be.
The parameters related to photometry are set to parameters for performing photometry with a highlight-weighted photometry method, a center-weighted photometry method, or a spot photometry method.
A parameter related to the gain of the image sensor 15 is set to a parameter that makes the gain of the image sensor 15 less than the reference gain. The reference gain may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may
A parameter related to the conversion efficiency of the image sensor 15 is set to a parameter that makes the photoelectric conversion efficiency of the image sensor 15 less than the reference conversion efficiency. The reference conversion efficiency may be a fixed value, or may be a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (such as subject temperature and/or imaging conditions). There may be.
A parameter related to the high dynamic range is set to a parameter that turns off the high dynamic range. Turning off the high dynamic range means setting the dynamic range to the reference range. The reference range may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may
A parameter related to anti-vibration control is set to a parameter for turning off anti-vibration control.

Further, the second image processing setting parameter 213B included in the temperature measurement mode parameter 211B is set as follows as an example.
A parameter related to noise reduction is set to a parameter that makes the degree of strength of noise reduction equal to or less than the first reference degree. The first reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be
The sharpness-related parameter is set to a parameter that makes the degree of strength of sharpness equal to or lower than the second reference degree. The second reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be
The parameter related to contrast is set to a parameter that makes the degree of strength of contrast equal to or lower than the third reference degree. The third reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be
The tone-related parameter is set to a parameter that makes the degree of intensity of the tone equal to or lower than the fourth reference degree. The fourth reference degree may be a fixed value, or a variable value that is changed according to instructions input by the user into the input device 78 and/or various conditions (eg, subject temperature and/or imaging conditions). may be

As an example, as shown in FIG. 23 , when the CPU 61 is in the imaging mode, the imaging control unit 123 controls imaging related to imaging according to the first imaging setting parameters 212A included in the imaging mode parameters 211A stored in the parameter storage area 201. Make settings.
(1) The imaging control unit 123 sets the light projector 14 to project light as a setting related to the light projector 14 , and outputs an ON command to the light projection control circuit 73 .
(2) The imaging control unit 123 adjusts the shutter speed by setting the shutter speed to be equal to or higher than the reference speed as the shutter speed setting.
(3) The imaging control unit 123 sets the diaphragm amount equal to or larger than the reference amount as the setting for the diaphragm 33 , and outputs the diaphragm command corresponding to the set diaphragm amount to the diaphragm drive circuit 53 .
(4) The imaging control unit 123 sets the photometry method to the average photometry method or the multi-pattern photometry method as a setting related to photometry.
(5) The imaging control unit 123 sets the gain of the image sensor 15 to be equal to or higher than the reference gain as the setting related to the gain of the image sensor 15 .
(6) The imaging control unit 123 sets the photoelectric conversion efficiency of the image sensor 15 to be equal to or higher than the reference photoelectric conversion efficiency as the setting regarding the photoelectric conversion efficiency of the image sensor 15 . The imaging control unit 123 outputs a sensitivity command corresponding to the set gain and photoelectric conversion efficiency to the image sensor driver 71 .
(7) The imaging control unit 123 sets the high dynamic range to ON as a setting related to the high dynamic range.
(8) The imaging control unit 123 sets anti-shake control to ON as a setting related to anti-shake control, and outputs a blur correction command to the blur correction drive circuit 54 .

Further, as shown in FIG. 23 as an example, when the CPU 61 is in the imaging mode, the display control unit 124 performs the Perform image processing settings related to image processing.
(9) The display control unit 124 sets the degree of strength of noise reduction to be higher than the first reference degree as a setting related to noise reduction.
(10) The display control unit 124 sets the degree of strength of sharpness to be higher than the second reference degree as a setting related to sharpness.
(11) The display control unit 124 sets the degree of strength of contrast to be higher than the third reference degree as the setting related to contrast.
(12) The display control unit 124 sets the degree of intensity of the tone to be higher than the fourth reference degree as the tone-related setting.

As an example, as shown in FIG. 24, when the CPU 61 is in the temperature measurement mode, the first imaging control unit 133 and the second imaging control unit 135 are included in the temperature measurement mode parameter 211B stored in the parameter storage area 201. Imaging settings related to imaging are performed according to the second imaging setting parameter 212B.
(1) The first image pickup control unit 133 and the second image pickup control unit 135 set the light projector 14 so that the light projector 14 does not project light, and output an OFF command to the light projection control circuit 73 .
(2) The first imaging control unit 133 and the second imaging control unit 135 adjust the shutter speed by setting the shutter speed to be less than the reference speed as the shutter speed setting.
(3) The first imaging control unit 133 and the second imaging control unit 135 set the aperture amount to be less than the reference amount as a setting related to the aperture 33, and send an aperture command corresponding to the set aperture amount to the aperture drive circuit 53. output.
(4) The first imaging control unit 133 and the second imaging control unit 135 set the photometry method to highlight-weighted photometry, center-weighted photometry, or spot photometry.
(5) The first imaging control unit 133 and the second imaging control unit 135 set the gain of the image sensor 15 to be less than the reference gain as the setting related to the gain of the image sensor 15 .
(6) The first imaging control unit 133 and the second imaging control unit 135 set the photoelectric conversion efficiency of the image sensor 15 to be less than the reference photoelectric conversion efficiency as the setting regarding the photoelectric conversion efficiency of the image sensor 15 . The first imaging control unit 133 and the second imaging control unit 135 output sensitivity commands corresponding to the set gain and photoelectric conversion efficiency to the image sensor driver 71 .
(7) The first imaging control unit 133 and the second imaging control unit 135 set the high dynamic range to OFF as a setting related to the high dynamic range.
(8) The first image pickup control unit 133 and the second image pickup control unit 135 set image stabilization control to OFF as settings related to image stabilization control, and output a motion compensation stop command to the motion compensation drive circuit 54 .

As an example, as shown in FIG. 24, when the CPU 61 is in the temperature measurement mode, the display control unit 137 sets the second image processing setting parameter included in the temperature measurement mode parameter 211B stored in the parameter storage area 201. 213B to set image processing.
(9) The display control unit 137 sets the degree of strength of noise reduction to be equal to or less than the first reference degree as a setting related to noise reduction.
(10) The display control unit 137 sets the degree of strength of sharpness to be equal to or lower than the second reference degree as a setting related to sharpness.
(11) The display control unit 137 sets the degree of strength of contrast to a third reference degree or less as a setting related to contrast.
(12) The display control unit 137 sets the degree of intensity of the tone to be equal to or lower than the fourth reference degree as the tone-related setting.

In the third embodiment, the settings related to the projector 14, the shutter speed settings, the aperture 33 settings, the photometry settings, the image sensor 15 sensitivity settings, the high dynamic range settings, and the anti-vibration control settings are the same as those described in the present disclosure. It is an example of the "imaging setting" which concerns on a technique. Also, the first imaging setting parameter 212A and the second imaging setting parameter 212B are examples of "imaging setting factor related to imaging setting" and "control factor" according to the technology of the present disclosure. Also, the settings related to noise reduction, the settings related to sharpness, the settings related to contrast, and the settings related to tone are examples of "image processing settings" according to the technology of the present disclosure. Also, the first image processing setting parameter 213A and the second image processing setting parameter 213B are examples of the “image processing setting factor related to image processing setting” and the “control factor” according to the technology of the present disclosure.

Next, a method for controlling the camera 1 will be described as an operation of the third embodiment.

In the third embodiment, mode switching processing executed by the mode switching processing unit 110, imaging processing executed by the imaging processing unit 120, and temperature measurement processing executed by the temperature measurement processing unit 130 are the same as in the first embodiment. . The third embodiment differs from the first embodiment in that a parameter change processing unit 190 executes parameter change processing. Parameter change processing executed by the parameter change processing unit 190 according to the third embodiment will be described below with reference to FIG. 25 .

At step S51, the mode determination unit 112 determines whether the operation mode of the CPU 61 is the imaging mode or the temperature measurement mode. If it is determined in step S51 that it is the imaging mode, the processing shown in FIG. 25 proceeds to step S52, and if it is determined that it is in the temperature measurement mode, the processing illustrated in FIG. 25 proceeds to step S53. .

In step S52, the mode-specific parameter setting unit 192 derives the imaging mode parameter 211A corresponding to the imaging mode, and sets the imaging mode parameter 211A as the mode-specific parameter 211 in the parameter storage area 201.

In step S53, the mode-specific parameter setting unit 192 derives the temperature measurement mode parameter 211B corresponding to the temperature measurement mode, and sets the temperature measurement mode parameter 211B as the mode-specific parameter 211 in the parameter storage area 201.

The control method of the camera 1 according to the third embodiment is an example of the "control method" according to the technology of the present disclosure.

Next, regarding the effects of the third embodiment, the differences from the first embodiment will be described.

In the third embodiment, imaging setting parameters regarding imaging settings are different between the imaging mode and the temperature measurement mode. That is, as an example, the first imaging setting parameter 212A is set in the imaging mode, and the second imaging setting parameter 212B is set in the temperature measurement mode. Therefore, compared to the case where the imaging setting parameters are the same in the imaging mode and the temperature measurement mode, for example, in the imaging mode, it is possible to obtain good image quality for the captured image, and in the temperature measurement mode, for example, the load on the CPU 61 is reduced. It is possible to secure the measurement accuracy while reducing the

The imaging settings also include settings related to the projector 14 . The CPU 61 sets the light projector 14 to emit light in the imaging mode, and sets the light projector 14 not to emit light in the temperature measurement mode. Therefore, in the imaging mode, by ensuring the amount of light emitted from the subject, it is possible to obtain a better image quality of the captured image than when the light projector 14 projects light. On the other hand, in the temperature measurement mode, compared to the case where the light projector 14 projects light, for example, by suppressing the near-infrared light emitted from the subject from being mixed with the illumination light from the light projector 14, measurement accuracy is ensured. can do.

Also, the imaging settings include settings related to shutter speed. The CPU 61 sets the shutter speed to a reference speed or higher in the imaging mode, and sets the shutter speed to less than the reference speed in the temperature measurement mode. That is, the CPU 61 sets the shutter speed in the imaging mode longer than the shutter speed in the temperature measurement mode. Accordingly, in the imaging mode, by ensuring the amount of light incident on the image sensor 15, it is possible to obtain a good image quality for the captured image. On the other hand, in the temperature measurement mode, it is possible to ensure measurement accuracy by suppressing noise from being included in the first analog image data and the second analog image data output from the image sensor 15 .

Also, the imaging settings include settings related to the aperture 33 . The CPU 61 sets the diaphragm amount to a reference amount or more in the imaging mode, and sets the diaphragm amount to less than the reference amount in the temperature measurement mode. That is, the CPU 61 sets the aperture amount in the imaging mode to be larger than the aperture amount in the temperature measurement mode. Accordingly, in the imaging mode, by ensuring the amount of light incident on the image sensor 15, it is possible to obtain a good image quality for the captured image. On the other hand, in the temperature measurement mode, it is possible to ensure measurement accuracy by suppressing noise from being included in the first analog image data and the second analog image data output from the image sensor 15 .

Also, the imaging settings include settings related to photometry. The CPU 61 sets the photometry method to the average photometry method or the multi-pattern photometry method in the imaging mode, and sets the photometry method to the highlight-weighted photometry method, the center-weighted photometry method, or the spot photometry method in the temperature measurement mode. Accordingly, in the imaging mode, by securing the amount of light emitted from the entire subject and incident on the image sensor 15, it is possible to obtain a good image quality of the captured image. On the other hand, in the temperature measurement mode, the measurement accuracy can be ensured by suppressing saturation of the amount of light emitted from the highest temperature region of the subject and incident on the image sensor 15 .

The imaging settings also include settings related to the sensitivity of the image sensor 15 . The parameters relating to the sensitivity of the image sensor 15 include parameters relating to the gain of the image sensor 15 and parameters relating to the conversion efficiency of the image sensor 15 . The CPU 61 sets the gain of the image sensor 15 to a reference gain or more in the imaging mode, and sets the gain of the image sensor 15 to less than the reference gain in the temperature measurement mode. That is, the CPU 61 sets the gain in the imaging mode higher than the gain in the temperature measurement mode. Accordingly, in the imaging mode, analog image data corresponding to the exposure can be obtained, so that a good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, noise is included in the first analog image data and the second analog image data output from the image sensor 15, and the light radiated from the area with the highest temperature of the object is subject to By suppressing saturation of the peak values of the first analog image data and the second analog image data, measurement accuracy can be ensured.

In addition, the CPU 61 sets the conversion efficiency of the image sensor 15 to be equal to or higher than the reference conversion efficiency in the imaging mode, and sets the conversion efficiency of the image sensor 15 to be lower than the reference conversion efficiency in the temperature measurement mode. That is, the CPU 61 sets the conversion efficiency in the imaging mode higher than the conversion efficiency in the temperature measurement mode. Accordingly, in the imaging mode, analog image data corresponding to the exposure can be obtained, so that a good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, noise is included in the first analog image data and the second analog image data output from the image sensor 15, and the light radiated from the area with the highest temperature of the object is subject to By suppressing saturation of the peak values of the first analog image data and the second analog image data, measurement accuracy can be ensured.

Also, the imaging settings include settings related to high dynamic range. The CPU 61 sets the high dynamic range to ON in the imaging mode, and sets the high dynamic range to OFF in the temperature measurement mode. As a result, in the imaging mode, the dynamic range is wider than the reference range, so that a good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, by setting the dynamic range to the reference range, measurement accuracy can be ensured.

Also, the imaging settings include settings related to anti-vibration control. The CPU 61 sets anti-vibration control to ON in the imaging mode, and sets anti-vibration control to OFF in the temperature measurement mode. As a result, in the imaging mode, image blurring is suppressed, so that a good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, the load on the CPU 61 can be reduced by reducing the amount of computation required by the CPU 61 to suppress blurring of the image.

Also, in the third embodiment, image processing setting parameters regarding image processing settings are different between the imaging mode and the temperature measurement mode. That is, as an example, the first image processing setting parameter 213A is set in the imaging mode, and the second image processing setting parameter 213B is set in the temperature measurement mode. Therefore, compared to the case where the image processing setting parameters are the same in the imaging mode and the temperature measurement mode, for example, in the imaging mode, it is possible to obtain good image quality for the captured image, and in the temperature measurement mode, for example, the CPU 61 The burden can be reduced.

In addition, the image processing settings include settings related to noise reduction. The CPU 61 sets the degree of noise reduction strength to be greater than the first reference degree in the imaging mode, and sets the degree of noise reduction strength to the first reference degree or less in the temperature measurement mode. That is, the CPU 61 sets the noise reduction in the imaging mode to be stronger than the noise reduction in the temperature measurement mode. Accordingly, in the imaging mode, the noise included in the captured image is reduced, so that the captured image can be obtained with good image quality. On the other hand, in the temperature measurement mode, the load on the CPU 61 can be reduced by reducing the amount of computation required by the CPU 61 to reduce noise.

Also, the image processing settings include settings related to sharpness. The CPU 61 sets the degree of sharpness to be greater than the second reference degree in the imaging mode, and sets the degree of sharpness to the second reference degree or less in the temperature measurement mode. That is, the CPU 61 sets the sharpness in the imaging mode stronger than the sharpness in the temperature measurement mode. As a result, in the imaging mode, the sharpness of the captured image is enhanced, so that a good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, the burden on the CPU 61 can be reduced by reducing the amount of arithmetic processing of the CPU 61 necessary for adjusting the sharpness.

Also, the image processing settings include settings related to contrast. The CPU 61 sets the degree of contrast strength to be greater than the third reference degree in the imaging mode, and sets the degree of contrast strength to the third reference degree or less in the temperature measurement mode. That is, the CPU 61 sets the contrast in the imaging mode higher than the contrast in the temperature measurement mode. As a result, in the imaging mode, the contrast of the captured image is enhanced, so that good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, the load on the CPU 61 can be reduced by reducing the amount of computation required by the CPU 61 for adjusting the contrast.

Also, the image processing settings include settings related to tone. The CPU 61 sets the degree of tone strength to be greater than the fourth reference degree in the imaging mode, and sets the degree of tone strength to the fourth reference degree or less in the temperature measurement mode. That is, the CPU 61 sets the tone in the imaging mode stronger than the tone in the temperature measurement mode. As a result, in the imaging mode, the contrast of the captured image is enhanced, so that good image quality can be obtained for the captured image. On the other hand, in the temperature measurement mode, the load on the CPU 61 can be reduced by reducing the amount of computation required by the CPU 61 for adjusting the contrast.

Next, a modified example of the third embodiment will be described.

In the third embodiment, in the imaging mode and the temperature measurement mode, settings related to the projector 14, settings related to the shutter speed, settings related to the aperture 33, settings related to photometry, settings related to the sensitivity of the image sensor 15, settings related to high dynamic range, and settings related to protection Although settings related to vibration control are different, combinations of imaging settings that are different between the imaging mode and the temperature measurement mode may be other than the above. For example, the setting for high dynamic range may be set to on in both imaging mode and temperature measurement mode. In addition, the setting regarding anti-vibration control may be set to ON in both the imaging mode and the temperature measurement mode.

In addition, the imaging settings that are made different between the imaging mode and the temperature measurement mode include settings related to the projector 14, settings related to the shutter speed, settings related to the aperture 33, settings related to photometry, settings related to the sensitivity of the image sensor 15, settings related to high dynamic range, and Any setting other than the above may be used as long as at least one setting related to anti-vibration control is included. In addition, the imaging settings that are made different between the imaging mode and the temperature measurement mode include settings related to the projector 14, settings related to the shutter speed, settings related to the aperture 33, settings related to photometry, settings related to the sensitivity of the image sensor 15, settings related to high dynamic range, and Various imaging settings related to the camera 1 may be included in addition to the settings related to anti-vibration control.

In addition, in the third embodiment, the noise reduction setting, the sharpness setting, the contrast setting, and the tone setting are different between the imaging mode and the temperature measurement mode. Combinations of settings may be other than the above.

Also, the image processing settings that are differentiated between the imaging mode and the temperature measurement mode may be other than the above, as long as they include at least one of noise reduction settings, sharpness settings, contrast settings, and tone settings. Also, the image processing settings that are made different between the imaging mode and the temperature measurement mode may include various image processing settings related to the camera 1 in addition to settings related to noise reduction, settings related to sharpness, settings related to contrast, and settings related to tone. .

In the third embodiment, the CPU 61 sets the light projector 14 not to project light in the temperature measurement mode, but sets the light projector 14 to suppress light projection (that is, the amount of light projected from the light projector 14 can be set).

[Fourth embodiment]
Next, a fourth embodiment will be described.

In the fourth embodiment, the configuration of the camera 1 is changed as follows from the first embodiment. Differences of the fourth embodiment from the first embodiment will be described below.

As shown in FIG. 26 as an example, the mode switching processing unit 110 includes a mode determination unit 112, a measured temperature acquisition unit 221, a temperature measurement mode end determination unit 222, a flag setting unit 113, a light projection control unit 114, and a mode setting unit. 115.

The mode determination unit 112 determines whether the operation mode of the CPU 61 is the imaging mode or the temperature measurement mode.

When the mode determination unit 112 determines that the operation mode of the CPU 61 is the temperature measurement mode, the measured temperature acquisition unit 221 acquires the temperature of the subject measured in the temperature measurement mode (hereinafter referred to as the measured temperature). do. The measured temperature is the temperature distribution value of the subject, the highest value of the temperature distribution of the subject, the mode of the temperature distribution of the subject, the median of the temperature distribution of the subject, and the subject Any of the average values of the temperature distribution of

The temperature measurement mode end determination unit 222 determines whether to end the temperature measurement mode based on the measured temperature acquired by the measured temperature acquisition unit 221 . As an example, the temperature measurement mode end determination unit 222 derives a value based on the measured temperature acquired by the measured temperature acquisition unit 221, determines whether the derived value is equal to or less than a threshold value, and ends the measurement mode. Determine whether to terminate. The value based on the measured temperature may be any value derived from the measured temperature. For example, it may be the value of the measured temperature itself, or the value of the amount of radiant heat calculated based on the measured temperature. Also, a calculation formula may be used to derive the value based on the measured temperature, or a data matching table may be used. The temperature measurement mode end determination unit 222 determines to end the temperature measurement mode when the value based on the measured temperature is equal to or less than the threshold.

When the temperature measurement mode end determination unit 222 determines to end the temperature measurement mode, the flag setting unit 113 sets the captured image display flag 151A as the display control flag 151 in the display control flag storage area 141, and A light projection ON control flag 152 A is set as the light projection control flag 152 in the light projection control flag storage area 142 .

The light projection control unit 114 outputs an ON command to the light projection control circuit 73 when the light projection ON control flag 152A is set by the flag setting unit 113 . The ON command is a command to switch the projector 14 ON.

The mode setting unit 115 sets the imaging mode as the mode of the CPU 61 .

Next, a method for controlling the camera 1 will be described as an operation of the fourth embodiment.

In the fourth embodiment, the imaging processing executed by the imaging processing unit 120 and the temperature measurement processing executed by the temperature measurement processing unit 130 are the same as in the first embodiment. In the third embodiment, mode switching processing executed by the mode switching processing unit 110 is different from that in the first embodiment. Mode switching processing executed by the mode switching processing unit 110 according to the fourth embodiment will be described below with reference to FIG. 27 .

At step S61, the mode determination unit 112 determines whether the mode of the CPU 61 is the imaging mode or the temperature measurement mode. If it is determined in step S61 that the imaging mode is set, the process shown in FIG. 27 proceeds to step S62, and if it is determined that the temperature measurement mode is set, the process shown in FIG. 27 ends.

At step S62, the measured temperature acquisition unit 221 acquires the measured temperature measured in the temperature measurement mode.

In step S63, the temperature measurement mode end determination unit 222 determines whether to end the temperature measurement mode based on the measured temperature acquired by the measured temperature acquisition unit 221. If it is determined in step S63 that the temperature measurement mode should be ended, the process shown in FIG. 27 proceeds to step S64, and if it is determined not to end the temperature measurement mode, the process shown in FIG. 27 ends.

In step S64, the flag setting unit 113 sets the captured image display flag 151A as the display control flag 151 in the display control flag storage area 141, and sets the light projection control flag 152 in the light projection control flag storage area 142 so that light projection is turned on. Set control flag 152A.

In step S65, the light projection control unit 114 switches the light projector 14 on.

In step S66, the mode setting unit 115 sets the imaging mode as the mode of the CPU61.

The control method of the camera 1 according to the fourth embodiment is an example of the "control method" according to the technology of the present disclosure.

Next, the effects of the fourth embodiment that differ from those of the first embodiment will be described.

In the fourth embodiment, the CPU 61 switches from the temperature measurement mode to the imaging mode according to the temperature of the subject in the temperature measurement mode. Therefore, convenience can be improved compared to the case where the temperature measurement mode is not switched to the imaging mode according to the temperature of the object, for example.

[Fifth embodiment]
Next, a fifth embodiment will be described.

In the fifth embodiment, the configuration of the camera 1 is changed as follows from the first embodiment. Differences of the fifth embodiment from the first embodiment will be described below.

As shown in FIG. 28 as an example, the CPU 61 functions as an integrated display processing section 230 . The integrated display processing unit 230 performs the operation of setting the imaging mode during the light emission period of the pulse light emission and setting the temperature measurement mode during the light emission stop period of the pulse light emission while causing the light projector 14 to emit light in accordance with the light emission timing of the pulse light emission. It is a processing unit that repeats The light projector 14 performs intermittent light projection by performing pulse light emission. The integrated display processing section 230 has a pulse emission control section 241 , an imaging processing section 120 , a pulse emission stop control section 242 , a temperature measurement processing section 130 and an integrated display control section 243 .

The pulse light emission control unit 241 outputs a pulse light emission command to the light projection control circuit 73 and controls the light projector 14 to emit pulse light.

The imaging processing unit 120 has a wavelength selection unit 121, a turret control unit 122, and an imaging control unit 123. The functions of the wavelength selection unit 121, the turret control unit 122, and the imaging control unit 123 are the same as in the first embodiment. The imaging processing unit 120 performs imaging processing for obtaining a captured image by causing the image sensor 15 to capture visible light or near-infrared light.

The pulse emission stop control unit 242 outputs a pulse emission stop command to the light emission control circuit 73 and controls the light projector 14 to stop pulse emission.

The temperature measurement processing unit 130 has a wavelength selection unit 131, a first turret control unit 132, a first imaging control unit 133, a second turret control unit 134, a second imaging control unit 135, and a temperature derivation unit 136. The functions of the wavelength selection unit 131, the first turret control unit 132, the first imaging control unit 133, the second turret control unit 134, the second imaging control unit 135, and the temperature derivation unit 136 are the same as in the first embodiment. . The temperature measurement processing unit 130 calculates the temperature distribution of the subject based on the near-infrared light image obtained by capturing the near-infrared light by the image sensor 15, and obtains the temperature information based on the temperature distribution of the subject. Run the generated temperature measurement process.

The integrated display control unit 243 outputs an integrated image 251 that integrates the captured image obtained by the imaging processing unit 120 and the temperature information obtained by the temperature measurement processing unit 130, and displays the integrated image 251 on the display 76. The integrated image 251 may be an image obtained by synthesizing the captured image obtained by the imaging processing unit 120 with the temperature information obtained by the temperature measurement processing unit 130, or the captured image obtained by the imaging processing unit 120. and the temperature information obtained by the temperature measurement processing unit 130 may be displayed side by side. Further, the temperature information includes, for example, information indicating an area where the temperature is equal to or higher than a predetermined threshold, information indicating a specific numerical value of the temperature, and information indicating a plurality of sections divided for each predetermined temperature range. or information indicating the temperature distribution in a color tone corresponding to the temperature.

In the fifth embodiment, the captured image is an example of the “captured image” and the “first captured image” according to the technology of the present disclosure, and the temperature information is an example of the “temperature information” according to the technology of the present disclosure. . Also, the integrated image 251 is an example of a “composite image” according to the technology of the present disclosure.

Next, a method for controlling the camera 1 will be described as an operation of the fifth embodiment.

The fifth embodiment differs from the first embodiment in that the integrated display processing unit 230 executes integrated display processing. The integrated display processing executed by the integrated display processing unit 230 according to the fifth embodiment will be described below with reference to FIG. 29 .

In step S71, the pulse light emission control unit 241 causes the light projector 14 to emit pulse light.

In step S72, the imaging processing unit 120 obtains a captured image by causing the image sensor 15 to capture visible light or near-infrared light.

In step S73, the pulse emission stop control unit 242 causes the light projector 14 to stop pulse emission.

In step S74, the temperature measurement processing unit 130 calculates the temperature distribution of the subject based on the near-infrared light image obtained by causing the image sensor 15 to capture the near-infrared light, and calculates the temperature distribution based on the temperature distribution of the subject. generate information

In step S75, the integrated display control unit 243 outputs the integrated image 251 obtained by integrating the captured image obtained by the imaging processing unit 120 and the temperature information obtained by the temperature measurement processing unit 130, and displays the integrated image 251. 76.

The control method of the camera 1 according to the fifth embodiment is an example of the "control method" according to the technology of the present disclosure.

Next, the effects of the fifth embodiment that differ from those of the first embodiment will be described.

In the fifth embodiment, the light projector 14 performs pulsed light emission, and the CPU 61 sets the imaging mode during the light emission period of the pulsed light emission and sets the temperature measurement mode during the light emission stop period of the pulsed light emission. Repeat in time. Thereby, an integrated image can be obtained by integrating the captured image obtained in the imaging mode with the temperature information obtained in the temperature measurement mode.

The CPU 61 also outputs an integrated image 251 that integrates the captured image obtained by the imaging processing unit 120 and the temperature information obtained by the temperature measurement processing unit 130 . Therefore, even without switching between the imaging mode and the temperature measurement mode, the display 76 displays the integrated image 251 in which temperature information is integrated with the captured image, thereby allowing the user to visually understand the relationship between the subject's condition and the temperature. can be grasped.

Next, modifications common to the first to fifth embodiments will be described.

In the first to fifth embodiments, the CPU 61 has an imaging mode and a temperature measurement mode, but may have modes other than the imaging mode and the temperature measurement mode.

Further, in the first to fifth embodiments, the CPU 61 sets the display control flag 151 and the light emission control flag 152 differently between the imaging mode and the temperature measurement mode. Control flags other than 152 may be varied.

In addition, in the first to fifth embodiments, the display control flag 151 includes a captured image display flag 151A for displaying the captured image 161A on the display 76, and a synthesized image 161B obtained by synthesizing temperature information with the captured image. is set on the display 76 to display the composite image display flag 151B. However, a display control flag 151 other than the captured image display flag 151A and the composite image display flag 151B may be set. Also, in the temperature measurement mode, a temperature information display flag for displaying temperature information on the display 76 may be set instead of the composite image display flag 151B, and the temperature information may be displayed on the display 76 . A temperature information display flag is an example of a "temperature information display factor" according to the technology of the present disclosure.

Further, in the first to fifth embodiments, image blur is corrected by moving the blur correction lens 34. For example, the image sensor 15 is used as an example of the "optical element" according to the technology of the present disclosure. Image blur may be corrected by the movement. Image blurring may also be corrected by an image processing technique based on a plurality of captured images.

Further, in the first to fifth embodiments, in the temperature measurement by the two-color thermometry method, the wavelength band from 950 nm to 1100 nm, the wavelength band from 1150 nm to 1350 nm, the wavelength band from 1500 nm to 1750 nm, and the wavelength from 200 nm to 2400 nm Two wavelength bands are selected from the bands, but two wavelength bands may be selected from other wavelength bands.

Further, in the first to fifth embodiments, near-infrared light is used in the temperature measurement by the two-color thermometry method, but light other than near-infrared light such as visible light may be used. .

Further, in the first to fifth embodiments, the camera 1 is given as an example of an imaging device, but the technology of the present disclosure is not limited to this, and smart devices, wearable terminals, cell observation devices, and ophthalmologic observations are possible. It may be a digital camera built into a device or various electronic equipment such as a surgical microscope.

Also, in the first to fifth embodiments, the functional configuration of the CPU 61 and the order of the processes executed by the CPU 61 are examples, and may be modified in various ways.

Also, among the multiple technologies in the first to fifth embodiments, technologies that can be combined may be combined as appropriate.

Also, in the first to fifth embodiments, the example of the mode in which the computer 60 in the camera 1 executes the imaging support process has been described, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 30, the imaging support processing may be executed by a computer 314 in an external device 312 communicably connected to the camera 1 via a network 310 such as LAN or WAN. In the example shown in FIG. 30, computer 314 comprises CPU 316 , storage 318 and memory 320 . The storage 318 stores the imaging support processing program 100 .

The camera 1 requests execution of imaging support processing from the external device 312 via the network 310 . In response, the CPU 316 of the external device 312 reads the imaging support processing program 100 from the storage 318 and executes the imaging support processing program 100 on the memory 320 . The CPU 316 performs imaging support processing according to the imaging support processing program 100 executed on the memory 320 . Then, the CPU 316 provides the camera 1 via the network 310 with the processing result obtained by executing the imaging support processing.

Alternatively, the camera 1 and the external device 312 may perform the imaging support processing in a distributed manner, or a plurality of devices including the camera 1 and the external device 312 may perform the imaging support processing in a distributed manner. may Note that in the example illustrated in FIG. 30 , the camera 1 and the external device 312 are examples of the “imaging device” according to the technology of the present disclosure.

Also, in the first to fifth embodiments, the NVM 62 stores the imaging support processing program 100, but the technique of the present disclosure is not limited to this. For example, as shown in FIG. 31, the imaging support processing program 100 may be stored in the storage medium 330. FIG. Storage medium 330 is a non-temporary storage medium. An example of the storage medium 330 includes any portable storage medium such as an SSD or USB memory.

The imaging support processing program 100 stored in the storage medium 330 is installed in the computer 60 . The CPU 61 executes imaging support processing according to the imaging support processing program 100 .

In addition, the imaging support processing program 100 is stored in a storage unit such as another computer or server device connected to the computer 60 via a communication network (not shown), and the imaging support processing program is executed in response to a request from the camera 1. 100 may be downloaded and installed on computer 60 .

It should be noted that it is not necessary to store all of the imaging support processing program 100 in another computer connected to the computer 60, a storage unit such as a server device, or the NVM 62, and a part of the imaging support processing program 100 may be stored. You can leave it.

Further, although the example shown in FIG. 31 shows a mode example in which the computer 60 is built in the camera 1, the technology of the present disclosure is not limited to this. may be made available.

Also, in the example shown in FIG. 31, the CPU 61 is a single CPU, but may be a plurality of CPUs. Also, a GPU may be applied instead of the CPU 61 .

Also, although the computer 60 is illustrated in the example shown in FIG. good too. Also, instead of the computer 60, a combination of hardware configuration and software configuration may be used.

Various processors shown below can be used as hardware resources for executing the imaging support processing described in the first to fifth embodiments. As a processor, for example, there is a CPU, which is a general-purpose processor that functions as a hardware resource that executes imaging support processing by executing software, that is, a program. Also, processors include, for example, FPGAs, PLDs, ASICs, and other dedicated electric circuits that are processors having circuit configurations specially designed to execute specific processing. A memory is built in or connected to each processor, and each processor uses the memory to execute imaging support processing.

The hardware resource that executes the imaging support processing may be configured with one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or (combination of CPU and FPGA). Also, the hardware resource for executing the imaging support process may be one processor.

As an example of configuration with one processor, first, there is a form in which one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource for executing imaging support processing. . Secondly, as typified by SoC, etc., there is a form that uses a processor that realizes the function of the entire system including multiple hardware resources for executing imaging support processing with a single IC chip. In this way, the imaging support processing is implemented using one or more of the above-described various processors as hardware resources.

Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit in which circuit elements such as semiconductor elements are combined can be used. Also, the imaging support process described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps added, and the order of processing may be changed without departing from the scope of the invention.

The descriptions and illustrations shown above are detailed descriptions of the parts related to the technology of the present disclosure, and are merely examples of the technology of the present disclosure. For example, the above descriptions of configurations, functions, actions, and effects are descriptions of examples of configurations, functions, actions, and effects of portions related to the technology of the present disclosure. Therefore, unnecessary parts may be deleted, new elements added, or replaced with respect to the above-described description and illustration without departing from the gist of the technology of the present disclosure. Needless to say. In addition, in order to avoid complication and facilitate understanding of the portion related to the technology of the present disclosure, the descriptions and illustrations shown above require particular explanation in order to enable implementation of the technology of the present disclosure. Descriptions of common technical knowledge, etc., that are not used are omitted.

In this specification, "A and/or B" is synonymous with "at least one of A and B." That is, "A and/or B" means that only A, only B, or a combination of A and B may be used. Also, in this specification, when three or more matters are expressed by connecting with "and/or", the same idea as "A and/or B" is applied.

All publications, patent applications and technical standards mentioned herein are expressly incorporated herein by reference to the same extent as if each individual publication, patent application and technical standard were specifically and individually noted to be incorporated by reference. incorporated by reference into the book.

Claims

a processor;
a memory connected to or embedded in the processor;
The processor has a first mode for imaging based on light received by an image sensor of an imaging device, and a second mode for deriving temperature based on near-infrared light received by the image sensor,
A control device in which control factors are different between the first mode and the second mode.
The control device according to claim 1, wherein the control factor includes a display control factor displayed on a display.
The processor
In the first mode, as the display control factor, a captured image display factor for displaying on the display a captured image obtained by receiving the light by the image sensor is set;
3. The control device according to claim 2, wherein, in said second mode, a temperature information display factor for displaying temperature information indicating said temperature on said display is set as said display control factor.
the control factor includes a light projection control factor that operates a light projector;
The control device according to any one of claims 1 to 3, wherein in the second mode, the processor sets, as the light emission control factor, a light emission suppression control factor for suppressing light emission of the light projector. .
The control device according to any one of claims 1 to 4, wherein the control factors include imaging setting factors related to imaging settings.
The imaging settings include at least one of settings related to projector, settings related to shutter speed, settings related to aperture, settings related to photometry, settings related to sensitivity of the image sensor, settings related to high dynamic range, and settings related to image stabilization control. 6. The control device of claim 5, comprising:
The control device according to any one of claims 1 to 6, wherein the control factors include image processing setting factors related to image processing settings.
8. The control device of claim 7, wherein the image processing settings include at least one of noise reduction settings, sharpness settings, contrast settings, and tone settings.
a processor;
a memory connected to or embedded in the processor;
The processor
A first mode for imaging based on light received by an image sensor of an imaging device, and a second mode for deriving temperature based on near-infrared light received by the image sensor,
A controller for setting the first mode if the state of the light projector is on and setting the second mode if the state of the light projector is off.
The control device according to claim 9, wherein, in the second mode, the processor switches from the second mode to the first mode according to the temperature.
The light projector emits pulsed light,
10. The processor repeats an operation of setting the first mode during a light emission period of the pulse light emission and setting the second mode during a light emission stop period of the pulse light emission according to the light emission timing of the pulse light emission. The control device according to .
12. The processor according to any one of claims 1 to 11, wherein the processor outputs a synthesized image obtained by synthesizing a first captured image obtained by receiving light from the image sensor and the temperature information indicating the temperature. Control device as described.
A control device according to any one of claims 1 to 12;
the image sensor;
An imaging device comprising:
switching between a first mode of imaging based on light received by an image sensor of an imaging device and a second mode of deriving temperature based on near-infrared light received by the image sensor;
A control method comprising: making control factors different between the first mode and the second mode.
switching between a first mode of imaging based on light received by an image sensor of an imaging device and a second mode of deriving temperature based on near-infrared light received by the image sensor;
setting the first mode when the operation of the light projector is on, and setting the second mode when the operation of the light projector is off.
to the computer,
switching between a first mode of imaging based on light received by an image sensor of an imaging device and a second mode of deriving temperature based on near-infrared light received by the image sensor;
A program for executing processing including different control factors between the first mode and the second mode.
to the computer,
switching between a first mode of imaging based on light received by an image sensor of an imaging device and a second mode of deriving temperature based on near-infrared light received by the image sensor;
Setting the first mode when the operation of the light projector is on, and setting the second mode when the operation of the light projector is off.