JP2022159983A - Illumination control device, imaging apparatus, and computer program - Google Patents

Abstract

To achieve an illumination control device or the like capable of achieving both color reproducibility and S/N of an image in infrared illumination.SOLUTION: The Illumination control device includes a visible light measuring unit for measuring the amount of visible light components, an infrared illumination unit for irradiating infrared illumination, and a control unit for gradually changing the intensity of the infrared illumination by the infrared illumination unit in accordance with a reduction in the amount of the visible light component measured in the visible light measuring unit from a first threshold value to a second threshold value.SELECTED DRAWING: Figure 3

Description

The present invention relates to a lighting control device and the like capable of infrared lighting.

2. Description of the Related Art Conventionally, there are models equipped with a night mode in which an infrared cut filter (hereinafter referred to as an IRCF) is removed from the imaging optical path for low-illumination photography in a surveillance camera or the like. In addition, in a dark environment where there is almost no outside light or illumination, the operation of monitoring cameras, monitoring systems, etc. that capture images while using infrared lighting is also widely used. For the same reason, there are cases where visible light illumination is used together, but there are cases where visible light illumination cannot be used depending on the object to be monitored or the installation environment, and under such circumstances, infrared illumination is utilized.

In the night mode of general surveillance cameras, etc., the IRCF is removed from the shooting optical path, and the infrared component is captured. In most cases, a black-and-white image is output by On the other hand, there is a user's desire to distinguish the color of an object even in a dark environment where there is almost no visible light.

For example, Patent Literature 1 discloses a technique of dividing an image into a plurality of areas and controlling a light source unit for areas where the amount of invisible light (infrared light) is equal to or less than a predetermined amount. Further, Patent Document 2 discloses a technique for adjusting image quality according to the ratio of visible light and non-visible light.

Japanese Patent Application No. 2017-63362 JP 2017-5484 A

However, the technical content of Patent Document 1 does not consider the color components of the photographed image when the invisible light is irradiated, so there is a problem that the color reproduction of the subject deteriorates. Further, according to the content of Patent Document 2, the color of the subject cannot be reproduced when the non-visible light is irradiated.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an illumination control apparatus and the like that can achieve both the color reproducibility of an image and the S/N ratio during infrared illumination.

In the lighting control device,
a visible light measurement unit that measures the amount of visible light components;
an infrared illumination unit that emits infrared illumination;
Control for gradually changing the intensity of the infrared illumination by the infrared illumination unit in response to the amount of visible light components measured by the visible light measurement unit decreasing from a first threshold to a second threshold. and a part.

Advantageous Effects of Invention According to the present invention, it is possible to realize an illumination control device and the like that can achieve both color reproducibility and S/N of an image when illuminated with infrared light.

1 is a block diagram showing the configuration of a lighting control device according to a first embodiment; FIG. 5 is a flowchart showing processing of the lighting control device according to the first embodiment; 4 is a graph showing the relationship between the threshold of visible light and the intensity of infrared illumination in Example 1. FIG. FIG. 11 is a block diagram showing the configuration of an imaging device according to a second embodiment; FIG. FIG. 11 is a block diagram showing the configuration of an imaging unit according to the second embodiment; 10 is a flow chart showing processing of the imaging device according to the second embodiment; 10 is a graph of distribution of color difference components when a predetermined subject is imaged with sufficient visible light in Example 2. FIG. 10 is a graph of the distribution of color difference components when a predetermined subject is imaged under a condition in which the illumination intensity of infrared light is dominant over that of visible light in Example 2. FIG. 7 is a graph showing the spectral sensitivity of a general color image sensor according to Example 2; 7 is a graph showing the relationship between the threshold of visible light and the intensity of infrared illumination in Example 2. FIG. 11 is a table showing states of the IRCF 501, the infrared illumination unit 100, and the output unit 405 in each of ranges A to D in the graph of FIG. 10; FIG. 11 is a block diagram showing the configuration of an imaging system according to Example 3; 11 is a flow chart showing processing of an imaging system according to Example 3. FIG. 14 is a flow chart showing another part of the processing of the imaging system according to Example 3. FIG. FIG. 13 is a diagram showing an example of each mode that can be set by the operation unit 1203 of the client device 1200 according to the third embodiment; 16 is a diagram showing an example of a combination state of IRCF, infrared illumination, and image output in each mode of FIG. 15; FIG. FIG. 11 is a diagram showing an example of a GUI for setting illuminance when automatically inserting and removing an IRCF according to the third embodiment; FIG. 11 is a diagram illustrating an example of a GUI for setting priorities when correcting illumination intensity for automatic dimming of infrared illumination according to Example 3; FIG. 11 is a diagram illustrating an example of a GUI for setting priority of a white balance control range according to the third embodiment; 20 is a graph showing an example of the white balance control range set in FIG. 19 according to Example 3;

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the present invention will be described below using examples with reference to the accompanying drawings. In each drawing, the same members or elements are denoted by the same reference numerals, and overlapping descriptions are omitted or simplified.

Also, in the embodiment, an example in which the imaging apparatus is applied to a network camera will be described. However, imaging devices include digital still cameras, digital movie cameras, smartphones with cameras, tablet computers with cameras, and electronic devices with imaging functions such as in-vehicle cameras.

First, the configuration and processing of the lighting control device according to the first embodiment will be described with reference to FIGS. 1 to 3. FIG.
FIG. 1 is a block diagram showing the configuration of a lighting control device according to a first embodiment. The illumination control apparatus of this embodiment is used in combination with a network camera as an example of an imaging apparatus, and is composed of an infrared illumination unit 100, an illumination control unit 101, a visible light measurement unit 102, a determination unit 103, a storage unit 104, and the like. It is

The infrared illumination unit 100 performs infrared illumination by emitting infrared light (for example, light having infrared spectral characteristics of approximately 800 nm to approximately 1000 nm). The illumination control unit 101 controls illumination intensity of the infrared illumination unit 100 . In addition to the illumination intensity of the infrared illumination unit 100, the illumination control unit 101 controls various controllable parameters such as illumination intensity change timing, illumination intensity change speed, and illumination resolution.
The visible light measurement unit 102 measures the imaging environment, particularly the visible light component amount (visible light intensity) of the subject. The determination unit 103 determines the visible light intensity and the like measured by the visible light measurement unit 102, and controls the emission intensity and the like of the infrared illumination unit 100 via the illumination control unit 101 based on the determination result.

The determination unit 103 and the like have a CPU as a computer, and function as control means (control unit) for controlling the operation of each unit of the entire apparatus based on the computer program stored in the storage unit 104 . Furthermore, the determination unit 103 is configured to transmit a control signal to the imaging device. In addition to the computer program, the storage unit 104 stores a plurality of illumination control patterns prepared in advance for controlling the intensity of the infrared illumination unit 100 by the illumination control unit 101, a predetermined threshold value used for determination, and the like. .

Next, FIG. 2 is a flowchart showing processing of the lighting control device according to the first embodiment, and the flow of processing of the lighting control device will be described using the flowchart of FIG. 2 is performed by a computer such as the determination unit 103 executing a computer program stored in the storage unit 104. FIG. Also, it is assumed that the imaging operation in the imaging apparatus is being performed during the operation of the flowchart shown in FIG.

First, in step S201, the visible light measurement unit 102 measures the amount of visible light components. The visible light measurement unit is configured to measure the illuminance (lux) of the environment using, for example, an illuminance sensor having sensitivity in the visible light wavelength region. The direction and range measured by the visible light measurement unit 102 are set to be substantially the same direction or range as the irradiation direction of the infrared illumination unit 100, and the direction and range are the imaging directions of the imaging lens and the imaging device. and the imaging range.
In step S202, the determination unit 103 determines whether the amount of the visible light component measured in step S201 is equal to or less than the first threshold.

Here, the first threshold is set to a value corresponding to a predetermined amount (brightness) of the visible light component. corresponds to the brightness at which the S/N ratio of the image deteriorates. Further, when the value becomes equal to or less than the first threshold value, the control signal instructs the imaging device to retract the IRCF (infrared cut filter) on the imaging optical path of the imaging lens and to generate a color image in the imaging device. to send. Note that the first threshold may be arbitrarily set by the user.
If the amount of the visible light component is greater than the first threshold, the illumination control unit 101 turns off illumination by the infrared illumination unit 100 (step S203).

If it is determined in step S202 that the amount of the visible light component is equal to or less than the first threshold, the determination unit 103 determines whether or not the amount of visible light is equal to or greater than the second threshold in step S204. In step S204, if the amount of visible light measured by the visible light measurement unit 102 is smaller than the second threshold, in step S205, the illumination intensity of the infrared illumination is increased to the output limit (upper limit), and the imaging device side sends a control signal to generate a black-and-white image. Here, the second threshold value is set to a predetermined brightness at which, for example, the subject cannot be recognized on the image or the color information of the image cannot be identified when color photography is performed by an imaging device for monitoring purposes. Alternatively, the user may arbitrarily set it.

If Yes in step S204, the process proceeds to step S206. In step S<b>206 , the illumination control unit 101 controls the intensity of the infrared illumination by the infrared illumination unit 100 so as to have a predetermined ratio with respect to the amount of the visible light component measured by the visible light measurement unit 102 . At this time, various parameters that can be controlled by the illumination control unit 101, such as change timing, change speed, and illumination resolution when changing the illumination intensity, may be changed. A detailed control example will be described with reference to FIG.

Here, FIG. 3 is a graph showing the relationship between the visible light threshold value and the infrared illumination intensity according to the first embodiment. That is, FIG. 3 shows an example of the first and second thresholds to be determined by the determination unit 103 in steps S202 and S204 of the flowchart of FIG. 2 and the intensity of the infrared illumination controlled in step S206. ing. Also, in FIG. 3, the upper limit of the output when the illumination control unit 101 performs control in step S205 is indicated by a horizontal dashed line. The first threshold is greater than the second threshold. In other words, there is a relatively large number of visible light components (brightness). Note that the upper limit of the illumination intensity of the infrared illumination may be changeable.

Further, the infrared illumination intensity controlled by the illumination control unit 101 in step S206 increases between the first threshold value and the second threshold value as the visible light component decreases (becomes darker). Control in the direction of weakening the outside illumination intensity. Here, in step S206, the intensity of the infrared illumination by the infrared illumination unit is adjusted as the visible light component amount measured by the visible light measurement unit decreases from the first threshold to the second threshold. It functions as a control unit that gradually changes (decreases) the

That is, in step S206, control is performed so that the ratio of the infrared light component and the visible light component is substantially constant. In that case, it is desirable that the infrared light component is larger than the visible light component. Therefore, it is possible to obtain an image with well-balanced S/N and color reproducibility. Note that it is desirable to perform control to emphasize the saturation of the color image generated by the imaging device when the visible light component is between the first threshold and the second threshold.

Furthermore, the distance to an object existing in the irradiation direction of the infrared illumination unit 100 may be measured using a distance measurement unit, and the infrared illumination intensity offset amount may be controlled according to the measured object distance. That is, for example, even if the inclination of the infrared illumination intensity from the first threshold value to the second threshold value is maintained as indicated by the dashed line in FIG. good. Also, the offset amount may be adjustable by the user while maintaining the gradient characteristic of the infrared illumination intensity between the first threshold and the second threshold.

Furthermore, the output of the infrared illumination unit during control by the illumination control unit 101 may be corrected according to the above-described infrared reflection characteristics of the subject. For example, the output of the infrared lighting unit controlled by the lighting control unit 101 may be controlled to relatively weaken as the infrared reflectance of the subject increases. Further, a plurality of illumination control patterns (variation patterns of infrared illumination intensity between the first threshold value and the second threshold value) in the illumination control unit 101 may be provided in advance and stored in the storage unit 104 in advance.

As described above, in this embodiment, an image is captured while changing the intensity of infrared light in accordance with a decrease in the visible light component in the brightness range from the first threshold to the second threshold. there is Therefore, in the brightness range from the first threshold to the second threshold, visible information (color information) can be obtained more effectively than a conventional imaging device using infrared illumination that only turns on and off the infrared light. You can increase the amount you get.

Furthermore, by irradiating the infrared light at the stage when the first threshold value is not reached, the S/N can be improved by the infrared sensitivity of the imaging element at the relatively dark stage. Therefore, both the color reproducibility and the S/N ratio of the captured image can be achieved. Furthermore, in the brightness range from the first threshold to the second threshold, the balance between S/N and color reproducibility can be achieved by imaging while decreasing the intensity of infrared light according to the decrease of the visible light component. can be properly maintained.

A second embodiment will be described below with reference to FIGS. FIG. 4 is a block diagram showing the configuration of an imaging device according to the second embodiment, and FIG. 5 is a block diagram showing the configuration of an imaging unit 400 according to the second embodiment.
As for 100 to 104 in FIG. 4, the same processes as those with the same numbers in FIG.

An imaging unit 400 in the imaging device 40 captures an image by taking in light from the outside. The imaging unit 400 includes an imaging lens 500 including a plurality of lens groups including a zoom lens and a focus lens, an IRCF 501 that can be inserted into and removed from the imaging optical path of the imaging lens 500, and an imaging element 502 such as a CCD or CMOS.

For example, one of R, G, and B color filters is arranged in front of each pixel of the image sensor 502 . Further, the R, G, and B color filters are arranged in a so-called Bayer arrangement or the like, in which R, G, and B are alternately arranged at predetermined intervals. Therefore, by sequentially reading out the signals of the respective pixels of the imaging element, the R, G, and B color signals are periodically read out in a predetermined order. The imaging unit 400 also includes a CDS circuit 503 that performs correlated double sampling (CDS) for noise reduction.

The imaging unit 400 also includes an AGC (Automatic Gain Control: AGC) amplifier 504 that automatically performs gain control of the signal obtained from the camera. The imaging unit 400 also includes an A/D conversion circuit 505 that converts an analog signal into a digital signal.

An imaging device 502 converts a subject image formed through an imaging lens 500 as an imaging optical system into an electrical signal, and a CDS circuit 503 performs correlated double sampling on the electrical signal output from the imaging device 502. Take action. The AGC amplifier 504 performs automatic gain control on the electrical signal output from the CDS circuit 503, and the A/D conversion circuit 505 converts the analog signal automatically gain-controlled by the AGC amplifier 504 into a digital signal.

The IRCF driving unit 401 moves the IRCF 501 in the imaging unit 400 in a direction perpendicular to the imaging optical path of the imaging lens 500 to put it in a retracted state or an inserted state with respect to the imaging optical path. With the IRCF 501 retracted, in order to make the optical path length the same as when the IRCF 501 is inserted, a glass plate or the like that can transmit light from the visible component to the infrared component may be inserted into the optical path. The exposure control unit 402 controls exposure based on the exposure time (accumulation time) of the image pickup device 502 in the image pickup unit 400 and exposure parameters such as gain and aperture value.

The determination unit 103 uses the exposure control unit 402 to estimate the brightness (illuminance) of the subject from the exposure amount (aperture value and exposure time after exposure control), gain, image luminance information, etc. measurement) to determine whether the IRCF 501 should be inserted or removed. That is, in this case, the exposure control unit 402 and the determination unit 103 function as a visible light measurement unit that measures the visible light component amount based on the information acquired from the imaging unit. Then, according to the determination result, a control signal is sent to the IRCF drive unit 401 to control the insertion/removal of the IRCF 501 . The image processing unit 403 performs image processing such as gamma correction, white balance processing, edge enhancement, saturation adjustment, contrast adjustment, and noise reduction processing on the captured image controlled by the exposure control unit 402 .

Here, the determination unit 103 may use the IRCF driving unit 401 to switch the image to a color image or a black-and-white image by the image processing unit 403 according to control of insertion/retraction of the IRCF 501 and brightness threshold. That is, when the IRCF 501 is inserted into the imaging optical path, the color image is displayed, and when the IRCF 501 is retracted from the imaging optical path and the brightness (illuminance) falls below the second threshold value, the image can be switched to the black-and-white image. good.

A calculation unit 404 calculates values related to luminance and color components for the image after image processing by the image processing unit 403 . For example, it is the average luminance value of the entire image, the color difference value of each predetermined divided area of the image, and the like. An output unit 405 outputs an image after the calculation unit 404 has calculated values related to luminance and color components.

The illumination control device having the infrared illumination unit 100 and the illumination control unit 101 may be provided inside the housing of the imaging device 40 or may be separate from the housing of the imaging device. Moreover, the wavelength component of the irradiated infrared light has a wavelength component within the range of approximately 800 nm to 1000 nm to which the imaging element 502 is sensitive.

FIG. 6 is a flowchart showing processing of the imaging apparatus according to the second embodiment, and the flow of processing in the imaging apparatus of FIG. 4 will be described using the flowchart of FIG. 6 while referring to FIGS. 4 to 11. FIG. 6 is performed by a computer such as the determination unit 103 executing a computer program stored in the storage unit 104. FIG.

First, in step S601, a color image is generated through the imaging unit 400, the exposure control unit 402, and the image processing unit 403. FIG. In step S602, the brightness (illuminance) of the subject in the shooting environment is estimated (measured). At this time, the illuminance may be estimated (measured) from at least one of the aperture, exposure time, and gain controlled by the exposure control unit 402, or the average luminance value of the image calculated by the calculation unit 404, or the like. Alternatively, an illuminance sensor such as the visible light sensor described in the first embodiment may be used to measure (estimate) the illuminance.

The determination unit 103 determines whether or not the illuminance estimated (measured) in step S602 is equal to or less than a predetermined value (step S603). In step S603, if the determination unit 103 determines that the illuminance is greater than the predetermined first threshold value, imaging is performed with the IRCF 501 in the imaging unit 400 inserted into the optical path (step S604). At this time, the image is normally output in color, but other output formats may be used (step S605). On the other hand, in step S603, if the determination unit 103 determines that the illuminance is equal to or less than the predetermined (first threshold), the IRCF 501 measures the color difference component (first color difference component) of the image with the IRCF 501 inserted into the optical path ( step S606).

Here, FIG. 7 is a graph of the distribution of color difference components when a predetermined subject is imaged with sufficient visible light in Example 2. In FIG. That is, FIG. 7 shows an image of a predetermined subject captured by the imaging unit 400 in a bright imaging environment with sufficient visible light, with the IRCF 501 inserted in the optical path. 4 is an example of a graph showing the distribution of color difference (RY, BY) components for each region; The vertical axis is the RY axis, and the horizontal axis is the BY axis.

On the other hand, FIG. 8 is a graph of the distribution of color difference components when a predetermined subject is imaged under conditions where the illumination intensity of infrared light is dominant over that of visible light in Example 2. In FIG. That is, FIG. 8 shows the same subject as in FIG. 7 captured by the imaging unit 400 under the condition that the illumination intensity of infrared light is dominant over that of visible light, with the IRCF 501 retracted from the imaging optical path. It is a graph showing the distribution of color difference components.

In the case of FIG. 7, the color differences are uniformly distributed, whereas in FIG. 8 they converge on a local region near the center. This is because, in the spectral sensitivity characteristics of a general image sensor as shown in FIG. 9, the sensitivity is almost the same on the longer wavelength side than near 800 nm regardless of the color. Note that FIG. 9 is a graph showing the spectral sensitivity of a general color imaging device according to the second embodiment.

That is, as the intensity of infrared light relative to visible light increases, the color difference distribution tends to concentrate near the center and the color difference components tend to decrease (become achromatic). Based on the change characteristic of the color difference component of the image with respect to the change in the ratio (ratio) of the intensity of the visible light component and the infrared light component, the IRCF 501 of the imaging unit 400 is removed from the optical path, and the infrared illumination for the visible light component It is possible to estimate the relative strength of

Various techniques are known for estimating the ratio of the visible light component and the infrared component from the color difference components of the image, and other known techniques may be used.
After the first color difference component is measured with the IRCF 501 inserted in step S606, the IRCF 501 is retracted from the optical path using the IRCF drive unit 401 in step S607. The color difference component (first color difference component) measured in step S606 may be, for example, an average value of RY and BY obtained from a plurality of divided areas of the screen.

After that, in step S608, the determination unit 103 determines whether or not the first color difference component measured in step S606 is greater than or equal to the fourth threshold. Here, the fourth threshold is a predetermined brightness (the second threshold in the first embodiment) at which the subject cannot be recognized in the image or the color information cannot be identified from the image, for example, when the imaging device 40 performs color photography. is the color difference value corresponding to the threshold value of Note that the fourth threshold may be arbitrarily set by the user. Also, the fourth threshold value may be a value of a color difference component that is about the color discrimination limit of a color discriminator or the human eye when saturation is emphasized in step S612, which will be described later.

If it is determined in step S608 that the first color difference component is less than the fourth threshold, the upper limit of the infrared illumination is raised to the output upper limit (step S613). At this time, the illumination control unit 101 does not necessarily have to forcibly control the intensity of the infrared illumination unit 100 up to the output upper limit. Further, the upper limit of the illumination intensity may be changed according to subject conditions such as subject distance, reflection characteristics of the subject, influence of shielding objects, and the like.

In addition, since there are few color difference components in the image at the time of inserting the IRCF 501 (substantially achromatic), it can be estimated that the subject itself is substantially achromatic. step S614). On the other hand, if the first color difference component is equal to or greater than the fourth threshold in step S608, the calculation unit 404 calculates (measures) the color difference component (second color difference component) while the IRCF 501 is retracted from the optical path in step S609. )do.

The color difference component (second color difference component) measured in step S609 may be, for example, an average value of RY and BY obtained from a plurality of divided areas of the screen. Next, in step S610, the determination unit 103 determines whether or not the second color difference component calculated in step S609 is equal to or less than the third threshold and equal to or greater than the fourth threshold.

Here, if the third threshold is equal to or less than the third threshold, the brightness (the first threshold in the first embodiment) deteriorates the S/N ratio of an image when color photographing is performed by an imaging device for monitoring purposes, for example. ) is the color difference value corresponding to Further, when it becomes equal to or less than the third threshold, the IRCF 501 is retracted from the imaging optical path, but color image generation is continued. Note that the third threshold may be arbitrarily set by the user. Also, the third threshold value or the fourth threshold value may be corrected according to the amount of the first color difference component with the IRCF 501 inserted. For example, the fourth threshold may be lowered as the first color difference component amount increases. This is because, in such a case, it is considered that the saturation of the subject is originally high.

If it is determined in step S610 that the second color difference component is greater than the third threshold or less than the fourth threshold, priority is given to sensitivity and the upper limit of the infrared illumination is raised to the output upper limit (step S613). . Also, the S/N is prioritized and a black-and-white image is output (step S614).

On the other hand, if it is determined as Yes in step S610, the illumination intensity of the infrared illumination unit 100 is set as indicated by the solid line between the third threshold and the fourth threshold in FIG. 10 according to the second color difference component. (step S611).
Here, FIG. 10 is a graph showing the relationship between the visible light threshold and the infrared illumination intensity in Example 2. In FIG. 11 is a table showing the states of the IRCF 501, the infrared illumination unit 100, and the output unit 405 in each of ranges A to D in the graph of FIG.

At this time, a control pattern is employed in which the timing of starting irradiation of the infrared illumination unit 100 is shifted in the direction in which the color difference component is reduced while maintaining the inclination of the part of the solid line portion as shown by the dashed-dotted line in FIG. The control pattern may be stored in advance in the storage unit 104 and used. Here, the one-dot chain line control pattern is an example of a control pattern that emphasizes color reproducibility rather than S/N as compared to the solid line control pattern.

Furthermore, as another control pattern, the infrared illumination intensity may be controlled upward to the right in the range B of FIG. In that case, color reproducibility is improved. These control patterns and a control pattern that emphasizes S/N over color reproducibility (pattern of solid line 10 in FIG. 10) are stored in storage unit 104 in advance. The user may select which control pattern to use. Also, the third threshold and the fourth threshold in FIG. 10 may be adjusted by the user. Further, the offset amount may be adjustable by the user while maintaining the gradient characteristics of the infrared illumination intensity in the ranges B and C.

Furthermore, in this embodiment, since the color tone of the image tends to become lighter due to the influence of the infrared illumination, the image processing unit 403 enhances the saturation of the image (step S612). Then, in steps S611 and S612, the illumination control unit 101 controls the illumination intensity of the infrared illumination unit 100 between the third threshold value and the fourth threshold value in FIG. It is emphasized and output as a color image in step S605.

As described above, in the second embodiment as well, when photographing a subject with an IRCF insertable/removable imaging device and an imaging device capable of controlling the intensity of infrared illumination, particularly in the range of the color difference between the third threshold and the fourth threshold, It is possible to increase the amount of visible information (color information) to be acquired compared to the conventional art.
Furthermore, by irradiating infrared light when the color difference falls below the third threshold, it is possible to improve the S/N by the infrared sensitivity of the imaging device in a relatively dark stage. Therefore, it is possible to maintain the color reproducibility and S/N of the captured image.

FIG. 12 is a block diagram showing the configuration of an imaging system according to the third embodiment. The configurations of the imaging device 40 and the infrared illumination unit 100 in FIG. 12 perform the same processes as those described with reference to FIG. 4, so description thereof will be omitted. Reference numeral 1200 in FIG. 12 denotes a client device, which has at least a communication control section 1201, an input image acquisition section 1202, and an operation section 1203. FIG. Also, the client device 1200 may include components included in a general-purpose personal computer (not shown).

A communication control unit 1201 in FIG. 12 controls various communications when the client device 1200 and the imaging device 40 are connected directly or via a network. A communication unit (not shown) is provided as communication means in the imaging device 40, and communication with the communication control unit 1201 is possible. Also, the communication control unit 1201 may communicate directly with the infrared lighting unit 100 or may communicate with the infrared lighting unit 100 via the imaging device 40 . For example, when a command for executing a command for each process of the imaging device 40 is transmitted to the imaging device 40, and the communication unit in the imaging device 40 receives the command from the outside via the network, Applies to each treatment.

An input image acquisition unit 1202 in FIG. 12 acquires an image output from the imaging device 40 and displays the image on, for example, a display unit (not shown). An operation unit 1203 in FIG. 12 generates and changes each command to be transmitted to the imaging device 40 on a GUI (graphic user interface) displayed on a display unit (not shown).

FIG. 13 is a flowchart representing processing of the imaging system according to the third embodiment, and FIG. 14 is a flowchart representing another part of the processing of the imaging system according to the third embodiment. The flow of processing in the imaging apparatus of FIG. 4 will be described using flowcharts of FIGS. 13 and 14 while referring to FIGS. 4 to 20. FIG. 13 and 14 are performed by a computer such as the determination unit 103 executing a computer program stored in the storage unit 104, a program for controlling each unit on the client device 1200, and the like.

Further, steps S601 to S614 in the flowcharts in FIGS. 13 and 14 are the same as those in the flowchart in FIG.
In step S1301, it is determined whether or not the current mode is the "automatic switching mode (illuminance interlocking)". Here, FIG. 15 is a diagram showing an example of each mode that can be set by the operation unit 1203 of the client device 1200 according to the third embodiment.

In the example of FIG. 15, the user can select one mode from "automatic switching mode", "infrared color mode", "color mode", and "black and white mode". Also, when the "automatic switching mode" is selected, it is possible to switch to "lighting interlocking" by clicking the check box of "lighting interlocking" for selecting whether or not to interlock with infrared lighting. Also, FIG. 16 is a diagram showing an example of a combination state of IRCF, infrared illumination, and image output in each mode of FIG. As described above, in this embodiment, a plurality of modes with different combinations of the control state of the infrared illumination, the insertion/removal state of the infrared cut filter with respect to the imaging optical path of the imaging lens, and the image output state are stored, and the mode can be selected. there is

In step S1301, if the "automatic switching mode (linked to illumination)" is not selected, it is determined whether or not the "automatic switching mode" is selected (step S1302).
If it is determined in step S1302 that the "automatic switching mode" has not been selected, then one of the "infrared color mode", "color mode", and "monochrome mode" has been selected. Therefore, a combination pattern of IRCF, infrared illumination, and image output corresponding to each mode in FIG. 16 is output (step S1303).

On the other hand, if it is determined in step S1302 that the "automatic switching mode" has been selected, insertion/removal of the IRCF and switching of image output are automatically performed according to the illuminance (step S1304). Automatic insertion and removal of the IRCF is performed according to the illuminance of the shooting environment. When the brightness is brighter than a predetermined threshold, a color image is output with the IRCF inserted. may be taken in. Further, the image output when the IRCF is removed may be black and white or color. After executing the processing of steps S1303 and S1304, the flow of FIGS. 13 and 14 ends.

If the "automatic switching mode (linked to illuminance)" is selected in step S1301, the IRCF switching illuminance specified by the user on the GUI as shown in FIG. 17 is obtained through illuminance estimation in step S602. (Step S1305). FIG. 17 is a diagram showing an example of the GUI for setting the illuminance when automatically inserting and removing the IRCF according to the third embodiment. If the direction is set, it is possible to relatively brighten the brightness for inserting and removing the IRCF. That is, in this embodiment, the user can select the brightness for inserting or retracting the infrared cut filter.

Next, through steps S603 to S610, the priority of illumination intensity correction of infrared illumination specified by the user on the GUI as shown in FIG. 18 is obtained (step S1306). FIG. 18 is a diagram illustrating an example of a GUI for setting priorities when correcting illumination intensity for automatic dimming of infrared illumination according to the third embodiment. In FIG. 18, the infrared illumination can be set relatively weak to give priority to color reproduction in the left direction, and the infrared illumination can be set to be relatively strong in order to give priority to sensitivity in the right direction.

Specifically, regarding the irradiation intensity of the infrared illumination between the range BC of the graph in FIG. 10 of Example 2, when the GUI of FIG. Correction is made in the direction of weakening the illumination intensity (for example, the dashed-dotted line curve). Also, when the sensitivity is prioritized, the intensity of the infrared illumination between the range BC can be corrected to be stronger (for example, the curve of the solid line). That is, in this embodiment, the intensity of the infrared illumination can be corrected according to the priority set by the user.

Next, through step S611, the control range of white balance specified by the user on the GUI as shown in FIG. 19 is acquired (step S1307). FIG. 19 is a diagram illustrating an example of a GUI for setting the priority of the white balance control range according to the third embodiment. In FIG. 19, sliding to the left relatively narrows the white balance control range to emphasize color, and sliding to the right sets the white balance control range relatively wide to emphasize white. can be done.

Specifically, when performing general auto white balance control, the control range of white balance processing in the image processing unit 403 is changed as shown in the graph of FIG. FIG. 20 is a graph showing an example of the white balance control range set in FIG. 19 according to the third embodiment. In FIG. 20, to relatively widen the white balance control range, the white balance gains (R gain and B gain) are controlled within the range 1901, for example. When narrowing the white balance control range, the white balance gain is controlled within the range 1902, for example.

In the above description, the image processing unit 403 functions as white balance control means, and the user can select the white balance control range of the white balance control means through the GUI of FIG. That is, the imaging device 40 can receive a command from the outside via a network and perform at least one control of the intensity of the infrared illumination, insertion/removal of an infrared cut filter in the imaging optical path of the imaging lens, and white balance control. .

As described above, in the third embodiment, it is possible to increase the amount of visible information (color information) to be acquired and to obtain the image quality desired by the user, particularly in the range of the color difference between the third threshold and the fourth threshold. , it is possible to output an image while reflecting it more.

The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention. They are not excluded from the scope of the invention.
It should be noted that a computer program that implements the functions of the above-described embodiments for part or all of the control in this embodiment may be supplied to the lighting control device, imaging device, or the like via a network or various storage media. The computer (or CPU, MPU, etc.) in the lighting control device, imaging device, or the like may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention.

100 infrared illumination unit 101 illumination control unit 102 visible light measurement unit 103 determination unit 104 storage unit

Claims (26)

a visible light measurement unit that measures the amount of visible light components;
an infrared illumination unit that performs infrared illumination;
Control for gradually changing the intensity of the infrared illumination by the infrared illumination unit in response to the amount of visible light components measured by the visible light measurement unit decreasing from a first threshold to a second threshold. and a lighting control device. The control unit gradually reduces the intensity of the infrared illumination by the infrared illumination unit as the visible light component amount decreases from the first threshold value to the second threshold value. The lighting control device according to claim 1. 2. The lighting control device according to claim 1, wherein the visible light measurement unit includes an illuminance sensor. 2. The illumination control device according to claim 1, wherein an upper limit of illumination intensity of said infrared illumination unit can be changed. 5. The method according to claim 4, wherein when the amount of visible light component measured by the visible light measurement unit is smaller than the second threshold value, the illumination intensity of the infrared illumination unit is increased to the upper limit. lighting controller. The control unit has a plurality of control patterns in advance for controlling the illumination intensity of the infrared illumination unit as the visible light component amount decreases from the first threshold value to the second threshold value. 2. A lighting control device according to claim 1. 7. The lighting control device according to claim 6, wherein the control pattern can be selected by a user. 2. The illumination control device according to claim 1, wherein the amount of offset of the intensity of the infrared illumination is adjustable within the range of the visible light component amount from the first threshold to the second threshold. Further equipped with a distance measuring unit that measures the distance to the subject,
9. The lighting control apparatus according to claim 8, wherein the control section changes the offset amount according to the distance to the subject measured by the distance measuring section. An imaging device combined with the lighting control device according to claim 1,
having an imaging lens and an imaging unit,
The imaging apparatus according to claim 1, wherein the visible light measuring section measures the amount of visible light component based on information acquired from the imaging section. The visible light measurement unit measures the visible light component amount from at least one of an exposure amount of the imaging unit, luminance information of the image acquired from the imaging unit, and a gain of the imaging unit. The imaging device according to claim 10. 11. The imaging apparatus according to claim 10, further comprising an infrared cut filter that is inserted into or retracted from an imaging optical path of said imaging lens. 13. The imaging apparatus according to claim 12, wherein the infrared cut filter is retracted from the imaging optical path when the visible light component amount is equal to or less than the first threshold. A calculation unit that calculates color difference components from the image acquired from the imaging unit,
The calculation unit calculates a first color difference component from the image with the infrared cut filter inserted in the imaging optical path,
13. The imaging apparatus according to claim 12, wherein the second color difference component is calculated from the image with the infrared cut filter retracted from the imaging optical path. The imaging device generates a color image with the infrared cut filter inserted into the imaging optical path, and generates the first color difference component, the second 15. The imaging apparatus according to claim 14, wherein the color image or the black-and-white image is generated according to the color difference components of , and the intensity of the infrared illumination emitted by the infrared illumination section. 16. The imaging apparatus according to claim 15, wherein the intensity of the infrared illumination unit is increased to an output upper limit when the first color difference component or the second color difference component is less than a fourth threshold. . 17. The imaging apparatus according to claim 16, wherein the image is switched to the monochrome image when the first color difference component or the second color difference component is less than the fourth threshold. When the second color difference component is greater than the fourth threshold and equal to or less than a third threshold, and when the first color difference component and the second color difference component are equal to or greater than the fourth threshold, 18. The image pickup apparatus according to claim 17, wherein the control section controls to lower the intensity of the infrared illumination section as the amount of the visible light component decreases. When the second color difference component is greater than the fourth threshold and equal to or less than a third threshold, and when the first color difference component and the second color difference component are equal to or greater than the fourth threshold, 18. The imaging apparatus according to claim 17, further comprising an image processing section that enhances saturation of an image. Furthermore, a white balance control means is provided,
20. The imaging apparatus according to any one of claims 10 to 19, wherein the white balance control range of said white balance control means is selectable. Furthermore, having a means of communication,
By receiving commands from the outside via the network by the communication means, the imaging device controls at least one of the intensity of the infrared illumination, the insertion/extraction of an infrared cut filter in the imaging optical path of the imaging lens, and the white balance control. 21. The image pickup apparatus according to claim 20, wherein control is performed. A plurality of modes having different combinations of a control state of the infrared illumination, a state of insertion/removal of an infrared cut filter with respect to the imaging optical path of the imaging lens, and an image output state are stored, and the modes can be selected. 22. The imaging device according to any one of 10 to 21. 23. The imaging apparatus according to any one of claims 1 to 22, wherein the intensity of said infrared illumination can be corrected according to a priority set by a user. 24. The imaging apparatus according to any one of claims 1 to 23, wherein a user can select the brightness at which the infrared cut filter is inserted or retracted. 21. The imaging apparatus according to claim 20, wherein the white balance control range is selectable by a user. A computer program for controlling each part of the imaging device according to any one of claims 10 to 25 by a computer.

Images