JP2020102703A - Electronic device, control method of the same, and program - Google Patents

Abstract

To improve detection accuracy of halation in consideration of an influence on visibility of a subject.SOLUTION: The electronic device includes: evaluation means for evaluating brightness of a subject based on an exposure level when image data of the subject is acquired; and detection means for detecting halation based on brightness of the subject, a first brightness region having brightness higher than a first brightness threshold, and a distribution situation of the other brightness regions distributed around the first brightness region, which is a region having lower brightness than the first brightness region.SELECTED DRAWING: Figure 1

Description

The present invention relates to an electronic device that detects halation, a method for controlling the electronic device, and a program.

When a subject is imaged in a low-illuminance (low-luminance) environment using a camera, if high-illumination light enters the camera, light leaks out around the image capturing position of the light source, causing a phenomenon called halation. .. Halation occurs when high-intensity incident light forms an image on the image sensor and excessive charges are generated in the image sensor, and propagates to surrounding pixels. For example, halation may occur when a subject is imaged using a camera that monitors a public road. When the headlights of a car are lit, a large contrast occurs between the ambient light and the light of the headlights. Therefore, by imaging the subject in this state, an image with halation around the headlights of the car is generated. Is obtained. Then, in the image with halation, the appearance of the car and the visibility of the license plate and the like deteriorate.

As a related technique, there has been proposed a technique for reducing the amount of incident light by using an electronic shutter generally provided in a CCD camera when halation occurs (see Patent Document 1). Patent Document 1 discloses that in order to mask excessive incident light, a light amount adjusting means made of a liquid crystal plate is provided in front of the lens of the camera to adjust the light amount.

JP, 2004-072415, A

For example, in Patent Document 1, the amount of incident light is reduced by changing the electronic shutter to a high speed value in order to eliminate the effect of halation. Alternatively, in order to eliminate the influence of halation, a liquid crystal plate may be provided in front of the lens of the camera to reduce the amount of incident light. Here, even if halation occurs in the image captured by the camera, the effect on the visibility of the image may be low. For example, even if halation occurs due to the headlights of a car in the image, the effect on the visibility of the license plate may be low.

On the other hand, an image obtained by reducing the amount of incident light by controlling the electronic shutter or the liquid crystal plate has a reduced image quality due to the reduction in the amount of incident light. Therefore, if the amount of incident light is reduced in order to eliminate the effect of halation, the image quality of the image will deteriorate even though the effect of visibility of the subject due to halation is small.

An object of the present invention is to improve the detection accuracy of halation in consideration of the influence on the visibility of a subject.

In order to achieve the above object, an electronic device of the present invention includes an evaluation unit that evaluates the brightness of a subject based on an exposure level when image data of the subject is acquired; a brightness of the subject; The halation is performed based on the first luminance area having a luminance higher than the luminance threshold and the distribution situation of other luminance areas distributed around the first luminance area and having a lower luminance than the first luminance area. And a detection unit for detecting.

According to the present invention, it is possible to improve the detection accuracy of halation in consideration of the influence on the visibility of the subject.

It is a figure which shows the structural example of the imaging device which concerns on 1st Embodiment. 6 is a flowchart showing the flow of halation detection processing in the first embodiment. It is a figure which shows an example of a brightness|luminance gradient and each brightness|luminance area|region. It is a figure which shows the structural example of the imaging device which concerns on 2nd Embodiment. It is a flow chart which shows a flow of halation detection processing in a 2nd embodiment. It is a figure which shows the example of the image data used for distinction between a still life and a moving body.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each of the embodiments.

<First Embodiment>
FIG. 1 is a diagram illustrating a configuration example of the image pickup apparatus 1 according to the first embodiment. The image pickup apparatus 1 includes a taking lens 3, an image pickup sensor 4, an A/D converter 5, a system controller 6, a memory 7 and a storage medium 8. Hereinafter, in each embodiment, the imaging device 1 will be described as being applied to a camera that performs public road monitoring. The imaging device 1 may be an unmanned public road surveillance camera. However, the imaging device 1 may be used for purposes other than public road monitoring. Further, although the image capturing apparatus 1 is described as capturing a moving image, the image capturing apparatus 1 may capture a still image. The imaging device 1 detects halation that affects the visibility of the subject. The subject may be a car license plate, a car appearance, or the like. For example, when the image pickup apparatus 100 performs public road monitoring in an environment where the illuminance is low, such as at night, a headlight of a vehicle may cause halation in an image captured by the image pickup apparatus 100. However, even if halation occurs in an image, the degree of influence of halation may be low. In this case, the vehicle number or the like as the subject may be discriminated from the image in which halation has occurred. Therefore, in each embodiment, halation is detected in consideration of the influence on the visibility of the subject.

The image pickup apparatus 1 has, as an image pickup optical system, a taking lens 3 showing an optical lens group for taking in an optical image of a subject. The image sensor 4 receives the incident light that has passed through the image pickup optical system. The image sensor 4 photoelectrically converts the received incident light to form a subject image as an optical image. The image sensor 4 outputs a photoelectrically converted analog electric signal (analog image pickup signal) to the A/D converter 5. The A/D converter 5 converts the analog electric signal into a digital signal (digital image pickup signal) and outputs the digital signal to the system control unit 6. The system control unit 6 has an image processing unit 10 and an exposure control unit 16. The image processing unit 10 performs various kinds of image processing on the digital image pickup signal input to the system control unit 6 to generate image data. Thereby, the image data is acquired. The exposure control unit 16 determines the brightness of the image data generated by the image generation unit 11 of the image processing unit 10 and controls either the electronic shutter, the aperture, or the gain of the image sensor 4. Thereby, the brightness of the image data is corrected.

The taking lens 3 is assumed to have a zoom lens that moves in the optical axis direction to change the focal length, and a focus lens that moves in the optical axis direction to adjust the focus. The image sensor 4 is a sensor using a solid-state image sensor (CCD). The image pickup apparatus 1 can be applied to a color camera or a monochrome camera. When the imaging device 1 is a color camera, a color filter is installed closer to the opening of the imaging device 1 than the imaging sensor 4. The exposure control unit 16 as an exposure control unit has an automatic exposure control function, and instructs the image sensor 4 to adjust the brightness of the image based on the image data generated by the image generation unit 11. send. As a result, the exposure level is controlled, so that the brightness of the image captured by the image capturing apparatus 1 is adjusted.

The system control unit 6 performs various controls. The memory 7 is, for example, a RAM. The storage medium 8 is, for example, a ROM. The system control unit 6 may be realized by a control program stored in the storage medium 8 being expanded in the memory 7 and the CPU executing the expanded control program. In the case of the configuration shown in FIG. 1, the imaging device 1 having the imaging sensor 4 and the system control unit 6 corresponds to an electronic device. The system control unit 6 may be provided as a control device (for example, a personal computer or the like) separate from the imaging device 1. In this case, an external control device that functions as the system control unit 6 communicates with the imaging device 1 to implement the processing of each embodiment. In this case, the external control device corresponds to the electronic device.

The image processing unit 10 includes an image generation unit 11, a first evaluation unit 12, a second evaluation unit 13, a halation evaluation unit 14, and a threshold value determination unit 15. The image generation unit 11 generates image data based on the digital image pickup signal. The first evaluation unit 12 evaluates the brightness of the image data, and classifies the whiteout area (first brightness area) and one or more other brightness areas distributed around the whiteout area. The first evaluation unit 12 corresponds to a classification means. The second evaluation unit 13 as an evaluation unit estimates the brightness of the subject and the brightness of the ambient light from the exposure level, and evaluates the brightness of the subject and the brightness of the ambient light. The halation evaluation unit 14 evaluates, based on the distribution of each luminance region detected by the first evaluation unit 12, whether or not halation that has a low influence on the visibility of the subject has occurred. Halation that has a small effect on the visibility of the subject is halation that does not require detection. In the present embodiment, the brightness of the exposure and the illuminance (luminance) is defined based on the APEX (ADDITIVE SYSTEM OF PHOTOGRAPHIC EXPOSURE) system. For example, the difference when the brightness of the subject is 1 BV and 2 BV corresponds to the difference in brightness of one level of the illuminance (luminance) of the subject in the APEX system.

The halation evaluation unit 14 considers not only the distribution of the overexposed areas and the high-intensity areas around the overexposed areas, but also the brightness of the subject and evaluates whether halation that affects the visibility of the subject has occurred. I do. The halation which affects the visibility of the subject is detected according to the evaluation result of the halation evaluation unit 14 as the detection means. The threshold value determining unit 15 determines a brightness threshold value used when the second evaluation unit 13 evaluates the brightness of the subject, and a brightness threshold value used when the halation evaluation unit 14 evaluates halation. The threshold value determining unit 15 as a changing unit changes the brightness threshold value according to the brightness of the subject evaluated by the second evaluation unit 13.

The flow of halation detection processing in the first embodiment will be described with reference to FIG. The second evaluation unit 13 estimates the brightness of the subject from the exposure level and evaluates the subject brightness (S100). The threshold value determination unit 15 determines whether the subject brightness is equal to or higher than a predetermined brightness (S101). The predetermined brightness can be set to any value. If YES in S101, the threshold value determination unit 15 increases the brightness threshold value used when classifying the brightness regions from the image data (S102). The first evaluation unit 12 detects a plurality of luminance areas from the image data using one or a plurality of luminance thresholds, and labels each detected luminance area (S103). Labeling is a process of classifying each luminance region in image data. If YES is determined in S101, the threshold value determination unit 15 sets the brightness threshold value to a high value. The initial value of the brightness threshold may be any value. For example, the initial value of the brightness threshold value may be a brightness threshold value for detecting halation in a low illuminance environment such as night, dawn, and evening. If YES is determined in S101, the threshold value determination unit 15 changes the brightness threshold value from the initial value to a high value.

The first evaluation unit 12 labels the area having the highest brightness level (brightness not less than the first brightness threshold) as the first brightness area. Then, the first evaluation unit 12 sets the brightness that is equal to or higher than the nth brightness threshold value (n is an integer of 2 or more) and is lower than the (n−1)th brightness threshold value with the first brightness region as the center. Label the existing region as the nth region. As the value of “n” increases, the brightness level of the nth region decreases. The first luminance area is, for example, an overexposed area.

FIG. 3 is a diagram showing an example of the brightness gradient and each brightness region. FIG. 3 shows the case of “n=3”, and the first brightness threshold value, the second brightness threshold value, and the third brightness threshold value are used. FIG. 3A shows the relationship between the brightness gradient of image data and each brightness threshold value. FIG. 3B shows a distribution of each luminance region (each labeled luminance region) corresponding to FIG. The first luminance region is a region having the highest luminance level and a region having a luminance equal to or higher than the first luminance threshold. As shown in FIG. 3B, the second luminance region is distributed concentrically around the first luminance region, and the third luminance region is distributed outside the second luminance region. .. As shown in FIG. 3(B), the halation evaluation unit 14 detects halation when the brightness regions are distributed such that the brightness decreases in the outer peripheral direction around the first brightness region. ..

Here, if YES is determined in S101, the process of S102 is executed. That is, when the subject brightness is bright, the threshold determination unit 15 increases the brightness threshold. As a result, when the subject brightness is bright, the gradation of the brightness around the overexposed area is maintained. Then, the halation evaluation unit 14 stops detecting halation that is unnecessary for detection.

Since the halation easily occurs around the overexposed area, the halation evaluation unit 14 extracts the labeled first luminance area as the overexposed area (S104). The halation evaluation unit 14 evaluates halation detection by paying attention to the distribution of luminance around the overexposed area. The halation evaluation unit 14 determines to what extent the halation detection is evaluated by referring to (or scanning) the first brightness area (whitened area) as a center, according to the subject brightness. This is because the degree of influence of halation on the recognizability of the subject changes depending on the luminance ratio between the ambient light and the high-luminance subject. As the high-brightness subject, for example, a white subject such as a license plate is assumed.

When a high-luminance subject enters the angle of view of the image pickup apparatus 1, the exposure amount changes for brightness correction. When the change in the exposure amount of the preceding and following frames (image data) in the moving image captured by the imaging device 1 increases, the brightness ratio between the ambient light and the high-brightness subject also increases. When the brightness ratio between the ambient light and the high-luminance subject is large, it is assumed that the degree of influence of halation on the visibility of the subject is large. On the other hand, when the change in the exposure amount between the front and rear frames is small, the brightness ratio between the ambient light and the high brightness subject is small. In this case, it is assumed that the degree of influence of halation on the visibility of the subject is small. The halation evaluation unit 14 does not detect halation that does not affect the visibility of the subject. When halation is detected, control is performed to suppress the effect of halation on image data. For example, when the image pickup apparatus 1 has an insertion/extraction mechanism that inserts/extracts a filter (for example, a visible light cut filter) that cuts light of a specific wavelength on the optical path of the taking lens 3, it is determined that halation is detected. Then, the insertion/removal mechanism inserts the filter into the optical path. Thereby, the influence of halation can be suppressed.

However, when the visible light cut filter is inserted in the optical path, the light of the visible light component is cut, so that the image quality of the image data deteriorates. Therefore, the halation evaluation unit 14 does not detect halation that does not affect the visibility of the subject. As a result, it is possible to avoid deterioration of the image quality of the image data due to detection of halation that does not need to be detected.

Here, in the halation detection focusing on the brightness gradient around the whiteout region (first brightness region), the detection accuracy tends to depend on the brightness of the subject around the whiteout region. Regardless of the degree of influence of halation, when the brightness of the subject around the overexposed area is high, the brightness of the subject around the overexposed area is detected as the brightness due to flare. Therefore, the reliability of halation detection is low in halation detection that simply focuses on the brightness gradient around the overexposed area. Therefore, the halation evaluation unit 14 determines whether the brightness ratio between the ambient light and the high-brightness subject is a predetermined value or less (S105). As described above, the brightness ratio between the ambient light and the high-brightness subject is acquired based on the change in the exposure amount of the preceding and following frames. For example, the brightness ratio between the ambient light and the high-brightness subject is acquired based on the value obtained by differentiating the change in the exposure amount of the preceding and following frames with respect to time. If YES is determined in S105, the luminance ratio between the ambient light and the high-luminance subject is small. In this case, the halation evaluation unit 14 expands the reference range for detecting halation (S106). For example, when it is determined to be NO in S105, the halation evaluation unit 14 may set the first and second brightness regions as the reference brightness regions for halation detection. On the other hand, if YES is determined in S105, the luminance area to be referred to for halation detection is expanded not only to the first and second luminance areas but also to the third luminance area. As a result, the reliability of halation detection can be improved even when the luminance ratio between the ambient light and the high-luminance subject is small.

The halation evaluation unit 14 determines whether or not the nth luminance region whose luminance is smaller than that of the (n-1)th luminance region is distributed in the outer peripheral direction within the reference range (S107). That is, the halation evaluation unit 14 determines the halation by evaluating that the brightness regions in which the brightness level decreases from the first brightness region shown in FIG. 3 toward the outer peripheral direction are distributed. If YES is determined in S107, halation is detected (S108). On the other hand, if NO in S107, the flow moves to S100. In this case, halation is not detected. That is, halation (halation that does not need to be detected) that has a low influence on the visibility of the subject is not detected. The above is the halation detection processing in the first embodiment.

When halation is detected based on the first luminance area and other luminance areas (luminance areas distributed around the first luminance area), halation that has a low influence on the visibility of the subject is also detected. I will end up. On the other hand, the system control unit 6 of the embodiment evaluates the brightness of the subject based on the exposure level when the image data was acquired. Then, the system control unit 6 detects halation based on the brightness of the subject, the first brightness area, and the other brightness areas. Therefore, it is possible to suppress the detection-unnecessary halation that has a low influence on the visibility of the subject.

If YES is determined in S101, the subject brightness is bright. When the subject brightness is bright, it is assumed that the degree of influence on the visibility of the subject due to halation caused by the headlights of the vehicle is low. In this case, the threshold value determination unit 15 increases the brightness threshold value for labeling each brightness region in S102. As a result, when the brightness of the subject is bright, the brightness level of the brightness region used in the determination of S107 becomes high. In this case, the distribution of the brightness area as shown in FIG. 3 is not detected. Therefore, the halation evaluation unit 14 does not detect unnecessary halation.

In the above, the halation evaluation unit 14 detects the halation that affects the visibility of the subject based on whether or not there is the n-th area in which the brightness decreases in the outer peripheral direction around the first brightness area. To do. The halation evaluation unit 14 may detect the halation that affects the visibility of the subject by a method other than the above. For example, the halation evaluation unit 14 evaluates the area ratio between the overexposed region and the flare region by using the first luminance region as the overexposed region and the luminance region other than the first luminance region as the flare region, thereby evaluating the halation. May be detected.

<Second Embodiment>
Next, a second embodiment will be described. Halation tends to occur when the luminance ratio between the ambient light and the high-luminance subject is large, and often occurs, for example, at night. When the imaging device 1 is applied to a public road surveillance camera or the like, image data captured by the imaging device 1 may include not only halation caused by a headlight of a vehicle but also halation caused by streetlight light. The imaging device 1 of the second embodiment excludes the light of a still life such as a street lamp from the target of halation detection. FIG. 4 is a diagram showing a configuration example of the image pickup apparatus 1 according to the second embodiment. The image pickup apparatus 1 of the second embodiment has a bandpass filter 9, a moving body detection unit 17, an infrared illumination control unit 18, an infrared illumination unit 20, and a timer 21, and is the same as that shown in FIG. The imaging device 1 of the first embodiment is different. Other configurations of the image pickup apparatus 1 according to the second embodiment are similar to those of the image pickup apparatus 1 according to the first embodiment, and thus the description thereof will be omitted.

The bandpass filter 9 is arranged between the taking lens 3 and the image sensor 4 so that it can be inserted and removed. The bandpass filter 9 is a filter that cuts light in a specific wavelength range. The bandpass filter 9 is inserted into or removed from the optical path of incident light entering from the taking lens 3 by a predetermined insertion/removal mechanism. The infrared illumination control unit 18 controls the infrared illumination unit 20 to emit infrared light. The timer 21 counts time and notifies the system control unit 6 when a predetermined time has elapsed.

The bandpass filter 9 has an infrared cut filter that cuts infrared light and a visible light cut filter that cuts visible light. For example, when the halation evaluation unit 14 detects the occurrence of halation, the predetermined insertion/removal mechanism described above is used for the purpose of recovering the visibility of the subject and the visible light of the band-pass filter 9 is visible in the optical path of the incident light incident from the taking lens 3. Insert a light cut filter. Then, the predetermined insertion/removal mechanism removes the infrared cut filter of the bandpass filter 9 from the optical path of the incident light incident from the taking lens 3. This reduces the influence of halation. The insertion/extraction control of the bandpass filter 9 is performed by, for example, the system control unit 6. When the system control unit 6 controls to insert the visible light cut filter in the optical path of the incident light, the infrared illumination control unit 18 causes the infrared illumination unit 20 to emit infrared light. At this time, the infrared illumination control unit 18 controls the infrared illumination unit 20 so that the infrared illumination unit 20 emits infrared light having an illuminance higher than that of the reduced halation light. As a result, the brightness of the image data acquired by the imaging device 1 is guaranteed, and halation is further reduced.

The image data generated by the image generation unit 11 is temporarily stored in the memory 7. When the system control unit 6 receives the notification indicating that the predetermined time has elapsed from the timer 21, the image processing unit 10 updates the reference image for the background difference in the motion detection with the image data acquired at the timing of receiving the notification. To do. In the second embodiment, the imaging device 1 distinguishes between halation by a still life and halation by a moving body, and narrows down the halation detection target to the moving body. A still life corresponds to a stationary object. The moving body corresponds to a moving object.

FIG. 5 illustrates the flow of halation detection processing according to the second embodiment. The flowchart of FIG. 5 is a flowchart in which S207, S208, S209 and S212 are added to the flowchart of FIG. 2 in the first embodiment. The steps other than S207, S208, S209, and S212 are the same as those in the flowchart of FIG. The moving body detection unit 17 performs motion detection assuming the background difference, and determines whether there is a moving body based on the difference between the background difference and the reference image (S207). FIG. 6 shows an example of the reference image and the image data acquired when the time t has elapsed. FIG. 6A shows a reference image (reference image data). FIG. 6B shows image data (frame t) acquired when time t has elapsed. The image data is divided into a plurality of grid-shaped areas, and both the reference image and the image data in the frame t include an area S that is an overexposed area (first luminance area). The area S is included in both the reference image and the image data in the frame t. Thereby, the area S can be determined to be a still life such as a street light.

On the other hand, the area M, which is an overexposed area in the frame t, is included in the image data of the frame t, but is not included in the reference image. Therefore, the region M can be determined to be a moving body such as a car. As described above, whether or not there is a moving object can be determined from the binary image using the first brightness threshold. If YES is determined in S207, there is a moving object, so the flow proceeds to S210. Therefore, the halation evaluation unit 14 can narrow down the object that detects the halation that affects the visibility of the subject to the moving body. As described above, the still life is excluded from the target for detecting the halation that affects the visibility of the subject.

If NO is determined in S207, the system control unit 6 determines whether a predetermined time has passed (S208). The timer 21 notifies the system control unit 6 when a predetermined time has elapsed. Therefore, while the system control unit 6 does not receive the notification from the timer 21, it is determined as NO in S208. If NO is determined in S208, the flow moves to S200. If YES is determined in S208, it is determined that the predetermined time has elapsed and that there is no moving object, so the image processing unit 10 updates the reference image (S209). As a result, the reference image can be updated to the latest image data every predetermined time.

As shown in FIG. 5, when the halation is detected in S211, the system control unit 6 performs an image improving process on the image data acquired after the halation is detected (S212). For example, the exposure control unit 16 may improve the image quality of image data acquired after halation detection by performing exposure control such as wide dynamic range processing (WDR processing). Further, the system control unit 6 may control the predetermined insertion/removal mechanism to insert the visible light cut filter of the bandpass filter 9 into the optical path of the incident light. This reduces halation and improves the image quality of image data acquired after halation detection. The process of S212 may be performed in the first embodiment.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications and changes can be made within the scope of the gist thereof. The present invention supplies a program that implements one or more functions of each of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors of a computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. The present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 1 Imaging device 6 System control part 10 Image processing part 12 1st evaluation part 13 2nd evaluation part 14 Halation evaluation part 15 Threshold value determination part 16 Exposure control part 17 Moving body detection part

Claims (10)

Based on the exposure level when the image data of the subject was acquired, an evaluation means for evaluating the brightness of the subject,
The brightness of the subject, a first brightness region having a brightness higher than a first brightness threshold, and a brightness region lower than the first brightness region and distributed around the first brightness region. Detection means for detecting halation based on the distribution state of
An electronic device comprising: Classifying means for classifying a brightness area including the first brightness area and the other brightness area using a single or a plurality of brightness threshold values;
Changing means for changing the brightness threshold value according to the brightness of the subject;
The electronic device according to claim 1, further comprising: The electronic device according to claim 2, wherein the changing unit increases the brightness threshold when the brightness of the subject is equal to or higher than a predetermined brightness. 4. The halation is detected when the detecting means detects a brightness area in which the brightness decreases with going toward the outer peripheral direction with the first brightness area as the center, and the halation is detected. The electronic device according to any one of the above. Exposure control means for controlling the exposure level of the image sensor,
The electronic device according to any one of claims 1 to 4, wherein the evaluation unit evaluates the brightness of the subject and the brightness of ambient light based on the exposure level detected by the exposure control unit. machine. The detection means expands a range of a brightness region to be referred to for detecting the halation when a brightness ratio between the brightness of the ambient light and the brightness of the subject is a predetermined value or less. The electronic device according to item 5. 7. The detection means detects the halation for a moving object among the objects of the image data, and does not detect the halation for a stationary object. The electronic device according to item 1. The electronic device according to claim 1, wherein the electronic device is an imaging device having an imaging element. A step of evaluating the brightness of the subject based on the exposure level when the image data of the subject was acquired;
The brightness of the subject, a first brightness region having a brightness higher than a first brightness threshold, and a brightness region lower than the first brightness region and distributed around the first brightness region. Detecting halation based on the distribution state of
A method of controlling an electronic device, comprising: A computer-readable program for causing a computer to execute the method for controlling an electronic device according to claim 9.

Images