JP4978607B2 - Image acquisition device - Google Patents

Description

  The present invention relates to an image acquisition device, and more particularly, to an image acquisition device used in security, surveillance, wildlife observation, mobile operation support (for example, aircraft, helicopter, ship, car, etc.), high-speed phenomenon observation, etc. in a dark field. .

Night vision devices have been developed in order to visually observe the state where there is almost no visible light. There are two types of night vision devices, one that uses a night vision tube that senses infrared rays and converts it into a visible image, and the other that uses a multistage image tube that strongly enhances extremely weak light (for example, see Patent Document 1). is there.
Here, a night vision apparatus using a multistage image tube will be described. FIG. 4 is a schematic configuration diagram showing a night vision apparatus 100 using a multistage image tube.
The night vision device 100 includes an objective lens (optical system) 1b provided on the IIT (image intensifier) input side, an IIT2 that multiplies luminance, a lens 1a (optical system) provided on the IIT output side, An image is displayed on a monitor (display device) 110 based on an electrical signal (luminance value) from a CCD (Charge Coupled Device) 103 that detects an amplified optical image of the IIT 2 via the lens 1a, and the IIT 2 The signal processing unit 104 outputs a gain signal for adjusting the gain (electronic multiplication factor) g of the signal and the monitor 110.

The IIT 2 is multiplied by a photocathode 2a for converting an incident optical image into a photoelectron image, an electron lens system 2b, and an MCP (Micro Channel Plate) 2c for multiplying an electron group of the photoelectron image. And a fluorescent screen 2d for generating an amplified optical image from the group of electrons. The MCP 2c is a bundle of a plurality of glass capillaries.
The CCD 103 has photoelectric conversion elements arranged in M rows and N columns. The CCD 103 is an element that stores charges generated by incident light in a potential well formed in a semiconductor, and transfers the charges along the semiconductor surface by sequentially moving the potential well by applying a transfer voltage from the outside. There is a solid-state imaging device in which a sensor and a scanning unit are integrated.

In such a night vision device 100, a faint light (optical image) from the subject is formed as a photoelectron image on the photocathode 2a of the IIT 2 through the objective lens 1b, and the intensity of the light from the cathodic photocathode 2a. Accordingly, photoelectrons are emitted and imaged on the input surface of the MCP 2c by the acceleration voltage applied to the anode and the electron lens system 2b. When a voltage is applied to both sides of the MCP 2c, when photoelectrons pass through the glass thin tube, the secondary electrons are repeatedly collided with the secondary electron emission surface of the glass thin tube wall, and the secondary electrons are multiplied, and finally collide with the phosphor screen 2d. An amplified optical image multiplied to high brightness is obtained. The amplified optical image of the fluorescent screen 2d is formed on the CCD 103 via the lens 1a and is photoelectrically converted by the CCD 103. Accordingly, the signal processing unit 104 sequentially reads the charges (luminance values) T _mn of the M × N photoelectric conversion elements of the CCD 103 and creates an image signal, thereby displaying an image on the monitor 110.

  By the way, since the night vision device 100 amplifies the acquired light several thousand times to several tens of thousands times, when the incident optical image contains light having a certain brightness or more (for example, in the nighttime When the light is directly incident), the monitor 110 displays a saturated image in which the portion emitting the strong light is light saturated, and as a result, the light saturated portion cannot be observed. There is a problem. In addition, this light saturation affects not only a portion that emits strong light but also a considerably wide range portion around it, and a similar saturated image is projected in a considerably wide range portion around the light saturation. Such a phenomenon is called halation, and the luminance value information of the image signal of the portion where halation has occurred is completely destroyed.

Therefore, the signal processing unit 104 adjusts the gain (luminance magnification) g of IIT2 based on the luminance value T _mn of the M × N photoelectric conversion elements of the CCD 103, in addition to displaying an image on the monitor 110. A gain signal is output. Specifically, the gain g of IIT2 is controlled by controlling the acceleration voltage applied between the cathode and anode of IIT2 and the voltage applied to both sides of MCP2c. Thereby, for example, when the incident optical image includes a luminance value T _mn that is equal to or higher than the first setting threshold T ₁ , halation is avoided by decreasing the gain g.
JP 2002-112109 A

However, if only the halation is to be avoided, the gain g of IIT2 may be lowered, but conversely, the incident light from the weak light portion also decreases, and in the portion emitting the weak light, the incident light shortage image is displayed. As a result, the image may become unobservable.
In addition, it is conceivable to use a filter that attenuates incident light by detecting that an incident optical image includes a luminance value T _mn that is equal to or greater than the first set threshold value T _1. However, the gain g of the IIT 2 is lowered. The image becomes unobservable in the portion emitting the weak light.
SUMMARY OF THE INVENTION An object of the present invention is to provide an image acquisition device that can satisfactorily observe an optical image in a dark portion without causing halation even when an optical image including intense light is incident.

  The image acquisition device of the present invention made to solve the above problems is multiplied by a photocathode for converting an incident optical image into a photoelectron image, an electron multiplier for multiplying an electron group of the photoelectron image, and a photomultiplier tube. An image intensifier having a fluorescent screen that generates an amplified optical image from the group of electrons, an image sensor on which the amplified optical image is formed, and a plurality of photoelectric conversion elements arranged two-dimensionally; Based on the luminance values obtained from the plurality of photoelectric conversion elements, an image is displayed on the display unit, and a gain signal for adjusting the gain of the image intensifier so as to suppress halation and blackout of the image is obtained. An image acquisition apparatus including a signal processing unit for outputting, wherein the imaging element is a CMOS, and the signal processing unit includes a maximum luminance value and a plurality of luminance values obtained from a plurality of photoelectric conversion elements. Average luminance value in the luminance value obtained from the photoelectric conversion elements, and, based on the maximum luminance value difference luminance value obtained by subtracting the average luminance value from, and to adjust the gain of the image intensifier.

According to the image acquisition device of the present invention, since the imaging element is a CMOS (Complementary Metal Oxide Semiconductor), the dynamic range is wider than that of a CCD, and as a result, even when an optical image including strong light is incident. An optical image in a dark part can be observed well without causing halation.
The signal processing unit calculates not only the maximum luminance value but also the average luminance value and the difference luminance value, and adjusts the gain of the IIT based on the maximum luminance value, the average luminance value, and the difference luminance value. For example, when the maximum luminance value is less than the first set threshold value, that is, no halation occurs at the current gain, the gain is increased in order to observe an optical image in a dark part satisfactorily. At this time, the smaller the difference luminance value, the larger the gain increase range. On the other hand, the maximum luminance value is equal to or higher than the first set threshold value, and the line maximum luminance value in the luminance value obtained from the photoelectric conversion element in the row direction is equal to or higher than the second set threshold value. In other words, the current gain has a high probability of halation, so the gain is lowered. At this time, the smaller the difference luminance value is, the larger the gain reduction range is.

  As described above, according to the image acquisition apparatus of the present invention, it is possible to satisfactorily observe an optical image in a dark portion without causing halation even when an optical image including intense light is incident.

(Means and effects for solving other problems)
In the above invention, the dynamic range of the CMOS may be 100 dB or more.
Here, “dynamic range (DR)” refers to the ratio of the brightest signal (Sig_upper) that can be detected by the image sensor to the darkest signal (Sig_low).
DR [dB] = 20log (Sig_upper / Sig_low)
In the above invention, in the CMOS, the photoelectric conversion elements are arranged in M rows and N columns, and when the maximum luminance value is less than a first set threshold value, the signal processing unit sets the difference luminance value to the difference luminance value. Based on this, the gain of the image intensifier is increased, while the maximum luminance value is equal to or greater than a first setting threshold value and the line maximum luminance value in the luminance value obtained from the photoelectric conversion element in the row direction or the column direction However, when the value is equal to or greater than the second set threshold value and continues for more than the set row or set column, the gain of the image intensifier may be lowered based on the difference luminance value.

Here, the “first setting threshold value” is a brightness value that is preliminarily stored in the image acquisition device and is likely to cause blackout in the image displayed on the display. For example, the maximum gradation in the image Is preferably 75% or less.
The “second set threshold value” is stored in advance in the image acquisition device and refers to a luminance value that is likely to cause halation in the image displayed on the display. For example, the maximum gradation 80 in the image is displayed. % Or more is preferable.

In the above invention, the signal processing unit is configured to detect the luminance value obtained from one photoelectric conversion element and the X row and Y column (X <M, Y <N) photoelectric conversion centered on one photoelectric conversion element. Display brightness obtained by locally contrast-correcting the brightness value obtained from one photoelectric conversion element based on the corrected brightness value calculated from the partial maximum brightness value and the partial minimum brightness value in the brightness value obtained from the conversion element The image may be displayed by calculating a value.
According to the image acquisition apparatus of the present invention, since the signal processing unit uses a CMOS, a partial processing is performed based on the luminance value obtained from the photoelectric conversion element of X rows and Y columns centering on one photoelectric conversion element. By calculating the maximum luminance value and the partial minimum luminance value, it is possible to calculate a display luminance value obtained by locally correcting the luminance value obtained from one photoelectric conversion element.

Furthermore, in the above invention, the signal processing unit calculates the display luminance value using the equation (1), and the further away the adjusted gain is from the reference gain, the more influence the corrected luminance value has on the display luminance value. The correction coefficient may be increased so as to increase the.
Display luminance value = (corrected luminance value × correction coefficient) + {luminance value × (1−correction coefficient)} (1)

Here, the “reference gain” is to amplify the acquired light at a magnification adjusted from several thousand times to several tens of thousands times in IIT. The magnification that can acquire a good image without performing contrast correction, which is stored in advance in the image acquisition device, and is, for example, a voltage value corresponding to 5,000 times.
According to the image acquisition device of the present invention, when the gain is significantly increased or decreased, the luminance value obtained from the photoelectric conversion element becomes unstable, so that the influence of the corrected luminance value on the display luminance value is strongly increased. can do.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments described below, and it goes without saying that various aspects are included without departing from the spirit of the present invention.

FIG. 1 is a diagram showing an external configuration of a night-vision device (image acquisition device) 10 according to an embodiment of the present invention. In addition, the same code | symbol is attached | subjected about the thing similar to the night vision apparatus 100. FIG.
The night vision apparatus 10 includes an objective lens 1b provided on the IIT input side, IIT2 for multiplying the brightness, a lens 1a provided on the IIT output side, and a wide optical lens for detecting an amplified optical image of the IIT2 via the lens 1a. A gain for displaying an image on a monitor (display) 110 based on an electrical signal (luminance value) from the dynamic range CMOS (imaging device) 3 and the wide dynamic range CMOS 3 and adjusting a gain (electronic multiplication factor) g of the IIT 2 The signal processing unit 4 outputs a signal and a monitor 110. The gain g of IIT2 can amplify the acquired light at a magnification adjusted from several thousand times to several tens of thousands times.
The wide dynamic range CMOS3 has photoelectric conversion elements arranged in M rows and N columns (for example, 480 rows and 640 columns). Compared with a CCD, the dynamic range is wide, and an electrical signal of, for example, 10 bits (1024 gradations) can be output to the signal processing unit 4.
The dynamic range of the wide dynamic range CMOS is preferably 100 dB or more from the viewpoint that even when an optical image containing intense light is incident, an optical image in a dark part can be satisfactorily observed without causing halation. .

The signal processing unit 4, area centered with the FPGA 40, for calculating a first set thresholds T ₁ and the second set threshold value T ₂ and sets the row and reference gain G and the correction luminance value, one of the photoelectric conversion element X × Y and the like are set in advance. Furthermore, it has the memory 30 which memorize | stores an image.
The control and calculation contents executed by the FPGA 40 will be described separately for each function. A gain signal output unit 41 that outputs a gain signal, a corrected luminance value calculation unit 42 that calculates a corrected luminance value T _mn ′, and an image on the monitor 110. Is displayed, and a correction coefficient calculation unit 44 that calculates the correction coefficient α is included.

The gain signal output unit 41 performs control to output a gain signal for adjusting the gain g of the IIT 2 based on the luminance value T _mn obtained from the M × N photoelectric conversion elements of the wide dynamic range CMOS 3.
For example, first, M × N number of the maximum luminance value T _max of the luminance value T _mn obtained from the photoelectric conversion element, M × N pieces of average luminance value T _ave in the luminance value T _mn obtained from the photoelectric conversion element And a difference luminance value ΔT obtained by subtracting the average luminance value T _ave from the maximum luminance value T _max and a line maximum luminance value T _m in the luminance value T _mn obtained from the N photoelectric conversion elements in the row direction are calculated. .
Then, when the maximum luminance value T _max is less than the first set threshold T ₁ (for example, 750), that is, since the current gain g does not cause halation, in order to observe a dark part optical image satisfactorily, A gain signal for increasing the gain g is output. At this time, the smaller the difference luminance value ΔT, the larger the gain g increase width Δg.
On the other hand, it is determined that the maximum brightness value T _max is equal to or greater than the first set threshold value T ₁ and the line maximum brightness value T _m is equal to or greater than the second set threshold value T ₂ (eg, 820). 50 lines), that is, since there is a high probability that halation has occurred at the current gain g, a gain signal that lowers the gain g is output. At this time, the smaller the difference luminance value ΔT is, the larger the decrease width Δg of the gain g is.

The corrected luminance value calculation unit 42 performs control to calculate a corrected luminance value T _mn ′ based on the luminance value T _mn obtained from the M × N photoelectric conversion elements of the wide dynamic range CMOS 3.
For example, first, the following formula (2) is used to calculate the luminance value T _mn obtained from one photoelectric conversion element and the X rows and Y columns (for example, 15 rows and 20 columns) centered on one photoelectric conversion device. A corrected luminance value T _mn ′ is calculated from the partial maximum luminance value T _max ′ and the partial minimum luminance value T _min ′ in the luminance value T _mn obtained from the photoelectric conversion element.
T _mn ′ = {(T _mn −T _min ′) / (T _max ′ −T _min ′)} × 1024 (2)

The display control unit 43 includes a luminance value T _mn obtained from M × N photoelectric conversion elements of the wide dynamic range CMOS 3, a corrected luminance value T _mn ′ calculated by the corrected luminance value calculating unit 42, and correction described later. Based on the correction coefficient α calculated by the coefficient calculation unit 44, the display luminance value S _mn is calculated, and control for displaying an image on the monitor 110 is performed.
For example, the display luminance value S _mn is calculated using the following formula (1) to display an image.
S _mn = (T _mn '× α) + {T _mn × (1-α)} (1)

The correction coefficient calculation unit 44 performs control to calculate a correction coefficient α that determines the weighting ratio between the luminance value T _mn and the correction luminance value T _mn ′ based on the adjusted gain g.
For example, the correction coefficient calculation unit 44 increases the correction coefficient α as the adjusted gain g is further away from the reference gain G (for example, a voltage value corresponding to 5,000 times). Specifically, as described above, when the maximum luminance value T _max is less than the first set threshold T ₁ , the gain g is increased, but when the gain g is significantly increased, the gain is obtained from the photoelectric conversion element. Since the luminance value T _mn becomes unstable, the correction coefficient α is increased so as to increase the influence of the correction luminance value T _mn ′ on the display luminance value S _mn .
The maximum luminance value T _max is at first set thresholds T ₁ or more, and, at the line maximum luminance value T _m is not more second setting threshold value T ₂ or more, setting lines (e.g., 50 lines) or When continuous, the gain g is lowered. However, if the gain g is greatly lowered, the luminance value T _mn obtained from the photoelectric conversion element becomes unstable, so that the corrected luminance value T _mn in the display luminance value S _mn . The correction coefficient α is increased so as to increase the influence of '.
When the gain g is not significantly increased or decreased, the luminance value T _mn obtained from the photoelectric conversion element is stable, so that the influence of the corrected luminance value T _mn ′ is strongly influenced on the display luminance value S _mn . Instead, the luminance value T _mn is used almost as it is.

Next, an example of a gain control method for adjusting the gain g of IIT2 by the gain signal output unit 41 will be described. FIG. 2 is a flowchart for explaining the gain control method.
First, in the process of step S101, the gain signal output unit 41 obtains a luminance value T _mn from M × N photoelectric conversion elements of the wide dynamic range CMOS3.

Next, in the process in step S102, calculates the line maximum luminance value T _m of the luminance value T _mn obtained from the row direction of the N photoelectric conversion elements. At this time, M line maximum luminance values _Tm are obtained by executing from the first line to the Mth line.
Next, in the process of step S103, and calculates the maximum luminance value _{T max} from the M line maximum luminance value _{T m.}

Next, in the process of step S104, an average luminance value T _ave is calculated for the luminance value T _mn obtained from M × N photoelectric conversion elements.
Next, in the process of step S105, a difference luminance value ΔT obtained by subtracting the average luminance value T _ave from the maximum luminance value T _max is calculated.
Next, it is determined in the processing in step S106, the maximum luminance value _{T max} is first set threshold value _{T 1} (e.g., 750) whether it is less than. Maximum luminance value T _max is the time it is determined that the first set lower than thresholds T _1, in the processing of step S107, on the basis of the difference luminance value [Delta] T, and outputs a gain signal to increase the gain g (gain-up). Then, this flowchart is ended. Thereafter, when the gain signal output unit 41 obtains the luminance value T _mn that is the next frame, the process of step S101 is started.

On the other hand, the maximum luminance value _{T max} is when it is determined not to be the first set lower than thresholds _{T 1,} in the processing of step S108, the line parameters m = 1, and the continuous count parameter a = 0.
Next, in the process in step S109, the line maximum luminance value _{T m} of a m-th row determines the second set threshold value _{T 2} (e.g., 820) to or greater than. Line maximum luminance value T _m of a m-th row, when it is determined that not the second set threshold value T ₂ or more, in the process of step S110, a continuous count parameter a = 0, further in the process in step S111, and m = m + 1 The process returns to step S109.

On the other hand, the line maximum luminance value _{T m} of a m-th row, when it is the second set threshold value _{T 2} or more, in the process of step S112, and a = a + 1.
Next, in the process of step S113, it is determined whether or not the continuous count parameter a is 50 or more. When it is determined that the continuous count parameter a is 50 or more, a gain signal for decreasing the gain g is output based on the difference luminance value ΔT in the process of step S114 (gain down). Then, this flowchart is ended. Thereafter, when the gain signal output unit 41 obtains the luminance value T _mn that is the next frame, the process of step S101 is started.

On the other hand, when it is determined that the continuous count parameter a is not 50 or more, it is determined whether or not the m-th row is M or more in the process of step S115. When it is determined that the m-th row is not M or more, the process returns to step S111.
On the other hand, when it is determined that the m-th row is M or more, this flowchart is ended. Thereafter, when the gain signal output unit 41 obtains the luminance value T _mn that is the next frame, the process of step S101 is started.

Next, an example of a display method for displaying an image by the night vision apparatus 10 will be described. FIG. 3 is a flowchart for explaining the display method.
First, in the process of step S201, a luminance value T _mn is obtained from M × N photoelectric conversion elements of the wide dynamic range CMOS3.
Next, in the process of step S202, the corrected luminance value calculation unit 42 has a partial maximum luminance value _Tmax ′ in the luminance value _Tmn obtained from the photoelectric conversion element of 15 rows and 20 columns centering on one photoelectric conversion element. And the partial minimum luminance value T _min ′ are calculated. At this time, M × N partial maximum luminance values T _max ′ and partial minimum luminance values T _min ′ are obtained by executing the processing on M × N photoelectric conversion elements.

Next, in the process of step S <b> 203, the corrected luminance value calculation unit 42 calculates a corrected luminance value T _mn ′. At this time, M × N corrected luminance values T _mn ′ are obtained by executing the processing for M × N photoelectric conversion elements.
Next, in the process of step S204, the correction coefficient calculation unit 44 calculates the correction coefficient α based on the adjusted gain g.

Next, in the processing of step S205, the image display unit 43 determines the luminance value T _mn obtained from the M × N photoelectric conversion elements of the wide dynamic range CMOS 3, the corrected luminance value T _mn ′, the correction coefficient α, The display brightness value S _mn is calculated based on the above and an image is displayed on the monitor 110.
And when the process of step S205 is complete | finished, this flowchart is complete | finished. Thereafter, when the luminance value T _mn that is the next frame is obtained, the process of step S201 is started.

  As described above, according to the night vision apparatus 10, it is possible to satisfactorily observe an optical image in a dark portion without causing halation even when an optical image including strong light is incident.

(Other embodiments)
(1) In the night vision apparatus 10 described above, the configuration using the luminance value T _mn obtained from the photoelectric conversion element is shown. However, M ′ rows and N ′ columns (for example, 3 rows) centered on one photoelectric conversion element. It is good also as a structure using the luminance value which is a moving average obtained from the photoelectric conversion element of 3 rows).
(2) In the night vision device 10 described above, the configuration using the IIT 2 is shown, but the configuration using the IIT 22 as shown in FIG. 5 may be used. The IIT 22 includes a photocathode 22a that converts an incident optical image into a photoelectron image, an MCP 22c that multiplies an electron group of the photoelectron image, and a phosphor screen 22d that generates an amplified optical image from the multiplied electron group.

  INDUSTRIAL APPLICABILITY The present invention can be used, for example, in an image acquisition device or the like used for guarding, monitoring, wildlife observation, support for moving body operation, high-speed phenomenon observation, or the like in a dark field.

It is a figure which shows the external appearance structure of the night vision apparatus which is one Embodiment of this invention. It is a flowchart for demonstrating a gain control method. It is a flowchart for demonstrating a display method. It is a schematic block diagram which shows the night vision apparatus using a multistage image tube. It is a schematic block diagram which shows another example of IIT.

Explanation of symbols

2 Image intensifier 2a Photocathode 2c MCP (electron multiplier)
2d phosphor screen 3 wide dynamic range CMOS (imaging device)
4 Signal Processing Unit 10 Night Vision Device (Image Acquisition Device)
110 Monitor (display)

Claims (5)

An image-in having a photocathode for converting an incident optical image into a photoelectron image, an electron multiplier for multiplying the electron group of the photoelectron image, and a phosphor screen for generating an amplified optical image from the multiplied electron group With the tensiifier,
An image sensor having a plurality of photoelectric conversion elements arranged two-dimensionally and on which the amplified optical image is formed;
Based on the luminance values obtained from the plurality of photoelectric conversion elements, an image is displayed on the display unit, and a gain signal for adjusting the gain of the image intensifier so as to suppress halation and blackout of the image is obtained. An image acquisition device comprising a signal processing unit for output,
The image sensor is a CMOS,
The signal processing unit is configured to calculate a difference between a maximum luminance value in luminance values obtained from a plurality of photoelectric conversion elements, an average luminance value in luminance values obtained from the plurality of photoelectric conversion elements, and an average luminance value from the maximum luminance values. An image acquisition device characterized by adjusting a gain of the image intensifier based on the difference luminance value.   The image acquisition apparatus according to claim 1, wherein a dynamic range of the CMOS is 100 dB or more. In the CMOS, the photoelectric conversion elements are arranged in M rows and N columns,
The signal processing unit increases the gain of the image intensifier based on the difference luminance value when the maximum luminance value is less than a first setting threshold value,
On the other hand, it is set that the maximum luminance value is not less than a first setting threshold value and the line maximum luminance value in the luminance value obtained from the photoelectric conversion element in the row direction or the column direction is not less than the second setting threshold value. 3. The image acquisition device according to claim 1, wherein a gain of the image intensifier is lowered based on the difference luminance value when the row or setting column is continuous. 4.   The signal processing unit includes a luminance value obtained from one photoelectric conversion element and a luminance value obtained from the photoelectric conversion element of X rows and Y columns (X <M, Y <N) centering on one photoelectric conversion element. Based on the corrected luminance value calculated from the partial maximum luminance value and the partial minimum luminance value in the value, by calculating a display luminance value obtained by locally correcting the luminance value obtained from one photoelectric conversion element, The image acquisition apparatus according to claim 3, wherein the image is displayed. The signal processing unit calculates a display luminance value using Equation (1),
The image acquisition apparatus according to claim 4, wherein the correction coefficient is increased so that the influence of the correction luminance value on the display luminance value becomes stronger as the adjusted gain becomes farther from the reference gain.
Display luminance value = (corrected luminance value × correction coefficient) + {luminance value × (1−correction coefficient)} (1)

Images