WO2012032916A1

WO2012032916A1 - Monitoring system

Info

Publication number: WO2012032916A1
Application number: PCT/JP2011/068758
Authority: WO
Inventors: 道輝柴原; 中村　和彦
Original assignee: 株式会社日立国際電気
Priority date: 2010-09-09
Filing date: 2011-08-19
Publication date: 2012-03-15
Also published as: JP2012080524A; CN203301595U

Abstract

The present invention has the objective of capturing a video image in which an object can be identified in a linear or columnar area to be monitored which is over a long distance, wherein particles of a diameter of approximately 1000 nm are present in the air. In a monitoring system in which such a leaky transmission path as a leaky coaxial cable, a combination of a ground plane and a transmission line, a twisted pair transmission line, a planar transmission path, a dielectric transmission path, or a waveguide has been laid along a railway line, a direct current of less than or equal to DC 36V and synchronization information are superimposed upon the leaky transmission path so that, on the basis of a power source and the synchronization information, a plurality of circuits for driving LEDs with individually phased pulses and LEDs of a wavelength which transmits through water vapor are placed along the railway line at predetermined intervals, and the electronic shutter phase of the CCD camera is varied in synchronization with the synchronization information of the leaky transmission path to perform image capture.

Description

Monitoring system

The present invention relates to an object monitoring system.

If there are particles with a diameter of about 1000 nm in the air, such as fog or dust, the short wavelength of visible light is diffusely reflected, and the black of a long-distance subject is raised and the contrast is reduced. Oxygen in the air slightly absorbs and attenuates about 680 nm of red and attenuates to 1/2 that of short wavelength visible light. It absorbs near-infrared light of about 760 nm and attenuates to 1/10 compared to short-wavelength visible light. Sunlight is absorbed and absorbed by water vapor etc. on the ground and the sea where the altitude is low, and red of about 700 nm is slightly absorbed and attenuated. Further, it absorbs and attenuates near-infrared light of about 820 nm, about 900 nm to about 1000 nm, about 1120 nm to about 1160 nm, and about 1300 nm to about 1500 nm. In particular, the transmittance of near infrared light in the vicinity of about 970 nm and about 1130 nm in the atmosphere is as small as 1/20 or less compared to visible light. Therefore, a bottle with water vapor absorbs near-infrared light, and the near-infrared light of a long-distance subject does not reach the camera (see Non-Patent Document 1).
Therefore, conventionally, yellow-orange illumination near 600 nm, such as a sodium lamp 590 nm, is used for visual observation, and the imaging device removes haze and other irregular reflections when imaging in the atmosphere in which fog or fog is generated. As a method, a target object is irradiated with a visible light short-time pulse LED (Light Emitting Diode), and irregularly reflected light other than a predetermined distance is removed by electronic shutter exposure, and only reflected light of the target object at a predetermined distance is photographed. It has been proposed (see Patent Document 1).
If you receive radio waves from four or more Global Positioning System (GPS) satellites or Quasi-Zenith Satellites (QZS), you can measure the position of each receiving point in cm, Time synchronization can be achieved with the accuracy of the mounted atomic clock.

Further, integration of digital signal processing circuits has progressed, and it is not only a memory-integrated DSP (Digital Signal Processor) dedicated to video that stores and arithmetically processes output signals of a plurality of lines, but also an inexpensive general-purpose FPGA (Field Programmable Gate). Array) can be realized easily.
Further, a CDS (Correlated Double Sampling) for removing noise from a signal output from a CCD (Charge Coupled Device), a dark current correction and variable gain amplification circuit (Automatic Gain Control, hereinafter referred to as AGC), and a digital video signal An AFE (Analog Front End) incorporating an ADC (Analog Digital Converter) that converts to Vi has been widespread. The gradation of ADC of AFE was 10 bits in the past, but 12 bits and 14 bits have become common. Furthermore, DNR (Digital Noise Reduction) ICs (Integrated Circuits) with built-in frame memories have also been released at a low price.

Further, an electron multiplying CCD image sensor (for example, Electron Multiplying-CCD: EM-CCD) can be combined with an electronic cooling unit to increase sensitivity. For this reason, in an imaging device (electron multiplying CCD imaging device) using the imaging element, it is possible to obtain a quasi-moving image without illumination at night or in a dark place by photographing with visible light and near infrared light. Especially used for monitoring purposes.
In addition, an image pickup device such as a CCD image pickup device in which the thickness from the surface of the on-chip lens to the photodiode is thinned to be highly sensitive to visible light is also commercially available. An image sensor that incorporates a light guide from the surface of an on-chip lens such as a CMOS image sensor to a photodiode and has high sensitivity to short wavelength light is also commercially available. An image sensor having a high sensitivity to short wavelength light by reducing the thickness from the surface of the on-chip lens to the photodiode as backside illumination of a CMOS image sensor or the like is also commercially available. Further, an imaging device including a deep photodiode or a photodiode having a fine structure that reflects near infrared light and having high sensitivity to near infrared light is also commercially available. High-speed global electronic shutter CMOS and CCD image sensors are also commercially available (see Non-Patent Document 2 to Non-Patent Document 4). Further, an electron multiplying CMOS image sensor has been developed. A CCD imaging device using InGaAs or the like and sensitive to near infrared light up to 1700 nm is also commercially available (see Non-Patent Document 5).
In the above-described prior art, if there are particles having a diameter of about 1000 nm in the air, whether it is fog or dust, the short wavelength of visible light is irregularly reflected, the black of a long-distance subject is raised, the contrast is reduced, and photographing is difficult. Furthermore, in a mountainous area without street lights, it is difficult to photograph even an EM-CCD or a new image sensor on a cloudy night and a track without sunlight, moonlight, or starlight. Further, in the thermal image of the far-infrared camera, the correlation with the visible light image is low and the degree of modulation is low, so that it is difficult to identify an intruding object such as an individual even though the intruding object can be detected.

JP 2009-94974 A JP 2003-259385 A

In view of the above problems, an object of the present invention is to capture an image capable of specifying an intruding object in a monitoring target region having particles having a diameter of about 1000 nm in the air.

In order to solve the above-described problems, the present invention has a lens that transmits at least visible light or near infrared light, and an imaging device that is sensitive to at least visible light or near infrared light, and picks up at least visible light or near infrared light of a subject. When the effective contrast of the video signal falls below the expected value, yellow-orange, red, or near-infrared LEDs that transmit gas molecules in the air emit time-synchronized light with a narrow ON ratio by transmitting synchronization information. A monitoring system that outputs a video imaged by performing a global electronic shutter synchronized with the transmitted synchronization information by the imaging device.
Further, the monitoring system includes horizontal contour correction generated from seven or more pixel delayed video signals and vertical contour correction generated from seven or more scanning line delays, and captures at least visible light and near infrared light of a subject. If the effective contrast of the video signal (such as a decrease in the highlight level or an increase in the black level) falls below the expected value, the black contour is lowered and a horizontal contour generated from seven or more pixel delayed video signals The monitoring system is characterized by lowering the correction emphasis frequency and increasing the number of emphasized scanning lines for vertical contour correction generated from seven or more scanning line delays.
In the monitoring system, the imaging device is a CCD imaging device in which an emission wavelength band pass optical filter of the LED is inserted in an incident optical path.
Further, in the monitoring system, the synchronization information may be a leaky coaxial cable laid in the vicinity of a long-distance line or row monitoring target area such as a railroad track, a road, a transmission line site, or a wind turbine row of wind power generation. A combination of a ground plane and a transmission line, a twisted pair transmission line, a planar transmission line, a dielectric transmission line, or a leaky transmission line such as a waveguide, and the like, A radio wave radiated in the vicinity of the monitoring target area from a directional transmitting antenna installed in the vicinity, an optical cable laid in the vicinity of the monitoring target area, and a high-precision position detection function such as GPS and a quasi-zenith satellite The monitoring system is characterized by being transmitted by at least one of radio waves including an accuracy synchronization function.
Furthermore, in the above monitoring system, the power source of the near-infrared LED is laid on the line, the superimposed power source of the leaky transmission line such as the leaky coaxial cable laid near the monitored region, the power line of the optical cable with the power line, and the line Supplied by at least one of a power line charged with a power line charged in the vicinity of the monitored area, an electric power line laid in the vicinity of the monitoring target area, and solar power generation, solar thermal power generation, wind power generation, or vibration of passing objects. It is a monitoring system.

Further, in a monitoring system in which a leaky transmission line such as a leaky coaxial cable is laid along the monitoring target area, a power supply (DC 36V or less direct current) is superimposed on the leaky transmission line of the leaky coaxial cable or the like, There are a plurality of LEDs that drive the LEDs with individual phase pulses based on the power supply and synchronization information along the line at a predetermined interval, and global electronics that synchronize with the synchronization information of the leaky transmission line such as the leaky coaxial cable. A solid-state imaging device having a solid-state imaging device that varies the shutter phase, and synchronization of a leaky transmission line such as the leaky coaxial cable (as in the case of photographing an intruder illuminated by an LED driven by the pulse of the individual phase) The monitoring system is characterized in that the global electronic shutter phase is varied in synchronization with information.
Further, in the monitoring system, the LED transmits oxygen and water vapor at a yellow-orange wavelength (about 600 nm), a red wavelength (about 660 nm) or near infrared light wavelength other than the oxygen absorption wavelength and the water vapor absorption wavelength. An LED having a wavelength (about 740 nm, about 780 nm, about 860 nm, about 1040 nm, about 1200 to about 1250 nm, about 1550 to about 1650 nm), and the solid-state imaging device transmits only the emission wavelength of the LED One of an optical LPF that transmits a wavelength longer than the emission wavelength of the BPF or the LED, and a structure and a light guide from the surface of the on-chip lens having a thickness less than 5 times (about 3.5 μm) of the red end wavelength of about 700 nm to the photodiode And photodiodes that are 8 times deeper than the red edge wavelength of about 700 nm (about 5.6 μm) and photodiodes A Si imaging element or an InGaAs imaging element having at least one of a fine structure that reflects near-infrared light and an electron multiplier electrode, and the solid-state imaging device has an optical filter function for imaging only the emission wavelength of the LED It is the monitoring system characterized by being an imaging device having.

According to the present invention, it is possible to capture an image capable of specifying an intruding object in a monitoring target region where particles having a diameter of about 1000 nm are present in the air.

1 is a block diagram (EM-CCD) showing a configuration of a line monitoring system according to an embodiment of the present invention. 1 is a block diagram (FIT-CCD) showing a configuration of a line monitoring system according to an embodiment of the present invention. 1 is a block diagram (IT-CCD) showing a configuration of a line monitoring system according to an embodiment of the present invention. Timing chart (FIT-CCD or EM-CCD) showing the operation of the line monitoring system of one embodiment of the present invention Timing chart (IT-CCD) showing the operation of the line monitoring system of one embodiment of the present invention The flowchart which shows operation | movement of the monitoring system of the track | line of one Example of this invention. Block diagram showing the configuration of wrinkle correction in a conventional monitoring system Timing chart showing operation of electronic shutter for wrinkle correction of conventional monitoring system The block diagram which shows the structure of the video signal processing part of one Example of this invention. The block diagram which shows the structure of the multi-pixel outline correction | amendment and correlation average part of one Example of this invention Schematic diagram showing multi-pixel contour correction and correlation average operation of one embodiment of the present invention The schematic diagram which shows operation | movement of the black level correction | amendment part of one Example of this invention. The schematic diagram which shows operation | movement of the black level correction | amendment part of one Example of this invention.

FIG. 1 is a block diagram showing the configuration of a line monitoring system according to one embodiment of the present invention and a leaky coaxial cable of the line monitoring system according to one embodiment of the present invention. 2A (FIT-CCD or EM-CCD) and FIG. 2B (IT-CCD) of timing charts showing the synchronization information, LED emission, and operation of the electronic shutter, and the line monitoring system of one embodiment of the present invention The operation will be described with reference to FIG.
The difference between FIG. 1A of the block diagram showing the configuration of the line monitoring system of one embodiment of the present invention and FIG. 3 of the block diagram showing the configuration of wrinkle correction of the conventional monitoring system is that transmission is performed with a leaky coaxial cable or ground plane. In combination with a line, twisted pair transmission line, planar transmission line, dielectric transmission line, or synchronization information of a leaky transmission line such as a waveguide, multiple LEDs are driven in multiple phases on the subject side in the air Emits yellow-orange (about 600 nm) or red (about 660 nm) or near-infrared (about 740 nm, about 780 nm, about 860 nm, about 1040 nm, about 1200 to about 1250 nm, about 1550 to about 1650 nm) that transmits a gas molecule of And that the imaging element drive phase is advanced from the output video signal based on the synchronization information of the leaky coaxial cable.

1A, 1B, and 1C are block diagrams showing the configuration of a line monitoring system according to an embodiment of the present invention. In FIG. 1, 1 is a lens, 2 is an EM-CCD, 3 is a FIT-CCD, and 4 is a video signal with memory. Processing unit, 5 is a CPU, 6 is an OD drive unit, 7 is a vertical transfer drive unit of an imaging unit, 8 is a vertical transfer drive unit of a storage unit, 9 is a TG, 10 is a CMG drive unit, 11 is a CDS, AGC, and A AFE including / D, 12 LED, 13 LED drive unit, 14 leaky transmission line such as leaky coaxial cable, and 15 IT-CCD. Reference numeral 16 is a temperature sensor, 17 is a cooling unit, and 18 is a cooling drive unit. Reference numeral 39 denotes an insertion / removal optical filter that inserts or removes the light emission wavelength band-pass optical filter of the LED into or from the incident optical path. The insertion / removal optical filter 39 is required when there is very strong external light such as foggy morning direct sunlight, and there are particles with a diameter of about 1000 nm in the air, whether it is fog or dust.
3 FIT-CCD or 15 IT-CCD, 6 OD drive unit, 7 vertical transfer drive unit of imaging unit, 8 vertical transfer drive unit of storage unit, 9 TG, 10 CMG drive unit, 11 The CDS, AGC, and AFE including A / D may be integrated into a high-sensitivity solid-state imaging device that changes the global electronic shutter phase, for example, a CMOS imaging device having a global electronic shutter function. The EM-CCD 2 may be integrated into a CMOS image sensor having a global electronic shutter function and an electron multiplication function. In any case, the imaging device is preferably an imaging device that is highly sensitive to visible light or near infrared rays, such as Non-Patent Document 2 to Non-Patent Document 5.

Reference numerals

21, 22 and 23 denote antennas, 24 denotes synchronization information radio waves, 25 denotes an overhead line, 26 denotes an optical fiber, and 27 denotes a lamp line. The combination of each element of FIG. 1A, FIG. 1B, and FIG. 1C is not restricted to illustration, Any combination may be sufficient.

2A (FIT-CCD or EM-CCD) and FIG. 2B (IT-CCD) of timing charts showing the operation of the line monitoring system according to one embodiment of the present invention, and an electronic shutter for correcting wrinkles of a conventional monitoring system. The timing chart showing the operation is different from FIG. 4 in that the LED emission is on the subject side, corresponding to the synchronous phase of the leaky coaxial cable laid along the railway line, and in a plurality of phases corresponding to the position of the railway line. This means that the CCD drive phase is advanced from the output video signal.
FIG. 2A (FIT-CCD or EM-CCD) of the timing chart showing the operation of the line monitoring system according to the embodiment of the present invention, the LED and the attenuated sun rather than the subject signal charge due to the attenuated sunlight. The subject signal charge due to light is large, and the subject can be photographed even when fog is generated at night or at night.
As shown in the lower part of FIG. 2B (IT-CCD) of the timing chart showing the operation of the track monitoring system according to the embodiment of the present invention, the subject signal charge due to the attenuated sunlight on the previous screen is caused by the attenuated sunlight on the current screen. Since it is almost the same as the subject signal charge, the result of subtracting the video signal of the subject signal charge due to the attenuated sunlight on the previous screen from the video signal of the subject signal charge due to the current screen LED and the attenuated sunlight is It becomes an electric charge.
Therefore, if the subject does not move quickly, the subject can be photographed even when fog is generated at night or at night.

That is, according to the present invention, in a (Shinkansen) line monitoring system in which a leaky coaxial cable is laid along a line, a power supply and synchronization information are superimposed on the leaky coaxial cable (DC 36V or less) and the synchronization information A solid-state imaging device that has a plurality of LEDs driven by individual phase pulses and a plurality of LEDs along the line at predetermined intervals, and changes the global electronic shutter phase in synchronization with the synchronization information of the leaky coaxial cable. And the global electronic shutter phase is varied in synchronization with synchronization information of the leaky coaxial cable (as in the case of photographing an intruder illuminated by the LED driven by the individual phase pulse). It is a monitoring system.
Regardless of whether day or night fog occurs, the position of the railway track, etc., corresponds to the synchronous phase of the leaky coaxial cable laid along either the railroad track or road or transmission line site or wind turbine row of wind power generation. Since a plurality of LEDs emit light at a plurality of phases corresponding to the above, a subject such as a railway track at a position corresponding to the global electronic shutter phase can be photographed.

FIG. 2C of the flowchart of one embodiment of the present invention, FIG. 5 of a block diagram showing the configuration of the video signal processing section of the first embodiment of the present invention, and multi-pixel contour correction and correlation averaging section of one embodiment of the present invention FIG. 6 is a block diagram showing the configuration of FIG. 6, FIG. 7 is a schematic diagram showing the operation of multi-pixel contour correction and correlation averaging in one embodiment of the present invention, and the operation of the black level correction unit in one embodiment of the present invention. Example 2 is demonstrated using FIG. 8A and FIG. 8B of the schematic diagram shown.

FIG. 2C is a flowchart of one embodiment of the present invention. After the start of 41, the determination of 42 “whether the dark part of the imaged video signal is greater than or equal to the first predetermined value” is No. Take an image and go to the end of 47. If the determination of 42 is Yes, the determination of 42 “is the highlight of the imaged video signal equal to or greater than the second predetermined value”? If Yes, the black level of the output video signal VideoOut of 46 is lowered and 48 multi-pixel contours are determined. The horizontal / vertical frequency of correction is lowered and the multi-pixel contour correction amount is increased, and the process goes to 47. If the determination in 42 is No, 45 leaky coaxial-synchronized LED wavelength bandpass optical filters are inserted, and leaky coaxial-synchronized electronic shutter operation is performed to reduce the black level of 46 output video signal VideoOut and 48 multi-pixel contour correction The horizontal / vertical frequency is decreased and the multi-pixel contour correction amount is increased.

FIG. 5 is a block diagram showing the configuration of the video signal processing unit of one embodiment of the present invention, and the video signals of FIG. 1B and FIG. 1C of the monitoring system using the solid-state imaging device of the first embodiment of the present invention. This corresponds to the processing unit 4A. 5A of FIG. 5 may be used for the video signal processing unit 4 of FIG. 1A. FIG. 6 is a block diagram showing the configuration of the multi-pixel contour correction and correlation average unit according to one embodiment of the present invention, which corresponds to 31 in FIG. FIG. 7 is a schematic diagram showing operations of multi-pixel contour correction and correlation averaging according to one embodiment of the present invention, and FIG. 8A is a schematic diagram showing operations of a black level correction unit according to one embodiment of the present invention. 8B is a schematic diagram illustrating the operation of the black level correction unit according to the first embodiment of the present invention.

The difference between the video signal processing unit 4 of one embodiment of the present invention and the block diagram of the monitoring system using the conventional solid-state imaging device in FIG. 7 is that there are 8 hygrometers and 10 optical filter pass wavelengths. And all-pixel dark current variation correction, correlation average, multi-pixel contour correction, black level is lowered and the gamma correction of the dark part is weakened, the compression of the dark part is restricted (dark part correction), or the highlight compression is weakened (highlight correction). 4 of the video signal processing unit including contrast enhancement for performing at least one of the above. In other words, although details will be described later, noise reduction by all-pixel dark current variation correction and correlation average, multi-pixel contour correction and black level are lowered to weaken the gamma correction of the dark area or to limit the compression of the dark area (dark area correction). The contrast enhancement that performs at least one of reducing the highlight compression (highlight correction) is processed in 14 bits in the video signal processing unit 4.

FIG. 3 shows an image pickup unit 3 using a CCD of a solid-state image pickup device, all-pixel dark current variation correction, correlation average, multi-pixel contour correction, black level is lowered, and dark portion gamma correction is weakened or the dark portion is compression limited ( This is an example of the video signal processing unit 4 including contrast enhancement that performs at least one of dark portion correction) and weakening highlight compression (highlight correction). A CMOS image sensor including all functions of the imaging unit 3 and the video signal processing unit 4 is provided. It may be used. In the imaging unit 3 of FIG. 3, 6 is a CPU, 12 is a vertical transfer drive unit, 22 is a cooling unit, 23 is a cooling drive unit, and 24 is a temperature sensor. In FIG. 3, all-pixel dark current variation correction, correlation average, multi-pixel contour correction, black level is lowered, and gamma correction in the dark part is weakened, compression of the dark part is restricted (dark part correction), or highlight compression is weakened (highlight correction). ) In the video signal processing unit 4 including contrast enhancement for performing at least one of the above, 25 is a contrast detection unit including OB detection, 26 is an all-pixel reference dark current screen memory, 27 is a multiplier, 28 is a subtractor, and 29 is a peripheral pixel 30 is a multi-pixel contour correction unit including a correlation average between the pixel and the surrounding pixels of the previous screen, and 30 reduces the black level and weakens the gamma correction of the dark part, restricts the compression of the dark part (dark part correction), or weakens the highlight compression ( A video signal processing unit including a contrast enhancement unit that performs at least one of (highlight correction).

The incident light Lin in FIG. 1A passes through an optical filter, is converted into an electric signal by the CCD 11 in FIG. 3, and is converted into an image signal Vi inside the imaging apparatus of about 14 bits by an analog front end (AFE) 13. The temperature sensor 24 in FIG. 3 detects the temperature of the CCD 11 and calculates the amount of change in dark current. When the CCD 11 is an EM-CCD, the amount of change in dark current is determined by the product of temperature and charge multiplication. As will be described in detail later, the contrast detection unit 25 that includes OB detection uses a dark current from the H-OB average value of V-OB. Is calculated. The reference dark current dispersed in all pixels stored in the all pixel reference dark current screen memory 26 is multiplied by the amount of change in dark current by the multiplier 27 to calculate the dark current varied in all pixels. Subtract from the signal Vi to correct the dark current variation of all pixels. Although details will be described later, the correlation average and the multi-pixel contour correction are performed in the multi-pixel contour correction unit 29 including the correlation average between the peripheral pixels and the peripheral pixels in the previous screen, and the black level is lowered to reduce the dark portion gamma correction or the dark portion. Whether to reduce the black level and weaken the gamma correction of the dark part in the video signal processing unit 30 including the contrast enhancement part that performs compression restriction (dark part correction) or weakening the highlight compression (highlight correction) Contrast is enhanced by limiting the compression of the dark part (dark part correction) or weakening the highlight compression (highlight correction), and the output video signal Vo of about 8 bits is obtained by performing gamma correction and knee correction.

In the following, at least one of lowering the black level of the video signal and weakening the gamma correction of the dark part, limiting the compression of the dark part (dark part correction), or weakening the highlight compression (highlight correction) according to one embodiment of the present invention. FIG. 4A and FIG. 4B showing the correspondence between the brightness enhancement distribution of the incident light, the luminance distribution of the internal video signal Vi, and the luminance distribution of the output video signal Vo in the solid-state imaging device according to the embodiment of the present invention. And will be described.

Even if the dark part of the incident light rises slightly, the contrast of the output video signal Vo can be secured by increasing the gamma correction by lowering the black level of the video signal. If the dark part of the incident light rises to around 25% of the rating (50-60% of the output signal), the contrast of the output can be secured without lowering the black level of the video signal and increasing the gamma correction. When the dark part of the incident light rises to around 30% of the rating (55 to 65% of the output signal), the dark part correction that reduces the black level of the video signal to reduce the gamma correction of the dark part and compresses and restricts the dark part is more output. Contrast can be secured. Video signal processing that reduces the black level of the video signal to reduce the gamma correction in the dark part and compresses and restricts the dark part is more effective when performed on each part of the screen or on a pixel basis than on the entire screen.
Also, if the iris control output is generated using a video signal with a constant dark part level, the iris will not be narrowed down, the highlight level will not decrease much, and it will recover to about 85% of the rating (85 to 95% of the output signal). To do.

4A and 4B show the correspondence between the luminance distribution of the incident light, the luminance distribution of the internal video signal Vi, and the luminance distribution of the output video signal Vo for the contrast enhancement of a predetermined portion of the screen of the imaging apparatus according to the embodiment of the present invention. Show. In FIG. 4A, the black level of the video signal is greatly lowered to around −50% of the rating to reduce the gamma correction in the dark part, and the dark part level of the video signal is lowered from about 55% to about 15% of the output signal. Also, increase the gain, open the aperture, or slow down the electronic shutter to increase the dark part of the incident light and decrease the highlight due to lowering the black level of the video signal. Recovery from about 80% of the signal to about 70% of the rating and about 85% of the output signal. The amplitude of the luminance output signal is about 70% 105 gradations and about 6.8 bits. Further enhances the edge enhancement.

FIG. 2B shows correspondence between the luminance distribution of incident light and the luminance distribution of the output video signal Vo of an imaging apparatus according to another embodiment of the present invention. In FIG. 2B, an increase in the dark level of the input video signal is detected, the black level of the video signal is greatly reduced to around −100% of the rated value, dark portion compression and dark portion restriction are performed, gamma correction in the dark portion is reduced, Edge enhancement is strengthened to make the dark level of the output video signal constant (about 5% in FIG. 2B). An iris control output is generated using a video signal in which the dark level of the video signal is constant, and the highlight of the output signal is restored. Furthermore, it detects a decrease in the highlight level of the input video signal, weakens the highlight compression, increases the highlight color expansion, increases the highlight enhancement of the highlight, and keeps the highlight level of the output video signal constant (Fig. 4 is about 95%). In a video signal in which contrast is ensured, the edge enhancement of dark portions and highlights is weakened.

The block diagram showing the configuration of the video signal processing unit according to the first embodiment of the present invention is an example of correcting the decrease in the modulation degree from the low frequency range with the soot, fog, or fine dust according to the first embodiment of the present invention. FIG. 5, FIG. 6 is a block diagram showing the configuration of the multi-pixel contour correction and correlation average unit of one embodiment of the present invention, and a schematic diagram showing the operation of multi-pixel contour correction and correlation average of one embodiment of the present invention. This will be described with reference to FIG.
In FIG. 5, 28 is an OB detection unit, 29 is an all pixel reference dark current frame memory, 30 is a video signal processing unit including black level detection, 31 is a pixel delay of 6 or more, line memory of 6 or more, and peripheral pixels including a frame memory. It is a correlation adaptive average with a surrounding pixel of a front screen, and a multi-pixel outline correction part.
In FIG. 6, 32 to 38 are pixel delay units, 59 is a video level determination unit, 41 and 42 are contour signal generation units, 44 is a correlation averaging unit, 50 to 58 are adders, 53 is positive / negative and amplification degree. Variable multiplier 61, compression limiter 61 for small amplitude and large amplitude, 55 for contour correction unit, 56 for switch, 60 for subtractor, M1 to M6 for line memory, M7 for screen memory, N0 to N6 for negative A multiplier, P1, is a positive multiplier.

In FIG. 7, the pre-correction signal is delayed by the scanning line (H) period in the line memory units M1 to M6 in the in-screen delay unit 31, and becomes a total 7H signal from 0H to 6H. The 3H signal further becomes a total of 7 sets of delayed signals by pixel time, that is, CCD clock time, in 6 pixel delay portions of 38 pixels. The total 7H signal and the total seven delay signals enter the contour

signal generation unit

41 and 42, become a vertical contour signal and a horizontal contour signal, are added by the adder 51, and have a small amplitude and large amplitude compression limit. The small amplitude and the large amplitude are compressed and limited by the unit 61, and a contour correction signal is generated by the positive / negative multiplier 53 under the control of the video level determination unit 40 to which the 3H3 pixel delay signal is input, and the 3H3 pixel delay signal or the correlation average addition 3H3 The corrected signal is added to the pixel delay signal.
As a result, the modulation factor decreases from the low frequency as shown in FIG. 7A of the schematic diagram showing the multi-pixel contour correction and correlation average operation of the embodiment of the present invention, as in the signal before correction from the low frequency to the low modulation factor. However, (b) the contour correction 7 pixel component, (c) the contour correction 5 pixel component, and (d) the contour correction 3 pixel component are combined, and (e) the signal after the present invention correction, The contour can be corrected. In other words, the contour can be reproduced even when the modulation degree is reduced from a low frequency.

Further, a total of 7H signals are delayed by a pixel time, that is, a CCD clock time by 6 to 32 pixel delays, and a total of 49 delay signals are obtained for each H by 7 sets. Further, the pre-correction signal is delayed in the vertical scan (V) period by M7 of the screen memory, and is delayed in the scanning line (H) period in the in-screen delay unit 43 in the same manner as in the in-screen delay unit 31. A total of 49 delay signals are obtained by delaying the pixel time, that is, the CCD clock time, and a total of 7 sets for each H.
A total of 98 delay signals of 49 delay signals from the in-screen delay unit 31 and 49 delay signals from the in-screen delay unit 43 are calculated by the correlation averaging unit 44 to correlate with the 3H3 pixel delay signal. Then, the signals with high correlation among the 98 delayed signals are weighted and averaged. As a result, noise is reduced.
Further, as shown in FIG. 7A of the schematic diagram illustrating the multi-pixel contour correction and the correlation average operation of the embodiment of the present invention, a dark current called a white defect is generated as in the signal before correction of a low modulation frequency from a low frequency. Even if there are abnormally large pixels or pixels with dark currents called black scratches, the left and right pixels rarely become white scratches or black scratches. When averaged, there is no contour correction component when there is no correlation average between (d) the contour correction three-pixel component dotted white circle or black scratch, and (d) the contour correction three-pixel component and (e) the signal after correction of the present invention. The effect of white and black scratches is almost eliminated.
When the dark level of the video signal of the visible light and near-infrared light imaging of the subject exceeds a predetermined level, increase the number of scanning lines serving as the enhancement center of the vertical contour correction of the multi-pixel contour correction, and the multi-pixel contour. Lower the frequency that is the center of enhancement for correction of horizontal contour correction, strengthen multi-pixel contour correction, strengthen correlation average, lower black level, and weaken gamma correction in dark areas And / or limiting the compression of the dark part.

In order to ensure Vo S / N after contrast enhancement, AFE 12 and all-pixel dark current variation correction, correlation average, multi-pixel contour correction, black level are lowered, and gamma correction of the dark part is weakened or compression of the dark part is restricted. The video signal processing unit 4 including contrast enhancement that performs at least one of (dark part correction) and highlight compression (highlight correction) is the video signal processing of FIG. 2 without increasing the AFE 13 amplification. In order to secure the dynamic range, at least 8 bits + 2 bits and 10 bits or more are necessary, and 12 bits or more are preferable. When the AFE amplification of 13 is increased by 4 bits (16 times), the AFE needs 10 bits + 4 bits and 14 bits or more to secure S / N. The same applies to the imaging units 3A, 3B, 3C, and 3D in FIGS. 1B and 1C.

In other words, in response to the synchronous phase of the leaky coaxial cable laid along either the railroad track or road or power transmission line site or wind turbine row, multiple phases with multiple phases corresponding to the position of the railroad track etc. Since the LED emits light, the black level of the video signal is lowered to weaken the gamma correction of the dark portion or the dark portion is compression-reduced in the first embodiment where the subject such as the railway line at the position corresponding to the global electronic shutter phase is photographed. By combining the second embodiment of the contrast enhancement correction that performs at least one of performing (dark part correction) and weakening highlight compression (highlight correction), it is possible to identify a subject in a thicker fog or a longer distance fog A simple image can be taken, and intruder detection can be easily monitored.

In Example 1 above, about 600 nm, about 660 nm, about 740 nm, about 780 nm, about 860 nm, about 1040 nm, about 600 nm, about 660 nm, about 740 nm of visible light or near-infrared light having a wavelength that passes through molecules (water vapor and oxygen) in air. The band passes through a narrow band of 1200 to about 1250 nm and about 1550 to about 1650 nm, and the energy of incident light is attenuated. Therefore, one of the following effective sensitivity improvement measures is required.
Incident light is increased using a lens with a large aperture stop. A lens with a large aperture is large and expensive.
As described in Non-Patent Document 3 described above, a high sensitivity (about +4.5 dB) is obtained by using a photodiode Si imaging device having a depth of about 5.6 μm or more. Alternatively, as in Non-Patent Document 2 described above, a Si imaging element that increases the absorption rate by confining near-infrared light by forming a fine structure that reflects near-infrared light under the photodiode (about +9 dB) is used. High sensitivity. A high-sensitivity imaging of about 400 to 1200 nm is possible even with a Si imaging element having a fine structure. Further, as in Non-Patent Document 4 described above, a high sensitivity (+40 dB or more) is obtained by using an image pickup device having a charge multiplication electrode. Improvements in image sensors are advancing due to advances in microfabrication. In any case, since the dark current is conspicuous, it is necessary to cool the image sensor or to correct the dark current variation in units of pixels.

The dark current of the image sensor is proportional to the exponential function of temperature. Specifically, the dark current of the Si image sensor doubles with a 6 ° C. temperature rise. Therefore, the temperature of the image sensor is detected by the temperature sensor, and the value obtained by subtracting the image sensor temperature at the time of measuring the reference dark current of all pixels on the imaging surface at the time of non-multiplication stored from the detected temperature is 2 By calculating the power and multiplying the stored reference dark current of all the pixels on the imaging surface at the time of non-multiplication and the amplification factor of 11 of the AFE, all the pixels on the imaging surface at the time of the current non-multiplication are calculated. A dark current correction value can be calculated.

In EM-CCD 2, the amount of change in dark current is determined by the product of temperature and charge multiplication. Therefore, a value obtained by averaging H-OBs in the vertical optical black pixel (V-OB) line of the CCD image sensor or a minimum value of H-OB in the V-OB line is This is a representative value of the dark current of H-OB in the V-OB line. The H-OB in the V-OB line has neither a vertical smear component nor a horizontal smear component. Therefore, the video signal processing unit detects the representative value of H-OB in the current V-OB line and stores the detected representative value of H-OB in the current V-OB line. Divide by the reference representative value of H-OB in the V-OB line at the time of multiplication and multiply by the stored reference dark current of all pixels on the imaging surface at the time of non-multiplication. A correction value for dark current of all pixels can be calculated. By subtracting the calculated dark current correction value for all pixels on the current imaging surface from the video signal, variations in dark current for all pixels on the imaging surface can be corrected, S / N can be improved, and sensitivity can be effectively improved. To do.
Alternatively, amplification of AFE 11 is increased, noise is reduced by adaptive averaging of peripheral pixels and previous screen pixels using a multi-line memory and a field memory, and effective (about 12 dB) high sensitivity is obtained. When increasing the amplification of AFE 11 by 4 bits (16 times), AFE needs 8 bits + 4 bits, 12 bits or more, and 14 bits or more is preferable to secure S / N. All-pixel dark current variation correction, correlation average, multi-pixel contour correction, black level is lowered to weaken gamma correction in dark areas, to limit compression of dark areas (dark area correction), or to weaken highlight compression (highlight correction) It is preferable that the video signal processing unit 4 including the contrast enhancement that performs at least one of them is equal to or more than the same bit as the AFE.

If the backside illumination structure in which the short-wavelength light described in Non-Patent Document 2 reaches the photodiode without being attenuated is applied to an imaging device having at least the photodiode made of InGaAs, high-sensitivity imaging of about 400 to 1700 nm is possible. If a light guide built-in structure in which short-wavelength light reaches the photodiode without being attenuated at least is applied to an imaging device made of InGaAs at least, high-sensitivity imaging of about 600 to 1700 nm is possible. If a light guide with good transmittance of short wavelength light and a light guide built-in structure is applied to an image sensor made of at least a photodiode of InGaAs, high-sensitivity imaging of about 400 to 1700 nm is possible. When the photodiode is made of InGaAs, high-sensitivity imaging with a wavelength of about 900 to 1700 nm is easy, and with Si other than the photodiode, microfabrication is easier.

Aspherical lens using glass with Abbe's number of 95 or more and almost no dispersion. Anti-reflection coating for broadband wavelength. Zoom with focal plane variation when the focal length is changed is mechanically corrected with a cam or motorized. If a varifocal lens equivalent to a lens is manufactured, the number of surfaces on which light is reflected at the boundary between air and the lens is reduced, the aberration is reduced, and an image with a high contrast of about 400 to 1700 nm is possible.

In other words, the present invention superimposes the power source and the synchronization information on the leaky coaxial cable, and the LED has a narrow band of only wavelengths that transmit oxygen and water vapor (about 600 nm, about 660 nm, about 740 nm, about 780 nm, about 860 nm). Passing, the energy of the incident light is attenuated. Therefore, a high-sensitivity image is taken with the above-described effective sensitivity improvement measure using a Si image sensor, and the dark current is conspicuous, so the image sensor is cooled or the variation in dark current is corrected on a pixel basis. Then, in the fog, the signal charges of the incident light of the fog other than the line position to be monitored are swept away by the electronic shutter, so that only the subject image signal emitted by the LED at the line position to be monitored is output.

Alternatively, the present invention superimposes power supply and synchronization information on a leaky coaxial cable, and narrow band pass of only the near infrared emission wavelength (about 1040 nm, about 1200 to about 1250 nm, about 1550 to about 1650 nm) of the LED. Thus, the energy of the incident light is attenuated. Therefore, an InGaAs imaging device is used to capture an image with high sensitivity by the above-described effective sensitivity improvement measure. Since the dark current is conspicuous, the imaging device is cooled or the variation in dark current is corrected on a pixel basis. Then, in the fog, the signal charges of the incident light of the fog other than the line position to be monitored are swept away by the electronic shutter, so that only the subject image signal emitted by the LED at the line position to be monitored is output.

In other words, in response to the synchronous phase of the leaky coaxial cable laid along either the railroad track or road or power transmission line site or wind turbine row, multiple phases with multiple phases corresponding to the position of the railroad track etc. Since the LED emits light, the above-described plurality of phases can be obtained by combining the above-described third embodiment of the effective sensitivity improvement measure with the first embodiment in which the subject such as the railway line at the position corresponding to the global electronic shutter phase is photographed. The light emission intensity of the LED that emits light can be reduced, and the degree of freedom of power supply increases.
Further, in the first embodiment and the third embodiment, at least whether the black level of the video signal is lowered and the gamma correction of the dark portion is weakened, the compression of the dark portion is restricted (dark portion correction), or the highlight compression is weakened (highlight correction). By combining the second embodiment of contrast enhancement correction that performs one, it is possible to capture a identifiable image of a subject in a thicker mist or a longer distance mist, which makes it easier to monitor intruder detection. In addition, it is possible to further reduce the light emission intensity of the LED that emits light in the plurality of phases.
Specifically, by combining the first embodiment and the third embodiment or the first embodiment, the second embodiment, and the third embodiment, the power is superimposed on the leaky coaxial cable, or the overhead cable of the train, the power transmission line, and the optical cable with the power supply line. LED that emits light in the above-mentioned multiple phases can be obtained simply by charging vibration power generation of passing objects such as solar power generation, solar thermal power generation, wind power generation, trains or automobiles without supplying power from It becomes possible to drive.

Furthermore, if you receive radio waves from four or more GPS satellites or quasi-zenith satellites (hereinafter referred to as GPS satellites), the position of each receiving point can be measured in cm, and the time of the atomic clock mounted on the satellites will be accurate. Can be synchronized. Therefore, each illuminator has a radio wave part such as a GPS satellite having a high-accuracy position detection function and a high-accuracy synchronization function by receiving a radio wave such as a GPS satellite, and each illumination also has an illumination driving part having a reception pulse lighting phase control function. Then, since a plurality of lights emit light at a plurality of phases corresponding to the position, it is possible to photograph a subject at a position corresponding to the global electronic shutter phase. As a result, the present invention can be applied to monitoring an object at an arbitrary point within the line-of-sight range.

Furthermore, if each illumination is set to pulse lighting synchronized with the monitor imaging screen cycle such as a sodium lamp 590 nm, or pulse lighting synchronized with the imaging screen cycle of yellow-orange LED illumination near 600 nm, each illumination is for visual confirmation. Shared with surveillance shooting.
Alternatively, if each illumination is pulsed in the near infrared, it can be applied to monitoring that is difficult to see. Specifically, it is difficult to see if the near-infrared wavelength of each illumination is about 850 nm, almost difficult to see if it is about 900 nm, and not visible if it is about 950 nm or more.

The present invention can be applied to the monitoring of an object existing in a long-distance line or row monitoring target area such as a railroad track, a road, a transmission line site, or a wind turbine row of wind power generation.
Furthermore, if each illumination has a high-accuracy position detection function such as GPS and quasi-zenith satellite, a high-accuracy synchronization function, and a pulse lighting phase control function, it can be applied to monitoring an object at an arbitrary point within the line-of-sight range. .
Further, the present invention can be applied to monitoring by pulse lighting of yellow-orange illumination for visual confirmation, or monitoring by pulse lighting of near infrared so as not to be visually confirmed.

1: Lens, 2: EM-CCD, 3: FIT-CCD,
4: Video signal processing unit including all-pixel dark current variation correction, correlation average, multi-pixel contour correction, and contrast enhancement,
5: CPU, 6: OD drive unit, 7: Vertical transfer drive unit of imaging unit, 8: Vertical transfer drive unit of storage unit,
9, 19: TG, 10: CMG drive unit, 11: AFE including CDS, AGC and A / D,
12: LED, 13: LED drive unit, 14: Leakage transmission line such as leaky coaxial cable,
15: IT-CCD, 16: temperature sensor, 17: cooling unit, 18: cooling drive unit,
20: video signal processing unit with memory,
21, 22, 23: antenna, 24: synchronization information radio wave, 25: overhead wire, 26: optical fiber,
27: Power line, 28: OB detection unit, 29: All pixel reference dark current frame memory,
30: Video signal processing unit including black level detection,
31: Correlation adaptive average and multi-pixel contour correction unit between peripheral pixels including 6 pixels or more, line memory 6 or more, frame memory, and peripheral pixels in the previous screen,
32 to 38: Pixel delay 6 parts, 39: Insertion / removal optical filter,
M1 to M6: line memory unit, N0 to N6: negative multiplier, P1: positive multiplier,
50 to 58: adder, 59: video level determiner, 60: subtractor,
61: Compression limiter with small amplitude and large amplitude,

Claims

It has a lens that transmits at least visible light or near-infrared light and an image sensor that is sensitive to at least visible light or near-infrared light, and at least the effective contrast of the image signal of visible light or near-infrared light imaging of the subject is higher than expected. In the case of a drop, the synchronization information transmitted causes the yellow-orange LED, red LED, or near infrared LED that transmits gas molecules in the air to emit light in a time-difference manner with a narrow ON ratio, and the synchronization information transmitted by the imaging device. A monitoring system that outputs a captured image by performing a global electronic shutter in synchronization with
2. The monitoring system according to claim 1, comprising horizontal contour correction generated from seven or more pixel delayed video signals and vertical contour correction generated from seven or more scanning line delays, and at least visible light and near infrared light of a subject. When the effective contrast of the image signal of the image pickup is lower than the expected value, the black level is reduced, the horizontal contour correction enhancement frequency generated from seven or more pixel delay image signals is reduced, and the scanning line delay 7 And increasing the number of scanning lines serving as an emphasis center for vertical contour correction generated from more than two.
3. The monitoring system according to claim 1, wherein the image pickup device is a CCD image pickup device in which a light emission wavelength band pass optical filter of the LED is inserted into an incident optical path. 4.
4. The monitoring system according to claim 1, wherein the synchronization information includes a long-distance linear or column-like monitoring target area such as a railroad track, a road, a transmission line site, or a wind turbine column. It is radiated from the leaky coaxial cable laid in the vicinity or a combination of the ground plane and the transmission line, a twisted pair transmission line, a planar transmission line, a dielectric transmission line or a waveguide, etc. to the vicinity of the monitored area. Radio waves, radio waves radiated in the vicinity of the monitored area from a directional transmission antenna installed in the vicinity of the monitored area, an optical cable laid in the vicinity of the monitored area, GPS and a quasi-zenith satellite A monitoring system that is transmitted by at least one of radio waves including a high-precision position detection function and a high-precision synchronization function.