CN110830729A

CN110830729A - Method and system for improving image sensing performance of image sensor at night and in severe weather

Info

Publication number: CN110830729A
Application number: CN201810953378.1A
Authority: CN
Inventors: 张烂熳
Original assignee: Individual
Current assignee: Individual
Priority date: 2018-08-13
Filing date: 2018-08-13
Publication date: 2020-02-21
Anticipated expiration: 2038-08-13
Also published as: CN110830729B

Abstract

The invention discloses a method and a system for improving image perception performance of an image sensor at night and in severe weather, wherein the system comprises a control circuit, the image sensor, a light-emitting drive circuit and a light-emitting tube; the control circuit generates a field synchronizing signal, and the main exposure signal and the exposure pulse signal are sent to the image sensor; meanwhile, the control circuit generates a light supplementing pulse signal and sends the light supplementing pulse signal to the light emitting driving circuit, the light emitting driving circuit drives the light emitting tube to emit strong light in each light supplementing narrow pulse gap period of the light supplementing pulse signal, and the light emitting tube does not emit light in other residual time; the system collects object reflected light respectively according to different distances and then superposes by controlling the light filling narrow pulse gap of the light filling pulse signal, the exposure gap of the exposure pulse signal and specific delay time between the light filling narrow pulse gap and the exposure gap, so that the shielding of night or rain, fog and snow basically has no influence on the collection of remote object images, and the image sensor improves the image perception performance at night and in severe weather.

Description

Method and system for improving image sensing performance of image sensor at night and in severe weather

The technical field is as follows:

the invention relates to the technical field of image sensors and cameras, in particular to a method and a system for improving image sensing performance of an image sensor at night and in severe weather.

Background art:

when an optical IMAGE generated by an outside scene through a LENS (LENS) is projected onto the surface of an IMAGE SENSOR (IMAGE SENSOR), the IMAGE SENSOR converts the optical signals into electric signals, the electric signals are converted into digital IMAGE signals through an A/D (analog/digital) converter, the digital IMAGE signals are sent into the IMAGE processor for processing, color video signals are generated, and the color video signals are sent out through various interfaces to a display for displaying IMAGEs.

The existing image sensor obtains an image by receiving light rays which are reflected by an external scene and then pass through a lens and converting the light signals into electric signals, and the intensity of the optical signals directly influences the intensity of the electric signals because the optical signals are shot. The perception capability of the image sensor is greatly reduced at night due to the fact that light rays are dark; under the severe weather conditions of rain, fog, snow and the like, the sensing performance of the conventional image sensor is greatly reduced because light reflected by an object and the environment is shielded by the rain, the fog and the snow.

An existing image PROCESSOR (IMAGE SIGNAL PROCESSOR, ISP for short) is mainly responsible for post-processing of digital image signals after being subjected to image sensor and analog-to-digital conversion; the main functions comprise linear correction, noise removal, dead pixel removal, interpolation, white balance, automatic exposure control, automatic gain control and the like; therefore, the image closer to the real world is obtained through the processing of the ISP.

In the prior art, a method of adding an infrared LED lamp panel is generally adopted, and a plurality of infrared lamp tubes irradiate towards the direction of an image collected by a camera, so that light rays in an image collecting area of the camera are enhanced, and the brightness and definition of the image collected by the camera can be improved; however, the LED lamp panel needs to be added to the outside in the mode, the structure is complex, a power supply needs to be added, the whole size of the camera is large, the camera is mostly applied to a commercial cart at present, and the camera cannot be implemented on a small car of a passenger car due to the size, the attractive effect and the like.

The invention patent 201710530046.8 "lens structure and camera including the lens structure" adopts the following method:

according to the lens structure and the camera comprising the lens structure, the heating device is arranged at the position close to or in contact with the lens device, and the imaging device is electrically connected with the heating device, so that the opening button of the heating device is arranged in the imaging device, and the function of automatically eliminating rain, snow, frost and fog adhered to the lens and the heating device of the lens structure is realized. The invention eliminates rain, snow, frost and fog adhered on the lens in a heating mode, so that the lens can work normally.

The invention irradiates the detected environment through the specially controlled luminous source, adds the control circuit and adjusts the working mode of the image sensor, thereby obtaining the improvement of the image sensing performance under night and severe weather conditions.

The invention content is as follows:

the invention aims to provide a method and a system for improving image sensing performance of an image sensor at night and in severe weather.

To achieve the above object, the method and system for improving image sensing performance of an image sensor at night and in bad weather comprises: a control circuit (102 of fig. 1), an image sensor (103 of fig. 1), a light emission drive circuit (101 of fig. 1), a light emitting tube (104 of fig. 1); a field synchronizing signal (208 of fig. 2) generated by a control circuit (102 of fig. 1), a main exposure signal (207 of fig. 2) and an exposure pulse signal (205 of fig. 2) are sent to an image sensor (103 of fig. 1), a supplementary light pulse signal (206 of fig. 2) is generated at the same time and sent to a light-emitting driving circuit (101 of fig. 1), the light-emitting driving circuit (101 of fig. 1) drives a light-emitting tube (104 of fig. 1) to emit strong light during each supplementary light narrow pulse gap (206-1 of fig. 2) of the supplementary light pulse signal (206 of fig. 2), and other residual time is not emitted; the main exposure signal (207 of fig. 2) sent by the control circuit (102 of fig. 1) is composed of main exposure cycles (207-0 of fig. 2), each main exposure cycle (207-0 of fig. 2) is composed of an exposure allowing period (207-1 of fig. 2) and an exposure closing period (207-2 of fig. 2), the two periods are distinguished by different levels of logic level, and different exposure modes and image sensors with different exposure brightness are set to different exposure closing periods (207-2 of fig. 2); the exposure pulse signal sent by the control circuit (102 in fig. 1) is composed of a series of periodic pulse signals, each periodic pulse signal can contain an exposure gap (205-1 in fig. 2) and an exposure closing gap (205-2 in fig. 2), and the two gaps are distinguished by different logic levels; the image sensor (103 of fig. 1) enters a possible exposure period only in an exposure allowable period (207-1 of fig. 2) of the main exposure signal (207 of fig. 2), a specific photosensitive unit of the image sensor (103 of fig. 1) receives external light during each exposure gap (205-1 of fig. 2) of the corresponding exposure pulse signal (205 of fig. 2) within the exposure allowable period, other periods do not receive light, and photoelectrically converted signals collected during each exposure gap (205-1 of fig. 2) are accumulated; each exposure gap (205-1 of fig. 2) has a corresponding fill light narrow pulse gap (206-1 of fig. 2) and is delayed from the corresponding fill light narrow pulse gap (206-1 of fig. 2) by a small specified time (205-3 of fig. 2); the field sync signal (208 of fig. 2) sent by the control circuit (102 of fig. 1) to the image sensor (103 of fig. 1) is correlated with the main exposure signal (207 of fig. 2); the image sensor (103 of fig. 1) outputs an image signal (209 of fig. 2) in synchronization with the field sync signal (208 of fig. 2), and the image signal (209-1 of fig. 2) output during each field interval (208-1 of fig. 2) is a result of exposure of the image sensor (103 of fig. 1) corresponding to this output.

Further, the method and system for improving the image sensing performance of the image sensor at night and in severe weather are characterized in that:

each exposure gap (205-1 of fig. 2) in the exposure pulse signal (205 of fig. 2) within the exposure allowed period (207-1 of fig. 2) of each main exposure signal (207 of fig. 2) is paired with a fill light narrow pulse gap (206-1 of fig. 2) in the fill light pulse signal (206 of fig. 2), temporally displaced by a specific time (205-3 of fig. 2), each exposure gap (205-1 of fig. 2) lags behind the paired fill light narrow pulse gap (206-1 of fig. 2) by a specific time (205-3 of fig. 2), the lagged specific time (205-3 of fig. 2) corresponding to the time when the light source of the fill light narrow pulse gap (206-1 of fig. 2) flies past the object emission of a specific distance range and then returns to the image sensor; in the same main exposure signal (207 in FIG. 2), the exposure amount of a certain distance range is adjusted by setting a pair of a certain number of exposure gaps (205-1 in FIG. 2) and a fill-in narrow pulse gap (206-1 in FIG. 2) corresponding to the light sensitivity of an environmental object in the certain distance range; and different exposure quantities are set according to different distances, so that balanced light sensing under different distances and different weather conditions is realized.

for the Rolling Shutter image sensor (103 in fig. 1), each line of output image of the same frame has a main exposure period (207-0 in fig. 2) corresponding to a complete period, a new main exposure period (207-0 in fig. 2) is started immediately after the end of each main exposure period (207-0 in fig. 2), and the characteristic parameters of each main exposure period (207-0 in fig. 2) corresponding to the same frame image are the same, and the characteristic parameters of the corresponding exposure pulse signal (205 in fig. 2) in the main exposure periods (207-0 in fig. 2) are the same; the main exposure periods (207-0 of fig. 2) corresponding to adjacent different rows may partially overlap, the length of the overlap being generally equal to: the time of one full main exposure period (207-0 of FIG. 2) minus the time of one line period; the characteristic parameters of the main exposure period (207-0 of fig. 2) may be different for different frames.

for a Global Shutter image sensor (103 of FIG. 1), all rows of the same frame correspond to the same full main exposure period (207-0 of FIG. 2).

the method for adjusting the light supplement amount at different distances comprises the following steps: the light emission intensity of the light emitting tube (104 in fig. 1) is adjusted, the time width of the fill light narrow pulse gap (206-1 in fig. 2) is adjusted, and the number of exposure gaps (205-1 in fig. 2) for each corresponding distance is adjusted.

and when the image detection unit at the rear end detects that the visual distance sensed by the image sensor is shorter than the normal distance, starting the supplementary lighting.

Description of the drawings:

FIG. 1 is a system for an image sensor to improve image sensing performance at night and in inclement weather

In the figure:

101 light-emitting drive circuit 102 control circuit

103 image sensor 104 light emitting tube

FIG. 2 is a timing diagram of the signals generated by the control circuit and the output image signals

In the figure:

205 exposure pulse signal 205-1 exposure gap

205-2 exposure close gap 205-3 for a specified time

206 fill light pulse signal 206-1 fill light narrow pulse gap

207 main exposure signal 207-0 main exposure period

207-1 Exposure allowed period 207-2 Exposure off period

208 field sync signal 208-1 field interval

209 output image signal 209-1 image signal

FIG. 3 is an example of a system and method applying the present invention

In the figure:

301 light filling narrow pulse period 302 light filling narrow pulse gap

303 exposure pulse period 304 exposure gap

3051 Exposure gap 3064 with specific time set to 0us and an Exposure gap 3064 with specific time set to 0.1us

3079 the specific time is set to 0.2us exposure gap

30816 specific times are set to 0.3us exposure gap

30925 specific times are set to an exposure gap of 0.4us

310196 specific times were set to an exposure gap of 1.3us

The specific time of T1 is set to 0us, the specific time of T2 is set to 0.1us

T3 specific time is set to 0.2us T4 specific time is set to 0.3us

T5 specific time is set to 0.4us T14 specific time is set to 1.3us

FIG. 4 is an example two of a method and system for applying the present invention

In the figure:

401 light filling narrow pulse period 402 light filling narrow pulse gap

403 exposure pulse period 404 exposure gap

4051 exposure gaps with a specific time of 0us 4064 exposure gaps with a specific time of 0.1us

4079 specific times were set to an exposure gap of 0.2us

40816 specific times were set to 0.3us exposure gap

40925 specific times are set to an exposure gap of 0.4us

410196 specific times were set to an exposure gap of 1.3us

The specific time of T1 is set to 0us, the specific time of T2 is set to 0.1us

T3 specific time is set to 0.2us T4 specific time is set to 0.3us

T5 specific time is set to 0.4us T14 specific time is set to 1.3us

The specific implementation mode is as follows:

the following embodiments are described in detail with reference to the accompanying drawings, and the following embodiments are merely used to more clearly illustrate the technical solutions of the present invention, and therefore are only used as examples, and the protection scope of the present invention is not limited thereby. Various modifications and adaptations can be made in accordance with the embodiments and are within the scope of the present disclosure.

As shown in fig. 3, it is an example of applying the method and system of the present invention, which is specifically as follows: the luminotron adopts an infrared laser diode with the wavelength of 940nm, the control circuit adopts GEO ISP GW5, and the image sensor adopts a specific ASIC; the exposure allowable period of the main exposure signal is set to 2ms, the main exposure period is set to 30ms, each fill light narrow pulse period is set to 2us (301 of fig. 3), the fill light narrow pulse gap is set to 0.1us (302 of fig. 3), each exposure pulse period is also set to 2us (303 of fig. 3), and the exposure gap is also set to 0.1us (304 of fig. 3).

The propagation speed of light waves in air is about 30 kilometres/second, namely 300m/us, and the reflection of light takes the same time, so that the light is emitted for 1us, which is equivalent to the light source flying for 150m distance and reflecting; if the light emission time is 0.1us, which is equivalent to the light source flying for 15m distance and reflecting, then one exposure gap at the same time of light emission is 0.1us, and images within the range of 15m distance are collected, and each exposure gap delayed for several times of the set time of 0.1us will collect images within the range of several times of 15m distance; assuming that an image of the object in the distance range of 210m furthest away is acquired, then the exposure gap is controlled to lag behind the particular time of the paired fill light narrow pulse gap in the following way, and the number of exposure gaps is increased by the power of 2 of the multiple of the corresponding distance in order to ensure a consistent and sufficient luminous flux image:

(1) when the specific time corresponding to the 1 st exposure interval is set to 0us (T1 of fig. 3), an image in the range of 0-15m is acquired; images within 15m within a distance S of 15m 1 are exposed 1²1 time, so long as 1 (305 of fig. 3) exposure gap of the same specific time (T1 of fig. 3);

(2) when the specific time corresponding to the 2 nd exposure interval is set to 0.1us (T2 of fig. 3), an image in the range of 15-30m is acquired; images within 15m 2 of S need to be exposed 2²4 times, so that 4 exposure gaps (306 of fig. 3) of the same specific time (T2 of fig. 3) are required, so the 2 nd to 5 th exposure gaps have the same specific time;

(3) when the specific time corresponding to the 6 th exposure interval is set to 0.2us (T3 of fig. 3), an image in the range of 30-45m is acquired; images within 15m within a distance S of 15m x 3 are exposed 3²This requires 9 (307 of fig. 3) exposure gaps of the same specific time (T3 of fig. 3), so the 6 th to 14 th exposure gaps have the same specific time;

(4)when the specific time corresponding to the 15 th exposure interval is set to 0.3us (T4 of fig. 3), an image in the range of 45-60m is acquired; images within 15m 4 of S need to be exposed 4²This requires 16 (308 of fig. 3) exposure gaps of the same specific time (T4 of fig. 3), so the 15 th to 30 th exposure gaps have the same specific time;

(5) when the specific time corresponding to the 31 st exposure interval is set to 0.4us (T5 of fig. 3), an image in the range of 60-75m is acquired; images within 15m within a distance S of 15m 5 times are exposed 5²This requires 25 (309 of fig. 3) exposure gaps of the same specific time (T5 of fig. 3), so the 31 st to 55 th exposure gaps have the same specific time;

(6) and so on;

(7) when the specific time corresponding to the 820 th exposure gap is set to be 1.3us (T14 in FIG. 3), acquiring an image within the range of 195- & ltSUB & gt 210 m; images within 15m by 14 times of S need to be exposed 14²Since 196 exposure gaps are required for 196 (310 in fig. 3) and the same specific time (T14 in fig. 3), the 820 th to 1000 th exposure gaps have the same specific time.

As shown in fig. 4, the second embodiment of the method and system of the present invention is as follows: the luminotron adopts an infrared laser diode with the wavelength of 940nm, the control circuit adopts GEO ISP GW5, and the image sensor adopts a specific ASIC; the exposure allowable period of the main exposure signal is set to 2ms, the main exposure period is set to 30ms, each fill light narrow pulse period is set to 2us (401 of fig. 4), the fill light narrow pulse gap is set to 0.1us (402 of fig. 4), each exposure pulse period is also set to 2us (403 of fig. 4), but the exposure gap is dynamically adjusted as needed (404 of fig. 4); it is known that if the light emission time is 0.1us, which corresponds to the light source flying a distance of 15m and reflecting, when the exposure gap time is several times the light emission time of 0.1us, the light source flies a distance of several times 15m and reflects; assuming that an image of the object in the distance range of 210m furthest away is acquired, then the exposure gap is controlled to lag behind the particular time of the paired fill light narrow pulse gap in the following way, and the number of exposure gaps is increased by the power of 2 of the multiple of the corresponding distance in order to ensure a consistent and sufficient luminous flux image:

(1) when the specific time corresponding to the 1 st exposure interval is set to 0us (T1 of fig. 4), an image in which the full distance range includes the range of 0-15m is acquired; images within a distance S of 15m by 1 are exposed 1²1 time, so long as 1 (405 of fig. 4) exposure gap of the same specific time (T1 of fig. 4);

(2) when the specific time corresponding to the 2 nd exposure interval is set to 0.1us (T2 in fig. 4), an image within a range of 15-30m is acquired, and an image within a range of 15m within a distance S of 15m × 2 needs to be exposed for 2²Since the previous 1 st exposure interval has acquired 1 image, 4-1 is 3, which requires 3 exposure intervals (406 of fig. 4) of the same specific time (T2 of fig. 4), the 2 nd to 4 th exposure intervals have the same specific time;

(3) when the specific time corresponding to the 5 th exposure interval is set to 0.2us (T3 of fig. 4), an image in the range of 30-45m is acquired; images within 15m within a distance S of 15m x 3 are exposed 3²Since the previous 4 exposure gaps have acquired 4 images, 9-4-5, thus requiring 5 (407 of fig. 4) exposure gaps of the same specific time (T3 of fig. 4), the 5 th to 9 th exposure gaps have the same specific time;

(4) when the specific time corresponding to the 10 th exposure interval is set to 0.3us (T4 of fig. 4), an image in the range of 45-60m is acquired; images within 15m 4 of S need to be exposed 4²Since the previous 9 exposure gaps have acquired 9 images, 16-9 is 7, thus requiring 7 (408 of fig. 4) exposure gaps of the same specific time (T4 of fig. 4), the 10 th to 16 th exposure gaps have the same specific time;

(5) when the specific time corresponding to the 17 th exposure interval is set to 0.4us (T5 of fig. 4), an image in the range of 60-75m is acquired; images within 15m within a distance S of 15m 5 times are exposed 5²25 times, since the first 16The exposure interval has acquired 16 images, 25-16 being 9, thus requiring 9 (409 of fig. 4) exposure intervals of the same specific time (T5 of fig. 4), so the 17 th to 25 th exposure intervals have the same specific time;

(6) and so on;

(7) when the specific time corresponding to the 170 th exposure gap is set to be 1.3us (T14 in FIG. 4), acquiring images within the range of 195- & ltSUB & gt 210 m; images within 15m by 14 times of S need to be exposed 14²196, since the 169 images have been acquired for the previous 169 exposure gaps, 196 — 169 — 27, which requires 27 (410 of fig. 4) exposure gaps of the same specific time (T14 of fig. 4), the 170 th to 195 th exposure gaps have the same specific time.

The system controls the light filling narrow pulse gap of the light filling pulse signal, the exposure gap of the exposure pulse signal and the specific delay time between the light filling narrow pulse gap and the exposure gap according to the above modes, and the reflected light of the object is respectively collected according to different distances and then superposed, so that the shielding of night or rain, fog and snow basically has no influence on the collection of the image of the object at a long distance, and the image sensor is realized to improve the image perception performance at night and in severe weather.

Claims

1. A method and system for an image sensor to improve image sensing performance at night and in inclement weather, the method and system comprising: a control circuit (102), an image sensor (103), a light emission drive circuit (101), and a light emitting tube (104); a field synchronization signal (208) generated by a control circuit (102), a main exposure signal (207) and an exposure pulse signal (205) are sent to an image sensor (103), a light supplementing pulse signal (206) is generated at the same time and sent to a light emitting driving circuit (101), the light emitting driving circuit (101) drives a light emitting tube (104) to emit strong light during each light supplementing narrow pulse gap (206-1) of the light supplementing pulse signal (206), and other residual time is not light emitting; the main exposure signal (207) sent by the control circuit (102) is composed of main exposure cycles (207-0) one by one, each main exposure cycle (207-0) is composed of an exposure allowing time period (207-1) and an exposure closing time period (207-2), the two time periods are distinguished through different levels of logic levels, and different exposure modes and image sensors with different exposure brightness are provided with different exposure closing time periods (207-2); the exposure pulse signal sent by the control circuit (102) is composed of a series of periodic pulse signals, each periodic pulse signal can contain an exposure gap (205-1) and an exposure closing gap (205-2), and the two gaps are distinguished by different logic levels; the image sensor (103) enters a possible exposure period only in an exposure allowable period (207-1) of the main exposure signal (207), receives external light during each exposure gap (205-1) of the corresponding exposure pulse signal (205) within the exposure allowable period, does not receive light for other periods of time, and accumulates the photoelectrically converted signals collected during each exposure gap (205-1); each exposure gap (205-1) has a corresponding fill light narrow pulse gap (206-1) and is delayed by a short specific time (205-3) from the corresponding fill light narrow pulse gap (206-1); the field synchronization signal (208) sent by the control circuit (102) to the image sensor (103) is correlated with the main exposure signal (207); the image sensor (103) outputs an image signal (209) in synchronization with the field synchronization signal (208), and the image signal (209-1) output during each field interval (208-1) is a result of the image sensor (103) performing exposure corresponding to this output.

2. The method and system for image sensor to improve image sensing performance at night and in bad weather as claimed in claim 1, wherein:

each exposure gap (205-1) in the exposure pulse signal (205) within the exposure allowable period (207-1) of each main exposure signal (207) is paired with a fill light narrow pulse gap (206-1) in the fill light pulse signal (206), and is displaced in time by a specific time (205-3), each exposure gap (205-1) lags behind the paired fill light narrow pulse gap (206-1) by the specific time (205-3), and the specific time (205-3) lags corresponds to the time when the light source of the fill light narrow pulse gap (206-1) flies through the object within a specific distance range and then returns to the image sensor for reception; in the same main exposure signal (207), corresponding to the light sensing of an environmental object in a specific distance range, the exposure amount in the specific distance range is adjusted by setting the pairing of a specific number of exposure gaps (205-1) and a light supplementing narrow pulse gap (206-1); and different exposure quantities are set according to different distances, so that balanced light sensing under different distances and different weather conditions is realized.

3. The method and system for image sensor to improve image sensing performance at night and in bad weather as claimed in claim 1, wherein:

for an image sensor (103) of a Rolling Shutter (Rolling Shutter), each line of output images of the same frame has a main exposure period (207-0) corresponding to a complete period, a new main exposure period (207-0) is started immediately after the end of each main exposure period (207-0), and the characteristic parameters of each main exposure period (207-0) corresponding to the image of the same frame are the same, and the characteristic parameters of corresponding exposure pulse signals (205) in the main exposure periods (207-0) are also the same; the main exposure periods (207-0) corresponding to adjacent different rows may partially overlap, the length of the overlap being generally equal to: the time of one complete main exposure period (207-0) minus the time of one line period; the characteristic parameters of the main exposure period (207-0) may be different for different frames.

4. The method and system for image sensor to improve image sensing performance at night and in bad weather as claimed in claim 1, wherein:

for a Global Shutter image sensor (103), all rows of the same frame correspond to the same full main exposure period (207-0).

5. The method and system for image sensor to improve image sensing performance at night and in bad weather as claimed in claim 1, wherein:

the method for adjusting the light supplement amount at different distances comprises the following steps: the luminous intensity of the luminous tube (104) is adjusted, the time width of the light filling narrow pulse gap (206-1) is adjusted, and the number of the exposure gaps (205-1) of each corresponding distance is adjusted.

6. The method and system for image sensor to improve image sensing performance at night and in bad weather as claimed in claim 1, wherein: