CN110493496B

CN110493496B - Image acquisition device and method

Info

Publication number: CN110493496B
Application number: CN201910517680.7A
Authority: CN
Inventors: 聂鑫鑫; 范蒙
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2019-06-14
Filing date: 2019-06-14
Publication date: 2021-11-09
Anticipated expiration: 2039-06-14
Also published as: CN110493496A

Abstract

The application discloses image acquisition device and method, and the device includes: the image sensor, light filling ware and filtering component, the light filling ware includes first light filling device, and filtering component includes first light filter. The first light supplement device at least has near-infrared light supplement in a part of exposure time period of the first preset exposure, and does not have near-infrared light supplement in the exposure time period of the second preset exposure, and the first optical filter enables visible light and part of near-infrared light to pass through. The image sensor performs multiple exposures in a global exposure starting mode, generates and outputs a first image signal in an exposure time period of a first preset exposure, and generates and outputs a second image signal in an exposure time period of a second preset exposure. This embodiment carries out multiple exposure through image sensor for this image acquisition device is more nimble.

Description

Image acquisition device and method

Technical Field

The embodiment of the application relates to the field of computer vision, in particular to an image acquisition device and method.

Background

With the rapid development of scientific technology, safety protection products have been applied to various fields, for example, government departments, large-sized enterprises, communities, and households. Specifically, the monitoring system belongs to an important component of a safety protection product, and image acquisition can provide a source for subsequent data analysis so as to meet different requirements of various application scenes.

Currently, an image acquisition system generally comprises: the device comprises a lens, an optical filter, a light supplementing device and an image sensor, wherein light is filtered by the optical filter switching device after passing through the lens, images are formed on the image sensor, and an image signal is generated and output by the image sensor; when the light supplement lamp is turned off, the optical filter is switched to be an infrared optical filter and used for collecting visible light images.

However, the images acquired through the optical filter are switched between the near-infrared supplementary lighting image and the visible light image, so that the two images cannot be acquired simultaneously within a period of time, and the flexibility of image acquisition is reduced.

Disclosure of Invention

The embodiment of the application provides an image acquisition device and method, which are used for overcoming the problem that a near-infrared supplementary lighting image and a visible light image cannot be acquired simultaneously within a period of time.

In a first aspect, an embodiment of the present application provides an image capturing apparatus, including: the device comprises an image sensor, a light supplementing device and a light filtering component, wherein the image sensor is positioned on the light emergent side of the light filtering component;

the image sensor is configured to generate and output at least one frame of first image signal and at least one frame of second image signal through multiple exposures, where the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure and the second preset exposure are two exposures of the multiple exposures, the image sensor is configured to perform multiple exposures in a global exposure starting manner, within an exposure time period of the first preset exposure, exposure starting times of all rows of effective images of the first image signal are the same, exposure ending times of at least two rows of effective images of the first image signal are different, within an exposure time period of the second preset exposure, exposure starting times of all rows of effective images of the second image signal are the same, the exposure end time of at least two rows of effective images of the second image signal is different;

the light supplement device comprises a first light supplement device, and the first light supplement device is used for performing near-infrared light supplement, wherein near-infrared light supplement exists at least in a part of the exposure time period of the first preset exposure, and near-infrared light supplement does not exist in the whole exposure time period of the second preset exposure;

the filter assembly comprises a first filter, and the first filter allows visible light and part of near infrared light to pass through.

Optionally, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than the exposure ending time of all the effective images in all the lines of the second image signal of the latest frame before the current frame, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is earlier than the exposure starting time of any effective image in any line of the second image signal of the latest frame after the current frame.

Optionally, the exposure end times of all the lines of effective images of the first image signal are different;

the exposure end times of all the lines of the effective image of the second image signal are different.

Optionally, in each frame of image signals corresponding to the image sensor, a difference between exposure end times of adjacent rows of effective images is less than 40 microseconds.

Optionally, a starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than an exposure ending time of a last line of effective images of a last frame of second image signal before the current frame, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is earlier than or equal to the exposure starting time of any line of effective images of a last frame of second image signal after the current frame.

Optionally, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than or equal to the exposure ending time of the last effective image line of the latest second image signal before the current frame, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is earlier than or equal to the exposure starting time of any effective image line of the latest second image signal after the current frame and is earlier than or equal to the exposure ending time of the first effective image line of the current frame first image signal.

Optionally, the first preset exposure comprises two continuous exposures to generate two continuous frames of first image signals, and the second preset exposure comprises one exposure to generate one frame of second image signals;

the starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than or equal to the exposure ending time of the last line of effective image of the latest frame of second image signal before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is earlier than or equal to the exposure starting time of any line of effective image of the latest frame of second image signal after the current first preset exposure.

Optionally, the first preset exposure includes two consecutive exposures to generate two consecutive frames of the first image signal, and the second preset exposure includes one exposure to generate one frame of the second image signal;

the starting time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures is later than or equal to the exposure ending time of the last line of effective image of the last frame of second image signal before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal of the current first preset exposure; and is

The starting time of the near-infrared supplementary lighting of the second exposure of the first supplementary lighting device in the two continuous exposures is later than or equal to the exposure ending time of the last line of effective image of the latest frame of second image signal before the current first preset exposure and later than the exposure ending time of the last line of effective image of the latest frame of first image signal before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the second exposure of the first supplementary lighting device in the two continuous exposures is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal before the current first preset exposure.

Optionally, when the first light supplement device performs near-infrared light supplement, the intensity of the near-infrared light passing through the first optical filter is higher than the intensity of the near-infrared light passing through the first optical filter when the first light supplement device does not perform near-infrared light supplement.

Optionally, the filter assembly further includes a second filter and a switching component, and both the first filter and the second filter are connected to the switching component;

the switching component is used for switching the second optical filter to the light incidence side of the image sensor;

after the second optical filter is switched to the light incident side of the image sensor, the second optical filter allows light in a visible light waveband to pass through and blocks light in a near-infrared light waveband, and the image sensor is used for generating and outputting a third image signal through exposure.

Optionally, the wavelength band range of the near-infrared light incident to the first optical filter is a first reference wavelength band range, and the first reference wavelength band range is 650 nm to 1100 nm.

Optionally, when the central wavelength of the near-infrared light supplement by the first light supplement device is a set characteristic wavelength or falls within a set characteristic wavelength range, the central wavelength and/or the band width of the near-infrared light passing through the first optical filter reach a constraint condition.

Optionally, the center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device is any wavelength within a wavelength range of 750 ± 10 nanometers; or

The center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device is any wavelength within the wavelength range of 780 +/-10 nanometers; or

The center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device is any wavelength within a wavelength range of 940 +/-10 nanometers; or

The central wavelength of the first light supplement device (021) for near-infrared light supplement is 810 +/-10 nanometers.

Optionally, the constraint condition includes:

the half bandwidth of the near infrared light passing through the first optical filter is less than or equal to 50 nanometers; or

The first wave band width is smaller than the second wave band width; the first wave band width refers to the wave band width of near infrared light passing through the first optical filter, and the second wave band width refers to the wave band width of the near infrared light blocked by the first optical filter; or

The third wave band width is smaller than the reference wave band width, the third wave band width refers to the wave band width of the near infrared light with the passing rate larger than the set proportion, and the reference wave band width refers to any wave band width in the wave band range of 50 nanometers to 150 nanometers.

Alternatively, the set ratio is any ratio within a ratio range of 30% to 50%.

Optionally, at least one exposure parameter of the first preset exposure and the second preset exposure is different, the at least one exposure parameter is one or more of exposure time, exposure gain, and aperture size, and the exposure gain includes analog gain, and/or digital gain.

Optionally, the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure.

Optionally, the image sensor comprises a plurality of light sensing channels, each light sensing channel being configured to sense light in at least one visible wavelength band and to sense light in a near infrared wavelength band.

Optionally, the plurality of light sensing channels are for sensing light in at least two different visible light bands.

Optionally, the image sensor comprises a plurality of photosensitive channels, wherein the plurality of photosensitive channels comprise at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, a Y photosensitive channel, a W photosensitive channel and a C photosensitive channel;

the light sensing device comprises a light sensing channel, a light sensing channel and a light sensing channel, wherein the light sensing channel is used for sensing light of a red light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a green light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a blue light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a yellow light wave band and a near infrared wave band, the light sensing channel is used for sensing light of a full wave band, and the light sensing channel is used for sensing light of the full wave band.

Optionally, the image sensor is a red, green, blue, RGB, white, RGBW sensor, or a red, white, blue, RCCB sensor, or a red, yellow, blue, RYYB sensor.

Optionally, the light supplement device further includes a second light supplement device, and the second light supplement device is configured to supplement visible light.

Optionally, the second light supplement device is configured to supplement visible light in a normally bright manner; alternatively, the first and second electrodes may be,

the second light supplement device is used for supplementing visible light in a stroboscopic mode, wherein the supplementary visible light exists at least in part of the exposure time period of the first preset exposure, and the supplementary visible light does not exist in the whole exposure time period of the second preset exposure; or

The second light supplement device is used for supplementing visible light in a stroboscopic mode, wherein the visible light supplement does not exist at least in the whole exposure time period of the first preset exposure, and the visible light supplement exists in a part of the exposure time period of the second preset exposure.

In a second aspect, an image capturing method is provided, which is applied to an image capturing device, where the image capturing device includes an image sensor, a light supplement device, and a light filtering component, the light supplement device includes a first light supplement device, the light filtering component includes a first light filter, the image sensor is located on a light exit side of the light filtering component, and the method includes:

performing near-infrared supplementary lighting through a first supplementary lighting device, wherein the stroboscopic mode means that near-infrared supplementary lighting exists at least in a part of exposure time period of a first preset exposure, and near-infrared supplementary lighting does not exist in an exposure time period of a second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures;

passing visible light and a portion of near-infrared light through the first filter;

performing multiple exposures through the image sensor to generate and output at least one frame of a first image signal and at least one frame of a second image signal, the first image signal is an image signal generated according to the first preset exposure, the second image signal is an image signal generated according to the second preset exposure, the image sensor is used for carrying out multiple exposures by adopting a global exposure starting mode, within the exposure time period of the first preset exposure, the exposure start time of all the lines of effective images of the first image signal is the same, the exposure end time of at least two lines of effective images of the first image signal is different, in the exposure time period of the second preset exposure, the exposure start times of all the rows of effective images of the second image signal are the same, and the exposure end times of at least two rows of effective images of the second image signal are different.

Optionally, the filter assembly further comprises a second filter and a switching member, and the method further comprises:

switching the second optical filter to the light incident side of the image sensor through the switching part;

after the second optical filter is switched to the light incidence side of the image sensor, light in a visible light waveband is enabled to pass through the second optical filter, and light in a near infrared light waveband is blocked;

after the second filter passes light in a visible light band and blocks light in a near infrared light band, exposure is performed by the image sensor to generate and output a third image signal.

Optionally, the light supplement device further includes a second light supplement device, and before the first optical filter included in the optical filter assembly passes light in the visible light band and light in the near-infrared light band, the second light supplement device further includes:

and performing visible light supplementary lighting through the second supplementary lighting device.

The application provides an image acquisition device and a method, multiple exposure is carried out according to first preset exposure and second preset exposure by setting an image sensor, so that near-infrared light supplement is carried out and a first image signal is generated in the first preset exposure process, near-infrared light supplement is not carried out and a second image signal is generated in the second preset exposure process, the data acquisition mode is simple in structure, the cost is reduced, a first image signal and a second image signal which are different in brightness information can be directly acquired, namely two different image signals which can be acquired simultaneously within a period of time, the image acquisition device is more flexible, and the acquisition of the first image signal and the second image signal is more efficient. And the ending time and the starting time of the near-infrared supplementary lighting of the first supplementary lighting device are set according to the exposure ending time and the exposure starting time of the effective images of the first image signal and the second image signal, so that the color of the second image signal output in a time period corresponding to the second preset exposure without the near-infrared supplementary lighting is accurate.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1A is a schematic structural diagram of a first image acquisition device provided in an embodiment of the present application;

fig. 1B is a schematic diagram of an image capturing device according to an embodiment of the present disclosure generating a first image signal;

fig. 1C is a schematic diagram of an image capturing device for generating a second image signal according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram illustrating a relationship between a wavelength and a relative intensity of a first light supplement device for performing near-infrared light supplement according to an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of a second image acquisition device provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an RGB sensor provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of an RGBW sensor provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of an RCCB sensor according to an embodiment of the present application; .

FIG. 7 is a schematic diagram of a RYYB sensor provided in an embodiment of the present application;

fig. 8 is a schematic diagram of an induction curve of an image sensor according to an embodiment of the present application;

fig. 9 is a schematic diagram illustrating a relationship between a wavelength and a transmittance of light passing through a first optical filter according to an embodiment of the present disclosure;

FIG. 10 is a schematic diagram of a global start exposure method according to an embodiment of the present application;

fig. 11 is a schematic view of a first light supplement device according to an embodiment of the present disclosure;

fig. 12 is a schematic view of a light supplement time period of a second first light supplement device according to an embodiment of the present disclosure;

fig. 13 is a schematic view of a light supplement time period of a third first light supplement device according to an embodiment of the present disclosure;

fig. 14 is a schematic view of a light supplement time period of a fourth light supplement device according to an embodiment of the present disclosure;

fig. 15 is a flowchart of an image capturing method according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1A is a schematic structural diagram of an image capturing device provided in an embodiment of the present application, and referring to fig. 1A, the image capturing device includes an image sensor 01, a light supplement device 02, and a light filtering component 03, where the image sensor 01 is located on a light exit side of the light filtering component 03.

The image sensor 01 is configured to generate and output at least one frame of a first image signal and at least one frame of a second image signal through multiple exposures. The first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure and the second preset exposure are two exposures of the multiple exposures, the image sensor is used for performing multiple exposures in a global exposure starting mode, the exposure starting time of all rows of effective images of the first image signal is the same in the exposure time period of the first preset exposure, the exposure ending time of at least two rows of effective images of the first image signal is different, the exposure starting time of all rows of effective images of the second image signal is the same in the exposure time period of the second preset exposure, and the exposure ending time of at least two rows of effective images of the second image signal is different. The light supplement device 02 includes a first light supplement device 021, and the first light supplement device 021 is configured to perform near-infrared light supplement, where at least a part of the exposure time period of the first preset exposure has near-infrared light supplement, and no part of the exposure time period of the second preset exposure has near-infrared light supplement. The optical filter assembly 03 includes a first optical filter 031, and the first optical filter 031 passes visible light and part of near-infrared light, wherein the intensity of the near-infrared light passing through the first optical filter 031 when the first light supplement device 021 performs near-infrared light supplement is higher than the intensity of the near-infrared light passing through the first optical filter 031 when the first light supplement device 021 does not perform near-infrared light supplement.

In the embodiment of the present application, referring to fig. 1A to 1C, the image capturing device may further include a lens 04, in this case, the filter assembly 03 may be located between the lens 04 and the image sensor 01, and the image sensor 01 is located on the light emitting side of the filter assembly 03. Alternatively, the lens 04 is located between the filter assembly 03 and the image sensor 01, and the image sensor 01 is located on the light emitting side of the lens 04.

As an example, the first filter 031 may be a filter film, such that the first filter 031 may be attached to a surface of the light-emitting side of the lens 04 when the filter assembly 03 is positioned between the lens 04 and the image sensor 01, or attached to a surface of the light-entering side of the lens 04 when the lens 04 is positioned between the filter assembly 03 and the image sensor 01.

It should be noted that the light supplement device 02 may be located inside the image capturing device or outside the image capturing device. The light supplement 02 may be a part of the image capturing device or may be a device independent of the image capturing device. When the light supplement 02 is located outside the image collection, the light supplement 02 may be in communication connection with an image collection device, and as an example, the image collection device may be an image collection device, such as a video camera, a snapshot machine, a face recognition camera, a code reading camera, a vehicle-mounted camera, a panoramic detail camera, and the like. Therefore, it can be ensured that the exposure time sequence of the image sensor 01 in the image acquisition device has a certain relationship with the near-infrared light supplement time sequence of the first light supplement device 021 included in the light supplement device 02, if at least a part of the exposure time period of the first preset exposure has the near-infrared light supplement, and the exposure time period of the second preset exposure does not have the near-infrared light supplement.

In addition, the first light supplement device 021 is a device capable of emitting near-infrared light, such as a near-infrared light supplement lamp, and the first light supplement device 021 can perform near-infrared light supplement in a stroboscopic manner, and also can perform near-infrared light supplement in other manners similar to stroboscopic manner, and this embodiment of the present application is not limited thereto.

In some examples, when the first light supplement device 021 performs near-infrared light supplement in a stroboscopic manner, the first light supplement device 021 may be controlled in a manual manner to perform near-infrared light supplement in the stroboscopic manner, or the first light supplement device 021 may be controlled in a software program or a specific device to perform near-infrared light supplement in the stroboscopic manner, which is not limited in this embodiment.

The time period of the near-infrared light supplement performed by the first light supplement device 021 may coincide with the exposure time period of the first preset exposure, or may be greater than the exposure time period of the first preset exposure or smaller than the exposure time period of the first preset exposure, as long as there is near-infrared light supplement in the entire exposure time period or a part of the exposure time period of the first preset exposure, and there is no near-infrared light supplement in the exposure time period of the second preset exposure.

It should be noted that there is no near-infrared fill light in the exposure time period of the second preset exposure, and for the global exposure mode, the exposure time period of the second preset exposure may be a time period between the start exposure time and the end exposure time, but is not limited thereto. For example, the exposure time period of the second preset exposure may also be an exposure time period corresponding to a target image in the second image signal, the target image is a plurality of lines of effective images corresponding to a target object or a target area in the second image signal, and a time period between the starting exposure time and the ending exposure time of the plurality of lines of effective images may be regarded as the exposure time period of the second preset exposure.

Specifically, the image sensor is actually a chip assembly including a pixel array, wherein the pixel array is composed of pixels, and corresponds to each pixel in an image finally presented, and the pixel array is specifically used for photoelectric conversion (converting an optical signal into an electrical signal), and when the pixel array completes one exposure, a frame of image signal is obtained.

As an example, the pixel array includes a plurality of rows, where each row corresponds to a respective exposure start time and an exposure end time, specifically, the exposure time period generally refers to an effective pixel exposure time period, and an effective pixel refers to a pixel that an image signal is processed and then output.

In this embodiment, the image sensor is configured to perform multiple exposures by using a global start exposure method, where exposure end times of all row effective images of the first image signal are different, and exposure end times of all row effective images of the second image signal are different, where a difference between exposure end times of adjacent row effective images is less than 40 microseconds, and those skilled in the art can understand that 40 microseconds is an empirical value obtained according to actual operation, which can be replaced according to actual requirements, where this is not an absolute limitation.

Illustratively, referring to fig. 10, fig. 10 is a schematic diagram of a global start exposure mode. Specifically, the exposure start time of all the active lines in each frame of image in the global start exposure is the same, and the exposure end time of all the active lines in each frame of image may be different.

In a possible implementation manner, the exposure end times of all the lines in each frame of image are different, or in another possible implementation manner, a preset number of effective lines may be used as a group (for example, adjacent 2 effective lines become a group), where the exposure end times of the effective lines in each group are different, and the exposure end times of the effective lines in the same group are the same, where how to group the effective lines may be set according to actual requirements, for example, a group of 3 adjacent effective lines may also be set, or a group of 4 effective lines with equal intervals is set, and the present embodiment does not limit this.

The global start exposure is described below by taking the example that the exposure end times of all the effective lines are different, and as can be seen from fig. 10, the effective images of all the lines start exposure at time T1, while the effective image of line 1 ends exposure at time T2, the effective image of line 2 ends exposure at time T3, time T3 is shifted backward by one period of time compared with time T2, and time T4 is shifted backward by one period of time compared with time T3.

When the exposure of the 1 st line effective image is finished and the data output is started at the time T2, the data output is finished at the time T4, the exposure of the nth line effective image is finished and the data output is started at the time T5, and the data output is finished at the time T6, the time between the times T2 and T6 is the readout time.

In this embodiment, the image sensor 01 performs multiple exposures in a global exposure manner, the exposure time periods of the first preset exposure and the second preset exposure are alternately and cyclically arranged, for any near-infrared supplementary lighting, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device 021 is later than the exposure ending time of the last effective image line of the last second image signal of the last frame before the current frame, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device 021 is earlier than the exposure starting time of the first effective image line of the last second image signal of the last frame after the current frame. Like this, can realize having near-infrared light filling in the partial exposure time quantum of first preset exposure at least, do not have near-infrared light filling in the whole exposure time quantum of second preset exposure, thereby alleviateed the problem of light pollution, and through the opening moment and the ending moment that set up near-infrared light filling lamp, thereby guarantee that the color of the second image signal that produces at the exposure time quantum of second preset exposure that does not carry out near-infrared light filling is accurate, can not influenced by infrared light filling.

For example, referring to fig. 11, where 1-N respectively represent N effective lines of images, and N-1, N +1, N, and N +2 respectively represent different frames, as can be seen from fig. 10 and 11, fig. 11 exemplarily shows an image signal in which 5 frames are exposed by using a global start exposure method, where the exposure start times of the N effective lines of each frame are the same, and the exposure end times are different, taking an N-2 th frame as an example, where a portion indicated by light gray shading is a portion from the exposure start of each effective line to the exposure end of a 1 st line, and a portion indicated by dark gray shading is a portion from the exposure end of each effective line, where the exposure start times of the N effective lines of each frame are the same, so that the fast moving object is shifted more lightly.

An implementation manner of the near-infrared supplementary lighting at any time is described below with reference to fig. 11 to 14, where a high level indicates turning on of the first supplementary lighting device, and a low level indicates turning off of the first supplementary lighting device, specifically, when the first supplementary lighting device is turned on, a first image signal is correspondingly generated corresponding to an exposure time period of a first preset exposure, and when the first supplementary lighting device is turned off, a second image signal is correspondingly generated corresponding to an exposure time period of a second preset exposure.

Referring to fig. 11, in an alternative implementation manner, it is assumed that a current frame first image signal corresponds to an N-1 th frame, a starting time of near-infrared supplementary lighting of a first supplementary lighting device 021 is later than or equal to an exposure ending time of a last effective image line of a last frame (N-2 frames) of a second image signal before the current frame, and an ending time of the near-infrared supplementary lighting of the first supplementary lighting device 021 is earlier than or equal to an exposure starting time of any effective image line of the last frame (N frames) of the second image signal after the current frame, in an alternative embodiment, the ending time of the near-infrared supplementary lighting of the first supplementary lighting device 021 may be later than the exposure ending time of the last effective image line of the first image signal of the current frame (N-1 frames).

Referring to fig. 12, in another alternative implementation, for any near-infrared supplementary lighting, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device 021 is later than or equal to the exposure ending time of the last effective image line of the second image signal of the latest frame (N-2 frames) before the current frame, the ending time of the near-infrared supplementary lighting of the first supplementary lighting device 021 is earlier than or equal to the exposure starting time of any effective image line of the second image signal of the latest frame (N frames) after the current frame and is earlier than or equal to the exposure ending time of the first effective image line of the first image signal of the current frame (N-1 frames), and the supplementary lighting times of all the effective lines are the same, thereby ensuring uniform supplementary lighting of the image signals.

Referring to fig. 13, in yet another possible implementation, the first preset exposure includes two consecutive exposures to generate two consecutive frames of the first image signal, and the second preset exposure includes one exposure to generate one frame of the second image signal.

The starting time of the near-infrared light supplement of the first light supplement device 021 is later than or equal to the exposure ending time of the last line of effective image of the second image signal of the last frame (N-3 frames) before the current first preset exposure, the ending time of the near-infrared light supplement of the first light supplement device 021 is earlier than or equal to the exposure starting time of any line of effective image of the second image signal of the last frame (N frames) after the current first preset exposure, in an optional implementation manner, the ending time of the near-infrared light supplement of the first light supplement device 021 can be later than or equal to the exposure ending time of the last line of effective image of the first image signal of the frame (N-1 frames) where the current first preset exposure is located, or the ending time of the near-infrared light supplement of the first light supplement device 021 can also be not later than the exposure ending time of the last line of effective image of the first image signal of the frame (N-1 frames) where the current first preset exposure is located, this embodiment is not limited thereto.

Referring to fig. 14, in yet another possible implementation, the first preset exposure includes two consecutive exposures to generate two consecutive frames of the first image signal, and the second preset exposure includes one exposure to generate one frame of the second image signal;

the starting time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device 021 is later than or equal to the exposure ending time of the last row of effective images of the second image signal of the latest frame (N-3 frames) before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device 021 is earlier than or equal to the exposure ending time of the first row of effective images of the first image signal of the current first preset exposure (N-2 frames).

The starting time of the near-infrared supplementary lighting of the second exposure of the first supplementary lighting device 021 in the two continuous exposures is later than or equal to the exposure ending time of the last row of effective images of the second image signal of the last frame (N-3 frames) before the current first preset exposure and later than the exposure ending time of the last row of effective images of the first image signal of the last frame (N-2 frames) before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the second exposure of the first supplementary lighting device 021 in the two continuous exposures is earlier than or equal to the exposure ending time of the first row of effective images of the first image signal of the frame (N-1 frames) where the current first preset exposure is located.

Fig. 11 to 14 are only examples, and implementations of the start time and the end time of the near-infrared supplementary lighting of the first supplementary lighting device 021 may not be limited to these examples.

The foregoing only provides four possible implementation manners of the starting time and the closing time of the near-infrared light supplement of the first light supplement device 021, and in practical application, the implementation manner is not limited to the four possible implementation manners, and this is not limited in this embodiment of the application.

Another point to be described is that, when the first light supplement device 021 performs near-infrared light supplement on an external scene, near-infrared light incident on the surface of an object may be reflected by the object, and thus enters the first optical filter 031. In addition, since the ambient light may include visible light and near infrared light in a normal condition, and the near infrared light in the ambient light is also reflected by the object when being incident on the surface of the object, so as to enter the first filter 031.

Therefore, the near-infrared light passing through the first optical filter 031 when the near-infrared light supplement exists may include near-infrared light entering the first optical filter 031 by being reflected by an object when the first light supplement device 021 performs near-infrared light supplement, and the near-infrared light passing through the first optical filter 031 when the near-infrared light supplement does not exist may include near-infrared light entering the first optical filter 031 by being reflected by the object when the first light supplement device 021 does not perform near-infrared light supplement. That is, the near-infrared light passing through the first optical filter 031 when there is near-infrared supplementary light includes near-infrared light emitted by the first supplementary light device 021 and reflected by the object and near-infrared light in the ambient light reflected by the object, and the near-infrared light passing through the first optical filter 031 when there is no near-infrared supplementary light includes near-infrared light in the ambient light reflected by the object.

Taking the image capturing device, the filter assembly 03 may be located between the lens 04 and the image sensor 01, and the image sensor 01 is located on the light-emitting side of the filter assembly 03, as an example, the process of capturing the first image signal and the second image signal by the image capturing device is as follows: referring to fig. 1B, when the image sensor 01 performs the first preset exposure, the first light supplement device 021 has near-infrared light supplement, and when the ambient light in the shooting scene and the near-infrared light reflected by an object in the scene when the first light supplement device performs the near-infrared light supplement pass through the lens 04 and the first optical filter 031, the image sensor 01 generates a first image signal through the first preset exposure; referring to fig. 1C, when the image sensor 01 performs the second preset exposure, the first light supplement device 021 does not have near-infrared light supplement, at this time, after the ambient light in the shooting scene passes through the lens 04 and the first optical filter 031, the image sensor 01 generates the second image signal through the second preset exposure, M first preset exposures and N second preset exposures may be provided within one frame period of image acquisition, there may be a sequence of multiple combinations between the first preset exposures and the second preset exposures, and in one frame period of image acquisition, values of M and N and a size relationship of M and N may be set according to actual requirements, for example, values of M and N may be equal or different.

It should be noted that the first filter 031 may allow a portion of the near-infrared light to pass through, and the near-infrared light passing through the first filter 031 may be a portion of the near-infrared light.

In addition, since the intensity of the near-infrared light in the ambient light is lower than the intensity of the near-infrared light emitted by the first light supplement device 021, the intensity of the near-infrared light passing through the first optical filter 031 when the first light supplement device 021 performs the near-infrared light supplement is higher than the intensity of the near-infrared light passing through the first optical filter 031 when the first light supplement device 021 does not perform the near-infrared light supplement.

The first light supplement device 021 can perform near-infrared light supplement within a second reference waveband range, and the second reference waveband range can be 700-800 nm or 900-1000 nm, so that the interference caused by a common near-red light of 850 nm can be reduced, and the color accuracy is ensured. In addition, the wavelength range of the near-infrared light incident to the first optical filter 031 may be a first reference wavelength range, which is 650 nm to 1100 nm.

When the near-infrared light compensation exists, the near-infrared light passing through the first optical filter 031 may include near-infrared light reflected by the object and entering the first optical filter 031 when the first light compensation device 021 performs near-infrared light compensation, and near-infrared light reflected by the object in the ambient light. The intensity of the near infrared light entering the filter assembly 03 is stronger at this time.

However, in the absence of the near-infrared light compensation, the near-infrared light passing through the first filter 031 includes near-infrared light reflected by the object in the ambient light and entering the filter assembly 03. Since there is no near infrared light supplemented by the first light supplement device 021, the intensity of the near infrared light passing through the first filter 031 is weak at this time. Therefore, the intensity of near-infrared light included in the first image signal generated and output according to the first preset exposure is higher than the intensity of near-infrared light included in the second image signal generated and output according to the second preset exposure.

The first light supplement device 021 can have multiple choices for the center wavelength and/or the waveband range of near-infrared light supplement, in this embodiment of the application, in order to make the first light supplement device 021 and the first optical filter 031 have better cooperation, the center wavelength of near-infrared light supplement can be designed for the first light supplement device 021, and the characteristic of the first optical filter 031 is selected, thereby make the center wavelength of near-infrared light supplement be for setting for the characteristic wavelength or fall when setting for the characteristic wavelength range at the first light supplement device 021, the center wavelength and/or the waveband width of near-infrared light through the first optical filter 031 can reach the constraint condition.

The constraint condition is mainly used to constrain the center wavelength of the near-infrared light passing through the first optical filter 031 to be as accurate as possible, and the band width of the near-infrared light passing through the first optical filter 031 to be as narrow as possible, so as to avoid the occurrence of wavelength interference caused by too wide band width of the near-infrared light.

The central wavelength of the near-infrared light supplement by the first light supplement device 021 may be an average value in a wavelength range where energy in a spectrum of the near-infrared light emitted by the first light supplement device 021 is the maximum, or may be a wavelength at an intermediate position in a wavelength range where energy in the spectrum of the near-infrared light emitted by the first light supplement device 021 exceeds a certain threshold.

The set characteristic wavelength or the set characteristic wavelength range may be preset. As an example, the center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device 021 may be any wavelength within a wavelength range of 750 ± 10 nanometers; or, the center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device 021 is any wavelength within the wavelength range of 780 ± 10 nanometers; or, the center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device 021 is any wavelength within the wavelength range of 940 ± 10 nanometers; alternatively, the center wavelength of the near-infrared light supplement performed by the first light supplement device 021 is 810 ± 10 nm, that is, the set characteristic wavelength range may be 750 ± 10 nm, 780 ± 10 nm, 940 ± 10 nm, or 810 ± 10 nm. Illustratively, the center wavelength of the first fill-in light device 021 for near-infrared fill-in light is 940 nm, and the relationship between the wavelength and the relative intensity of the first fill-in light device 021 for near-infrared fill-in light is shown in fig. 2. As can be seen from fig. 2, the wavelength band of the first light supplement device 021 for performing near-infrared light supplement is 900 nm to 1000 nm, wherein at 940 nm, the relative intensity of the near-infrared light is the highest.

Since most of the near-infrared light passing through the first optical filter 031 is near-infrared light entering the first optical filter 031 after being reflected by the object when the first fill-in light device 021 performs near-infrared fill-in light, in some embodiments, the constraint conditions may include: the difference between the central wavelength of the near-infrared light passing through the first optical filter 031 and the central wavelength of the near-infrared light supplemented by the first light supplementing device 021 is within a wavelength fluctuation range, which may be 0 to 20 nm, as an example.

The central wavelength of the near-infrared supplementary light passing through the first optical filter 031 may be a wavelength at a peak position in a near-infrared band range in the near-infrared light transmittance curve of the first optical filter 031, or may be a wavelength at a middle position in a near-infrared band range in which a transmittance exceeds a certain threshold in the near-infrared light transmittance curve of the first optical filter 031.

In order to avoid introducing wavelength interference due to too wide band width of the near infrared light passing through the first filter 031, in some embodiments, the constraint conditions may include: the first band width may be less than the second band width. The first wavelength band width refers to the wavelength band width of the near-infrared light passing through the first filter 031, and the second wavelength band width refers to the wavelength band width of the near-infrared light blocked by the first filter 031. It should be understood that the band width refers to the width of the wavelength range in which the wavelength of the light is located. For example, the wavelength of the near infrared light passing through the first filter 031 is in the wavelength range of 700 nm to 800 nm, and then the first wavelength band width is 800 nm minus 700 nm, i.e., 100 nm. In other words, the wavelength band width of the near infrared light passing through the first filter 031 is smaller than the wavelength band width of the near infrared light blocked by the first filter 031.

For example, referring to fig. 3, fig. 3 is a schematic diagram illustrating a relationship between a wavelength of light that can pass through the first filter 031 and a pass rate. The band of the near-infrared light incident to the first optical filter 031 is 650 nm to 1100 nm, the first optical filter 031 allows visible light having a wavelength of 380 nm to 650 nm to pass through, near-infrared light having a wavelength of 900 nm to 1100 nm to pass through, and near-infrared light having a wavelength of 650 nm to 900 nm to be blocked. That is, the first band width is 1000 nanometers minus 900 nanometers, i.e., 100 nanometers. The second band has a width of 900 nm minus 650 nm plus 1100 nm minus 1000 nm, i.e., 350 nm. 100 nm is smaller than 350 nm, that is, the band width of the near infrared light passing through the first optical filter 031 is smaller than the band width of the near infrared light blocked by the first optical filter 031. The above relation is only an example, and the wavelength range of the near-red light band that can pass through the filter may be different for different filters, and the wavelength range of the near-infrared light that is blocked by the filter may also be different.

In order to avoid introducing wavelength interference due to too wide band width of the near-infrared light passing through the first filter 031 during the non-near-infrared light supplement period, in some embodiments, the constraint conditions may include: the half-bandwidth of the near infrared light passing through the first filter 031 is less than or equal to 50 nm. The half bandwidth refers to the band width of near infrared light with a passing rate of more than 50%.

In order to avoid introducing wavelength interference due to too wide band width of the near infrared light passing through the first filter 031, in some embodiments, the constraint conditions may include: the third band width may be less than the reference band width. The third wavelength band width is a wavelength band width of the near infrared light having a transmittance greater than a set ratio, and as an example, the reference wavelength band width may be any one of wavelength band widths in a wavelength band range of 50 nm to 100 nm. The set proportion may be any proportion of 30% to 50%, and of course, the set proportion may be set to other proportions according to the use requirement, which is not limited in the embodiment of the present application. In other words, the band width of the near infrared light having the passing rate larger than the set ratio may be smaller than the reference band width.

For example, referring to fig. 3, the wavelength band of the near infrared light incident to the first filter 031 is 650 nm to 1100 nm, the set ratio is 30%, and the reference wavelength band width is 100 nm. As can be seen from fig. 3, in the wavelength band of the near-infrared light of 650 nm to 1100 nm, the wavelength band width of the near-infrared light with the transmittance of more than 30% is significantly less than 100 nm.

Because the first light supplement device 021 provides near-infrared light supplement at least in the partial exposure time period of the first preset exposure, the near-infrared light supplement is not provided in the whole exposure time period of the second preset exposure, and the first preset exposure and the second preset exposure are two exposures of multiple exposures of the image sensor 01, that is, the first light supplement device 021 provides near-infrared light supplement in the exposure time period of the partial exposure of the image sensor 01, and the near-infrared light supplement is not provided in the exposure time period of the other partial exposure of the image sensor 01. Therefore, the number of light supplement times of the first light supplement device 021 in a unit time length can be lower than the number of exposure times of the image sensor 01 in the unit time length, wherein one or more exposures are spaced in each interval time period of two adjacent light supplement.

In one possible implementation manner, since human eyes easily mix the color of the near-infrared light supplementary lighting performed by the first supplementary lighting device 021 with the color of the red light in the traffic light, referring to fig. 4, the supplementary lighting device 02 may further include a second supplementary lighting device 022, and the second supplementary lighting device 022 is used for performing visible light supplementary lighting. Like this, if second light filling device 022 provides the visible light filling at the partial exposure time of first preset exposure at least, promptly, has near-infrared light filling and visible light filling in the partial exposure time quantum of first preset exposure at least, and the mixed colour of these two kinds of light can be distinguished from the colour of the red light in the traffic light to the colour that the people's eye carries out near-infrared light filling with light filling ware 02 and the colour of the red light in the traffic light is confused has been avoided. In addition, if the second light supplement device 022 provides supplementary lighting for visible light in the exposure time period of the second preset exposure, since the intensity of visible light in the exposure time period of the second preset exposure is not particularly high, the brightness of visible light in the second image signal can be further improved when the supplementary lighting for visible light is performed in the exposure time period of the second preset exposure, and the quality of image acquisition is further ensured.

In some embodiments, the second light supplement device 022 can be used for supplementing visible light in a normally bright manner; or, the second light supplement device 022 may be configured to supplement the visible light in a stroboscopic manner, where the supplementary visible light is present at least in a partial exposure time period of the first preset exposure, and the supplementary visible light is absent in the entire exposure time period of the second preset exposure; or, the second light supplement device 022 may be configured to perform light supplement of visible light in a stroboscopic manner, where the light supplement of visible light does not exist at least in the whole exposure time period of the first preset exposure, and the light supplement of visible light exists in a part of the exposure time period of the second preset exposure. When the second light supplement device 022 is normally on, visible light is supplemented, so that the color of the first light supplement device 021 for near-infrared light supplement can be prevented from being mixed up with the color of the red light in the traffic light by human eyes, the brightness of the visible light in the second image signal can be improved, and the quality of image acquisition is ensured. When second light filling device 022 carries out visible light filling with the stroboscopic mode, can avoid the colour that human eye carries out near-infrared light filling with first light filling device 021 and the colour of the red light in the traffic light to obscure, perhaps, can improve the luminance of the visible light in the second image signal, and then guarantee image acquisition's quality, but also can reduce the light filling number of times of second light filling device 022 to prolong the life of second light filling device 022.

In some embodiments, the multiple exposure refers to multiple exposures within one frame period, that is, the image sensor 01 performs multiple exposures within one frame period, so as to generate and output at least one frame of the first image signal and at least one frame of the second image signal. For example, the image sensor 01 performs exposure for a plurality of times in each frame period for 1 second, thereby generating at least one frame of the first image signal and at least one frame of the second image signal, and the first image signal and the second image signal generated in one frame period are referred to as a set of image signals, so that 25 sets of image signals are generated in 25 frame periods. The first preset exposure and the second preset exposure may be adjacent two exposures in multiple exposures within one frame period, or may also be nonadjacent two exposures in multiple exposures within one frame period, which is not limited in this embodiment of the application.

The first image signal is generated and output for a first preset exposure, the second image signal is generated and output for a second preset exposure, and the first image signal and the second image signal may be processed after the first image signal and the second image signal are generated and output. In some cases, the first image signal and the second image signal may be used differently, so in some embodiments, at least one exposure parameter of the first preset exposure and the second preset exposure may be different. As an example, the at least one exposure parameter may include, but is not limited to, one or more of exposure time, analog gain, digital gain, aperture size. Wherein the exposure gain comprises an analog gain and/or a digital gain.

In some embodiments. It is understood that, compared to the second preset exposure, when the near-infrared light supplement exists, the intensity of the near-infrared light sensed by the image sensor 01 is stronger, and accordingly, the brightness of the near-infrared light included in the generated and outputted first image signal is higher. But the higher brightness near infrared light is not favorable for the acquisition of external scene information. Also, in some embodiments, the larger the exposure gain, the higher the brightness of the image signal output by the image sensor 01, and the smaller the exposure gain, the lower the brightness of the image signal output by the image sensor 01, and therefore, in order to ensure that the brightness of the near-infrared light included in the first image signal is within a suitable range, in the case where at least one exposure parameter of the first preset exposure and the second preset exposure is different, as an example, the exposure gain of the first preset exposure may be smaller than the exposure gain of the second preset exposure. Thus, when the first light supplement device 021 performs near-infrared light supplement, the brightness of near-infrared light included in the first image signal generated and output by the image sensor 01 is not too high due to the near-infrared light supplement performed by the first light supplement device 021.

In other embodiments, the longer the exposure time, the higher the brightness included in the image signal obtained by the image sensor 01, and the longer the motion smear of the moving object in the external scene in the image signal; the shorter the exposure time, the lower the brightness included in the image signal obtained by the image sensor 01, and the shorter the motion smear of the moving object in the external scene in the image signal. Therefore, in order to ensure that the brightness of the near-infrared light contained in the first image signal is within a proper range, and the motion tail of the moving object in the external scene in the first image signal is short. In a case where at least one exposure parameter of the first preset exposure and the second preset exposure is different, as an example, the exposure time of the first preset exposure may be smaller than the exposure time of the second preset exposure. Thus, when the first light supplement device 021 performs near-infrared light supplement, the brightness of near-infrared light included in the first image signal generated and output by the image sensor 01 is not too high due to the near-infrared light supplement performed by the first light supplement device 021. And the shorter exposure time makes the motion smear of the moving object in the external scene appearing in the first image signal shorter, thereby facilitating the identification of the moving object. Illustratively, the exposure time of the first preset exposure is 40 milliseconds, the exposure time of the second preset exposure is 60 milliseconds, and so on.

It is noted that, in some embodiments, when the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure, the exposure time of the first preset exposure may be not only smaller than the exposure time of the second preset exposure, but also equal to the exposure time of the second preset exposure. Similarly, when the exposure time of the first preset exposure is shorter than the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be smaller than or equal to the exposure gain of the second preset exposure.

In other embodiments, the first image signal and the second image signal may be used for the same purpose, for example, when both the first image signal and the second image signal are used for intelligent analysis, at least one exposure parameter of the first preset exposure and the second preset exposure may be the same in order to enable the same definition of the human face or the target under intelligent analysis when the human face or the target moves. As an example, the exposure time of the first preset exposure may be equal to the exposure time of the second preset exposure, and if the exposure time of the first preset exposure is different from the exposure time of the second preset exposure, a motion smear may exist in one path of image signals with a longer exposure time, resulting in different resolutions of the two paths of image signals. Likewise, as another example, the exposure gain of the first preset exposure may be equal to the exposure gain of the second preset exposure.

It is noted that, in some embodiments, when the exposure time of the first preset exposure is equal to the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be smaller than or equal to the exposure gain of the second preset exposure. Similarly, when the exposure gain of the first preset exposure is equal to the exposure gain of the second preset exposure, the exposure time of the first preset exposure may be shorter than the exposure time of the second preset exposure, or may be equal to the exposure time of the second preset exposure.

The image sensor 01 may include a plurality of light sensing channels, each of which may be configured to sense light in at least one visible light band and to sense light in a near infrared band. That is, each photosensitive channel can sense light of at least one visible light band and light of a near-infrared band, so that complete resolution can be ensured in the first image signal and the second image signal, and pixel values are not lost. In one possible implementation, the multiple sensing channels may be used to sense light in at least two different visible wavelength bands.

In some embodiments, the plurality of photosensitive channels may include at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, a Y photosensitive channel, a W photosensitive channel, and a C photosensitive channel. The light sensing device comprises a light sensing channel, a light sensing channel and a light sensing channel, wherein the light sensing channel R is used for sensing light of a red light wave band and a near infrared wave band, the light sensing channel G is used for sensing light of a green light wave band and a near infrared wave band, the light sensing channel B is used for sensing light of a blue light wave band and a near infrared wave band, and the light sensing channel Y is used for sensing light of a yellow light wave band and a near infrared wave band. Since in some embodiments, the photosensitive channel for sensing the light of the full wavelength band may be denoted by W, and in other embodiments, the photosensitive channel for sensing the light of the full wavelength band may be denoted by C, when the plurality of photosensitive channels include the photosensitive channel for sensing the light of the full wavelength band, the photosensitive channel may be the photosensitive channel of W, and may also be the photosensitive channel of C. That is, in practical applications, the photosensitive channel for sensing the light of the full wavelength band can be selected according to the use requirement. Illustratively, the image sensor 01 may be an RGB sensor, an RGBW sensor, or an RCCB sensor, or an ryb sensor. The distribution mode of the R photosensitive channels, the G photosensitive channels and the B photosensitive channels in the RGB sensor can be seen in fig. 5, the distribution mode of the R photosensitive channels, the G photosensitive channels, the B photosensitive channels and the W photosensitive channels in the RGBW sensor can be seen in fig. 6, the distribution mode of the R photosensitive channels, the C photosensitive channels and the B photosensitive channels in the RCCB sensor can be seen in fig. 7, and the distribution mode of the R photosensitive channels, the Y photosensitive channels and the B photosensitive channels in the RYYB sensor can be seen in fig. 8.

In other embodiments, some of the photosensitive channels may sense only light in the near infrared band and not light in the visible band, so as to ensure complete resolution in the first image signal without missing pixel values. As an example, the plurality of photosensitive channels may include at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and an IR photosensitive channel. The R light sensing channel is used for sensing light of a red light wave band and a near infrared wave band, the G light sensing channel is used for sensing light of a green light wave band and a near infrared wave band, the B light sensing channel is used for sensing light of a blue light wave band and a near infrared wave band, and the IR light sensing channel is used for sensing light of a near infrared wave band.

Illustratively, the image sensor 01 may be an rgbiir sensor, wherein each IR photosensitive channel in the rgbiir sensor may sense light in the near infrared band, but not light in the visible band.

When the image sensor 01 is an RGB sensor, compared with other image sensors such as an rgbiir sensor, RGB information acquired by the RGB sensor is more complete, and a part of photosensitive channels of the rgbiir sensor cannot acquire visible light, so that color details of an image acquired by the RGB sensor are more accurate.

It is noted that the image sensor 01 may include a plurality of photosensitive channels corresponding to a plurality of sensing curves. Illustratively, referring to fig. 9, an R curve in fig. 9 represents a sensing curve of the image sensor 01 for light in a red wavelength band, a G curve represents a sensing curve of the image sensor 01 for light in a green wavelength band, a B curve represents a sensing curve of the image sensor 01 for light in a blue wavelength band, a W (or C) curve represents a sensing curve of the image sensor 01 for light in a full wavelength band, and an NIR (Near infrared) curve represents a sensing curve of the image sensor 01 for light in a Near infrared wavelength band.

In summary, when the intensity of visible light in ambient light is weak, for example, at night, the first light supplement device 021 may be used to perform flash light supplement, so that the image sensor 01 generates and outputs a first image signal containing near-infrared luminance information and a second image signal containing visible light luminance information, and both the first image signal and the second image signal are acquired by the same image sensor 01, so that the viewpoint of the first image signal is the same as the viewpoint of the second image signal, and thus the complete information of an external scene may be acquired through the first image signal and the second image signal. When the visible light intensity is strong, for example, in the daytime, the proportion of near infrared light in the daytime is strong, the color reduction degree of the acquired image is poor, and a third image signal containing visible light brightness information can be generated and output by the image sensor 01, so that even in the daytime, an image with good color reduction degree can be acquired, the real color information of an external scene can be efficiently and simply acquired no matter the intensity of the visible light intensity, or no matter day or night, the use flexibility of the image acquisition device is improved, and the image acquisition device can be conveniently compatible with other image acquisition devices.

This application is through setting up image sensor and according to first preset exposure and the preset exposure of second and carrying out multiple exposure, so that carry out near-infrared light filling and produce first image signal at the in-process of first preset exposure, do not carry out near-infrared light filling and produce the second image signal at the in-process of the preset exposure of second, such data acquisition mode, can be simple structure, directly gather the first image signal and the second image signal that luminance information is different in the time of reduce cost, also be the image signal of two kinds of differences that can gather simultaneously in a period, make this image acquisition device more nimble, and then make and acquire first image signal and second image signal also more high-efficient. And setting the ending time of the near-infrared supplementary lighting and the starting time of the near-infrared supplementary lighting of the first supplementary lighting device according to the exposure ending time and the exposure starting time of the effective images of the first image signal and the second image signal, so that the color of the second image signal output in a time period corresponding to the second preset exposure without the near-infrared supplementary lighting is accurate.

Based on the above description of the image pickup apparatus, the image pickup apparatus can generate and output the first image signal and the second image signal by a plurality of exposures. Next, an image capturing method will be described with an image capturing apparatus provided based on the embodiment shown in fig. 1A to 14 described above. Referring to fig. 15, the method includes:

step 1501: and performing near-infrared light supplement through a first light supplement device included by the light supplement device, wherein the near-infrared light supplement exists at least in part of the exposure time period of the first preset exposure, the near-infrared light supplement does not exist in the exposure time period of the second preset exposure, and the first preset exposure and the second preset exposure are two exposures of the multiple exposures.

Step 1502: the pass filter assembly includes a first filter that passes visible light and a portion of near infrared light.

Step 1503: the method comprises the steps of carrying out multiple exposure through an image sensor to generate and output at least one frame of first image signal and at least one frame of second image signal, wherein the first image signal is an image signal generated according to first preset exposure, the second image signal is an image signal generated according to second preset exposure, the image sensor is used for carrying out multiple exposure in a global exposure starting mode, in an exposure time period of the first preset exposure, exposure starting moments of all rows of effective images of the first image signal are the same, exposure ending moments of at least two rows of effective images of the first image signal are different, in an exposure time period of the second preset exposure, exposure starting moments of all rows of effective images of the second image signal are the same, and exposure ending moments of at least two rows of effective images of the second image signal are different.

Optionally, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than the exposure ending time of all the effective images in all the lines of the latest frame of second image signal before the current frame, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is earlier than the exposure starting time of any effective image in any line of the latest frame of second image signal after the current frame.

Optionally, the exposure end times of all the lines of the effective image of the first image signal are different;

the exposure end timings of all the lines of the effective image of the second image signal are different.

Optionally, in each frame of image signal corresponding to the image sensor, the difference between the exposure end times of the adjacent lines of effective images is less than 40 microseconds.

Optionally, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than the exposure ending time of the last line of effective image of the last frame of second image signal before the current frame, and the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is earlier than or equal to the exposure starting time of any line of effective image of the last frame of second image signal after the current frame.

the starting time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device is later than or equal to the exposure ending time of the last line of effective image of the last frame of second image signal before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal of the current first preset exposure; and is

Optionally, the filter assembly may further include a second filter and a switching member, and at this time, the second filter may be further switched to the light incident side of the image sensor by the switching member. After the second optical filter is switched to the light incident side of the image sensor, light in a visible light waveband is made to pass through the second optical filter, light in a near infrared light waveband is blocked, and after the second optical filter passes through the light in the visible light waveband and blocks the light in the near infrared light waveband, exposure is carried out through the image sensor, so that a third image signal is generated and output.

Optionally, the light supplement device may further include a second light supplement device, and at this time, before the light in the visible light band and the light in the near-infrared light band pass through the first optical filter included in the optical filtering component, the second light supplement device is further used for supplementing the visible light.

Optionally, the intensity of the near-infrared light passing through the first optical filter when the first light supplement device performs near-infrared light supplement is higher than the intensity of the near-infrared light passing through the first optical filter when the first light supplement device does not perform near-infrared light supplement.

Optionally, the exposure time periods of the first preset exposure and the second preset exposure are alternately arranged in a cycle.

Optionally, the wavelength range of the near-infrared light incident to the first optical filter is a first reference wavelength range, and the first reference wavelength range is 650 nm to 1100 nm.

Optionally, when the central wavelength of the near-infrared light supplement by the first light supplement device is the set characteristic wavelength or falls within the set characteristic wavelength range, the central wavelength and/or the band width of the near-infrared light passing through the first optical filter reach the constraint condition.

The center wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device is any wavelength within the wavelength range of 940 +/-10 nanometers; or

The central wavelength of the near-infrared supplementary lighting performed by the first supplementary lighting device is 810 +/-10 nanometers.

Optionally, the constraint includes:

The third wave band width is smaller than the reference wave band width, the third wave band width is the wave band width of the near infrared light with the passing rate larger than the set proportion, and the reference wave band width is any wave band width in the wave band range of 50 nanometers to 150 nanometers.

Alternatively, the set ratio is any ratio within a ratio range of 30% to 50%.

Optionally, the at least one exposure parameter of the first preset exposure and the second preset exposure is different, the at least one exposure parameter is one or more of exposure time, exposure gain, aperture size, the exposure gain comprises analog gain, and/or digital gain.

Optionally, the image sensor comprises a plurality of photosensitive channels, each photosensitive channel being configured to sense light in at least one visible wavelength band and to sense light in a near infrared wavelength band.

Optionally, a plurality of light sensing channels are used to sense light in at least two different visible light bands.

Optionally, the plurality of photosensitive channels includes at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, a Y photosensitive channel, a W photosensitive channel, and a C photosensitive channel;

Optionally, the image sensor is an RGB sensor, an RGBW sensor, or an RCCB sensor, or an ryb sensor.

Optionally, the second light supplement device is configured to supplement the visible light in a normally bright manner; or

The second light supplement device is used for supplementing visible light in a stroboscopic mode, wherein the visible light supplement exists at least in part of the exposure time period of the first preset exposure, and the visible light supplement does not exist in the whole exposure time period of the second preset exposure; or

The second light supplement device is used for supplementing visible light in a stroboscopic mode, wherein the visible light supplement does not exist at least in the whole exposure time period of the first preset exposure, and the visible light supplement exists in part of the exposure time period of the second preset exposure.

It should be noted that, since the present embodiment and the embodiment shown in fig. 1A to 14 may adopt the same application concept, for the explanation of the present embodiment, reference may be made to the explanation of the relevant content in the embodiment shown in fig. 1A to 14, and details are not described here.

In the embodiment of the application, multiple exposures are carried out by arranging the image sensor according to the first preset exposure and the second preset exposure, so that near-infrared light supplement is carried out and a first image signal is generated in the first preset exposure process, and near-infrared light supplement is not carried out and a second image signal is generated in the second preset exposure process. And the ending time and the starting time of the near-infrared supplementary lighting of the first supplementary lighting device are set according to the exposure ending time and the exposure starting time of the effective images of the first image signal and the second image signal, so that the color of the second image signal output in a time period corresponding to the second preset exposure without the near-infrared supplementary lighting is accurate.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image capturing apparatus, characterized in that the image capturing apparatus comprises: the device comprises an image sensor (01), a light supplementing device (02) and a light filtering component (03), wherein the image sensor (01) is positioned on the light emitting side of the light filtering component (03);

the image sensor (01) is configured to generate and output at least one frame of first image signal and at least one frame of second image signal through multiple exposures, where the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure and the second preset exposure are two exposures of the multiple exposures, the image sensor is configured to perform multiple exposures in a global exposure starting manner, within an exposure time period of the first preset exposure, exposure starting times of all row effective images of the first image signal are the same, exposure ending times of all row effective images of the first image signal are different, within an exposure time period of the second preset exposure, exposure starting times of all row effective images of the second image signal are the same, the exposure end time of all the effective images of all the lines of the second image signal are different;

the light supplement device (02) comprises a first light supplement device (021), the first light supplement device (021) is used for performing near-infrared light supplement, wherein at least near-infrared light supplement exists in the exposure time period of the first preset exposure, near-infrared light supplement does not exist in the exposure time period of the second preset exposure, and the starting time of the near-infrared light supplement of the first light supplement device (021) is later than or equal to the exposure ending time of the last row of effective images of the second image signal of the latest frame before the current frame and is earlier than the exposure ending time of the first row of effective images of the first image signal of the current frame; the ending time of the near-infrared supplementary lighting of the first supplementary lighting device (021) is later than the exposure ending time of the last row of effective images of the first image signal of the current frame and earlier than the exposure starting time of any row of effective images of the second image signal after the current frame;

the filter assembly (03) comprises a first filter (031), and the first filter (031) allows visible light and part of near infrared light to pass through.

2. The image capturing device as claimed in claim 1, wherein the difference between the exposure end times of the adjacent rows of effective images in each frame of image signals corresponding to the image sensor is less than 40 μ sec.

3. The image capturing device according to claim 1, wherein the filter assembly (03) further comprises a second filter and a switching member, and the first filter (031) and the second filter are connected to the switching member;

the switching component is used for switching the second optical filter to the light inlet side of the image sensor (01);

after the second optical filter is switched to the light incidence side of the image sensor (01), the second optical filter allows light in a visible light wave band to pass and blocks light in a near infrared light wave band, and the image sensor (01) is used for generating and outputting a third image signal through exposure.

4. The image capturing apparatus of claim 1,

when the central wavelength of the near-infrared light supplement performed by the first light supplement device (021) is a set characteristic wavelength or falls within a set characteristic wavelength range, the central wavelength and/or the waveband width of the near-infrared light passing through the first optical filter (031) reach a constraint condition.

5. The image capturing device according to claim 4, wherein the first fill-in light device (021) performs near-infrared fill-in light at any wavelength within a wavelength range of 750 ± 10 nm; or

The center wavelength of the first light supplement device (021) for near-infrared light supplement is any wavelength within the wavelength range of 780 +/-10 nanometers; or

The center wavelength of the first light supplement device (021) for near-infrared light supplement is any wavelength within the wavelength range of 940 +/-10 nanometers; or

6. The image capturing apparatus according to claim 4, wherein the constraint condition includes:

the half bandwidth of the near infrared light passing through the first optical filter (031) is less than or equal to 50 nanometers; or

The first wave band width is smaller than the second wave band width; wherein the first wavelength band width refers to a wavelength band width of near infrared light passing through the first filter (031), and the second wavelength band width refers to a wavelength band width of near infrared light blocked by the first filter (031); or

And the third wave band width is smaller than the reference wave band width, the third wave band width refers to the wave band width of the near infrared light with the passing rate larger than the set proportion, and the reference wave band width is any wave band width in a wave band range of 50-150 nanometers.

7. The image capturing apparatus of claim 1,

the first preset exposure and the second preset exposure are different in at least one exposure parameter, the at least one exposure parameter is one or more of exposure time, exposure gain and aperture size, and the exposure gain comprises analog gain and/or digital gain.

8. The image capture device of claim 7, wherein an exposure gain of the first pre-set exposure is less than an exposure gain of the second pre-set exposure.

9. The image capturing device according to claim 1, wherein the image sensor (01) comprises a plurality of photosensitive channels, each photosensitive channel being configured to sense light in at least one visible wavelength band and to sense light in a near infrared wavelength band.

10. The image acquisition device according to claim 1 or 2, wherein the fill-in light device (02) further comprises a second fill-in light device (022), the second fill-in light device (022) being adapted to fill in visible light; or

The second light supplement device (022) is used for supplementing visible light in a normally bright mode; or

The second light supplement device (022) is configured to supplement the visible light in a stroboscopic manner, wherein the supplementary visible light exists at least in a partial exposure time period of the first preset exposure, and the supplementary visible light does not exist in the whole exposure time period of the second preset exposure; or

The second light supplement device (022) is configured to supplement the visible light in a stroboscopic manner, wherein the first light supplement device is not present during the entire exposure time period of the first preset exposure, and the second light supplement device is present during a partial exposure time period of the second preset exposure.

11. An image capturing apparatus, characterized in that the image capturing apparatus comprises: the device comprises an image sensor (01), a light supplementing device (02) and a light filtering component (03), wherein the image sensor (01) is positioned on the light emitting side of the light filtering component (03);

the image sensor (01) is configured to generate and output at least one frame of first image signal and at least one frame of second image signal through multiple exposures, where the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure and the second preset exposure are exposures in the multiple exposures, the image sensor is configured to perform multiple exposures in a global exposure starting manner, exposure starting times of all row effective images of the first image signal are the same within an exposure time period of the first preset exposure, exposure ending times of all row effective images of the first image signal are different, exposure starting times of all row effective images of the second image signal are the same within an exposure time period of the second preset exposure, the exposure end time of all the effective images of all the lines of the second image signal are different;

the light supplement device (02) comprises a first light supplement device (021), the first light supplement device (021) is used for performing near-infrared light supplement, wherein the near-infrared light supplement exists at least in the exposure time period of the first preset exposure, the near-infrared light supplement does not exist in the exposure time period of the second preset exposure, the first preset exposure comprises two times of continuous exposure to generate two continuous frames of first image signals, and the second preset exposure comprises one time of exposure to generate one frame of second image signals; the starting time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device (021) is later than or equal to the exposure ending time of the last line of effective image of the last frame of second image signal before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device (021) is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal of the current first preset exposure; the starting time of the near-infrared supplementary lighting of the second exposure in the two continuous exposures of the first supplementary lighting device (021) is later than the exposure ending time of the last line of effective image of the second image signal of the latest frame before the current first preset exposure and later than the exposure ending time of the last line of effective image of the first image signal of the latest frame before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the second exposure in the two continuous exposures of the first supplementary lighting device (021) is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal of the current first preset exposure;

12. The image capturing device as claimed in claim 11, wherein the difference between the exposure end times of the adjacent rows of effective images in each frame of image signals corresponding to the image sensor is less than 40 μ sec.

13. The image capturing device according to claim 11, wherein the filter assembly (03) further comprises a second filter and a switching member, wherein the first filter (031) and the second filter are connected to the switching member;

14. The image capturing apparatus of claim 11,

15. The image capturing device according to claim 14, wherein the first fill-in light device (021) performs near-infrared fill-in light at any wavelength within a wavelength range of 750 ± 10 nm at a center wavelength; or

16. The image capturing apparatus according to claim 14, wherein the constraint condition includes:

17. The image capturing apparatus of claim 11,

18. The image capture device of claim 17, wherein an exposure gain of the first pre-set exposure is less than an exposure gain of the second pre-set exposure.

19. The image capturing device according to claim 11, wherein the image sensor (01) comprises a plurality of photosensitive channels, each photosensitive channel being configured to sense light in at least one visible wavelength band and to sense light in a near infrared wavelength band.

20. The image acquisition apparatus according to any of the claims 11-12, wherein the fill-in light device (02) further comprises a second fill-in light device (022), the second fill-in light device (022) being adapted to fill in visible light; or

21. An image capturing apparatus, characterized in that the image capturing apparatus comprises: the device comprises an image sensor (01), a light supplementing device (02) and a light filtering component (03), wherein the image sensor (01) is positioned on the light emitting side of the light filtering component (03);

the image sensor (01) is configured to generate and output at least one frame of first image signal and at least one frame of second image signal through multiple exposures, where the first image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure includes two continuous exposures to generate two continuous frames of first image signals, the second preset exposure includes one exposure to generate one frame of second image signal, the image sensor is configured to perform multiple exposures in a global exposure starting manner, exposure starting times of all row effective images of the first image signal are the same in an exposure time period of the first preset exposure, exposure ending times of all row effective images of the first image signal are different in an exposure time period of the second preset exposure, the exposure starting time of all the line effective images of the second image signal is the same, and the exposure ending time of all the line effective images of the second image signal is different;

the light supplement device (02) comprises a first light supplement device (021), the first light supplement device (021) is used for performing near-infrared light supplement, wherein at least near-infrared light supplement exists in the exposure time period of the first preset exposure, near-infrared light supplement does not exist in the exposure time period of the second preset exposure, the starting time of the near-infrared light supplement of the first light supplement device (021) is later than or equal to the exposure ending time of the last line of effective images of the last frame of second image signals before the current first preset exposure, and the ending time of the near-infrared light supplement of the first light supplement device (021) is earlier than or equal to the exposure starting time of any line of effective images of the last frame of second image signals after the current first preset exposure;

22. The image capturing device as claimed in claim 21, wherein the difference between the exposure end times of the adjacent rows of effective images in each frame of image signals corresponding to the image sensor is less than 40 μ sec.

23. The image capturing device according to claim 21, wherein the filter assembly (03) further comprises a second filter and a switching component, wherein the first filter (031) and the second filter are connected to the switching component;

24. The image capturing apparatus of claim 21,

25. The image capturing device according to claim 24, wherein the first fill-in light device (021) performs near-infrared fill-in light at any wavelength within a wavelength range of 750 ± 10 nm at a center wavelength; or

26. The image capturing apparatus of claim 24, wherein the constraint condition includes:

27. The image capturing apparatus of claim 21,

28. The image capturing device of claim 27, wherein an exposure gain of the first pre-set exposure is less than an exposure gain of the second pre-set exposure.

29. The image capturing device according to claim 21, wherein the image sensor (01) comprises a plurality of photosensitive channels, each photosensitive channel being configured to sense light in at least one visible wavelength band and to sense light in a near infrared wavelength band.

30. The image acquisition apparatus according to claim 21 or 22, wherein the fill-in light device (02) further comprises a second fill-in light device (022), the second fill-in light device (022) being configured to fill in visible light; or

31. An image acquisition method is applied to an image acquisition device, and the image acquisition device comprises: image sensor, light filling ware and filtering component, the light filling ware includes first light filling device, filtering component includes first light filter, image sensor (01) are located filtering component's (03) light-emitting side, its characterized in that, the method includes:

near-infrared supplementary lighting is carried out through a first supplementary lighting device which is included by a supplementary lighting device, wherein near-infrared supplementary lighting exists at least in a part of exposure time period of first preset exposure, near-infrared supplementary lighting does not exist in the exposure time period of second preset exposure, the first preset exposure and the second preset exposure are two times of multiple exposure, the starting time of the near-infrared supplementary lighting of the first supplementary lighting device is later than or equal to the exposure ending time of the last line of effective images of the second image signal of the latest frame before the current frame, and is earlier than the exposure ending time of the first line of effective images of the first image signal of the current frame; the ending time of the near-infrared supplementary lighting of the first supplementary lighting device is later than the exposure ending time of the last row of effective images of the first image signal of the current frame and earlier than the exposure starting time of any row of effective images of the second image signal after the current frame;

the first filter included in the filter assembly is used for enabling visible light and partial near infrared light to pass through;

performing multiple exposures through the image sensor to generate and output at least one frame of a first image signal and at least one frame of a second image signal, the first image signal is an image signal generated according to the first preset exposure, the second image signal is an image signal generated according to the second preset exposure, the image sensor is used for carrying out multiple exposures by adopting a global exposure starting mode, within the exposure time period of the first preset exposure, the exposure start time of all the line effective images of the first image signal is the same, the exposure end time of all the line effective images of the first image signal is different, in the exposure time period of the second preset exposure, the exposure start times of all the row effective images of the second image signal are the same, and the exposure end times of all the row effective images of the second image signal are different.

32. The method of claim 31, wherein the difference between the exposure end times of the adjacent lines of the active images in each frame of the image signal corresponding to the image sensor is less than 40 μ sec.

33. The method of claim 31, wherein the filter assembly further comprises a second filter and a switching member, the method further comprising:

34. The method of claim 31, wherein the fill-in light further comprises a second fill-in light device, and before the passing of the light in the visible wavelength band and the light in the near infrared wavelength band through the first filter included in the filter assembly, the method further comprises:

35. An image acquisition method is applied to an image acquisition device, and the image acquisition device comprises: image sensor, light filling ware and filtering component, the light filling ware includes first light filling device, filtering component includes first light filter, image sensor (01) are located filtering component's (03) light-emitting side, its characterized in that, the method includes:

performing near-infrared supplementary lighting through a first supplementary lighting device included by a supplementary lighting device, wherein the near-infrared supplementary lighting exists at least in a part of exposure time period of a first preset exposure, the near-infrared supplementary lighting does not exist in the exposure time period of a second preset exposure, the first preset exposure and the second preset exposure are exposures in multiple exposures, the first preset exposure comprises two continuous exposures to generate two continuous frames of first image signals, and the second preset exposure comprises one exposure to generate one frame of second image signals; the starting time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device (021) is later than or equal to the exposure ending time of the last line of effective image of the last frame of second image signal before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the first exposure in the two continuous exposures of the first supplementary lighting device (021) is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal of the current first preset exposure; the starting time of the near-infrared supplementary lighting of the second exposure in the two continuous exposures of the first supplementary lighting device (021) is later than the exposure ending time of the last line of effective image of the second image signal of the latest frame before the current first preset exposure and later than the exposure ending time of the last line of effective image of the first image signal of the latest frame before the current first preset exposure, and the ending time of the near-infrared supplementary lighting of the second exposure in the two continuous exposures of the first supplementary lighting device (021) is earlier than or equal to the exposure ending time of the first line of effective image of the first image signal of the current first preset exposure;

36. The method of claim 35, wherein the difference between the exposure end times of the adjacent lines of the active images in each frame of the image signal corresponding to the image sensor is less than 40 μ sec.

37. The method of claim 35, wherein the filter assembly further comprises a second filter and a switching member, the method further comprising:

38. The method of claim 35, wherein the fill-in light further comprises a second fill-in light device, and before the passing of the light in the visible wavelength band and the light in the near infrared wavelength band through the first filter included in the filter assembly, the method further comprises:

39. An image acquisition method is applied to an image acquisition device, and the image acquisition device comprises: image sensor, light filling ware and filtering component, the light filling ware includes first light filling device, filtering component includes first light filter, image sensor (01) are located filtering component's (03) light-emitting side, its characterized in that, the method includes:

the near-infrared supplementary lighting is carried out through a first supplementary lighting device which is included by the supplementary lighting device, wherein the near-infrared supplementary lighting exists at least in a part of the exposure time period of the first preset exposure, the near-infrared supplementary lighting does not exist in the exposure time period of the second preset exposure, the first preset exposure comprises two times of continuous exposure to generate two continuous frames of first image signals, the second preset exposure comprises one exposure to generate a frame of second image signal, the starting time of the near infrared light supplement of the first light supplement device (021) is later than or equal to the exposure ending time of the last line of effective image of the last frame of second image signal before the current first preset exposure, the ending time of the near-infrared supplementary lighting of the first supplementary lighting device (021) is earlier than or equal to the exposure starting time of any effective image in any row of a latest frame of second image signal after the current first preset exposure;

40. The method of claim 39, wherein the difference between the exposure end times of adjacent lines of active images in each frame of image signal corresponding to the image sensor is less than 40 μ sec.

41. The method of claim 39, wherein the filter assembly further comprises a second filter and a switching member, the method further comprising:

42. The method of claim 39, wherein the fill-in light further comprises a second fill-in light device, and before the passing of the light in the visible wavelength band and the light in the near infrared wavelength band through the first filter included in the filter assembly, the method further comprises: