WO2020238806A1

WO2020238806A1 - Image collection apparatus and photography method

Info

Publication number: WO2020238806A1
Application number: PCT/CN2020/091912
Authority: WO
Inventors: 范蒙
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2019-05-31
Filing date: 2020-05-22
Publication date: 2020-12-03
Also published as: CN110493491A; CN110493491B

Abstract

The present application relates to an image collection apparatus and a photography method, which fall within the field of monitoring. The image collection apparatus comprises: an image sensor, a light compensation device and a light filter assembly, wherein the image sensor is located at a light-emergent side of the light filter assembly; and further comprises an image signal processing unit, a coding compression unit and an analysis unit, wherein the image sensor is used for generating a first original image signal and a second original image signal by means of exposures multiple times, and outputting same; the light compensation device comprises a first light compensation apparatus, and the first light compensation apparatus is used for performing near-infrared light compensation by stroboscopic means; the light filter assembly comprises a first light filter sheet, and the first light filter sheet is transmitted by visible light and part of near-infrared light; the image signal processing unit is used for obtaining a grayscale image according to the first original image signal, and obtaining a color image according to the second original image signal; the coding compression unit is used for performing compression coding on the color image; and the analysis unit is used for intelligently analyzing the grayscale image. According to the present application, precision of an analysis result can be improved.

Description

Image acquisition device and camera method

This application claims the priority of a Chinese patent application filed on May 31, 2019 with the application number 201910472689.0 and the invention title "an image acquisition device and imaging method", the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the field of surveillance, and in particular to an image acquisition device and a camera method.

Background technique

Image acquisition equipment can be installed in places such as buildings and roads. The image acquisition equipment photographs the place, not only to obtain the video stream, but also to perform intelligent analysis based on the captured video. The image capture device can store the captured video stream in the storage device, and perform intelligent analysis such as face recognition based on the captured video.

However, in a scene with low illumination, such as at night, intelligent analysis based on the captured video, the accuracy of the analysis result obtained is low.

Summary of the invention

The embodiments of the present application provide an image acquisition device and a camera method to improve the accuracy of intelligent analysis of images. The technical solution is as follows:

On the one hand, the present application provides an image acquisition device, including: an image sensor (01), a light supplement (02) and a filter assembly (03), the image sensor (01) is located in the filter assembly (03) ) On the light emitting side; also includes an image signal processing unit (04), a coding compression unit (05) and an analysis unit (06);

The image sensor (01) is used to generate and output a first original image signal and a second original image signal through multiple exposures, wherein the first original image signal is an image signal generated according to a first preset exposure, The second original image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two of the multiple exposures;

The light fill device (02) includes a first light fill device (021), and the first light fill device (021) is used to perform near-infrared fill light in a stroboscopic manner, wherein, at least in the first preset Performing near-infrared supplementary light during a part of the exposure time period of the exposure, and not performing near-infrared supplementary light during the exposure time period of the second preset exposure;

The filter assembly (03) includes a first filter (031), and the first filter (031) passes visible light and part of near-infrared light;

The image signal processing unit (04) is configured to obtain a gray image according to the first original image signal, and obtain a color image according to the second original image signal;

The encoding compression unit (05) is configured to compress and encode the color image;

The analysis unit (06) is used for intelligent analysis of the gray image.

On the other hand, the present application provides a camera method applied to an image acquisition device, the image acquisition device includes an image sensor (01), a light supplement (02) and a filter component (03), the image sensor ( 01) Located at the light exit side of the filter assembly (03), the light fill device (02) includes a first light fill device, the filter assembly (03) includes a first filter, and the method includes:

The image acquisition device performs near-infrared light supplementation through the first light-filling device, wherein the first light-filling device performs near-infrared light at least within a partial exposure time period of the first preset exposure of the multiple exposures of the image sensor. Infrared supplementary light, no near-infrared supplementary light is performed during the exposure time period of the second preset exposure of the multiple exposures, and the first preset exposure and the second preset exposure are two of the multiple exposures;

The image acquisition device generates a first original image signal and a second original image signal through the multiple exposures, wherein the first original image signal is an image signal generated according to the first preset exposure, and the second The original image signal is an image signal generated according to the second preset exposure;

The image acquisition device passes through the first filter to pass visible light and part of the near-infrared light;

The image acquisition device obtains a grayscale image according to the first original image signal, and obtains a color image according to the second original image signal;

The image acquisition device compresses and encodes the color image, and performs intelligent analysis on the gray image.

The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

Since the image sensor is exposed based on the near-infrared light-filling timing of the first light-filling device, the image sensor generates the first original image signal when the near-infrared light-filling is performed during the first preset exposure. The image sensor generates the second original image signal when the near-infrared supplementary light is not performed in the process. This data collection method can directly collect the first original image signal and the second original image signal with different infrared components, so that the color can be easily obtained. Image signal (second original image signal) and infrared image signal (first original image signal). Then the first original image signal is used to obtain a grayscale image, and the second original image signal is used to obtain a color image. Since the first original image signal is generated during near-infrared fill light, the grayscale image obtained based on the first original image signal The mid-near infrared brightness information is prominent, and the gray-scale image has a sharp contrast. Compared with color images in the night scene, it is beneficial to obtain better intelligent analysis results, which can improve the accuracy of intelligent analysis. Since the second original image signal is generated when near-infrared supplementary light is not performed, the RGB color information in the color image obtained based on the second original image signal is prominent, and it is more advantageous to output for viewing in a night scene than a gray image.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments that conform to the application, and are used together with the specification to explain the principle of the application.

FIG. 1 is a schematic structural diagram of an image acquisition device provided by an embodiment of the present application;

2 is a schematic structural diagram of a filter assembly provided by an embodiment of the present application;

3 is a time sequence diagram of the exposure time period of the first preset exposure and the exposure time of the second preset exposure provided by an embodiment of the present application;

4 is a schematic diagram of a rolling shutter exposure method provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a first preset exposure and a second preset exposure according to an embodiment of the present application;

6 is a schematic diagram of a second type of first preset exposure and a second preset exposure provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a third type of first preset exposure and a second preset exposure provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of the first rolling shutter exposure method and near-infrared light supplement provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a second rolling shutter exposure method and near-infrared fill light provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of a third rolling shutter exposure method and near-infrared light supplement provided by an embodiment of the present application;

11 is a schematic diagram of the wavelength band corresponding to near-infrared light provided by an embodiment of the present application;

FIG. 12 is a schematic diagram of the relationship between the wavelength and relative intensity of near-infrared supplement light performed by a first light supplement device according to an embodiment of the present application;

FIG. 13 is a schematic structural diagram of another image acquisition device provided by an embodiment of the present application;

FIG. 14 is a schematic structural diagram of an RGBW sensor provided by an embodiment of the present application;

FIG. 15 is a schematic structural diagram of an RCCB sensor provided by an embodiment of the present application;

FIG. 16 is a schematic structural diagram of an RGB sensor provided by an embodiment of the present application;

FIG. 17 is a schematic diagram of the structure of the RYYB sensor provided by an embodiment of the present application;

FIG. 18 is a schematic diagram of wavebands corresponding to each photosensitive channel provided by an embodiment of the present application;

FIG. 19 is a schematic diagram of image fusion provided by an embodiment of the present application;

FIG. 20 is a flowchart of an imaging method provided by an embodiment of the present application.

Through the above drawings, the specific embodiments of the present application have been shown, which will be described in more detail below. These drawings and text description are not intended to limit the scope of the concept of the present application in any way, but to explain the concept of the present application to those skilled in the art by referring to specific embodiments.

Detailed ways

Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

In a weak light environment, such as a night environment, the image signal captured by the image acquisition device has greater noise, and the grayscale image generated based on the image signal also has greater noise, so based on the grayscale image The accuracy of the analysis results obtained during intelligent analysis processing is low. In order to improve the accuracy of the analysis results, this application provides the following image acquisition equipment.

Referring to Fig. 1, the embodiment of the present application provides an image acquisition device, and the image acquisition device includes:

Image sensor 01, light supplement 02, and filter assembly 03. Image sensor 01 is located on the light exit side of filter assembly 03; it also includes image signal processing unit 04, encoding compression unit 05, and analysis unit 06;

The image sensor 01 is used to generate and output a first original image signal and a second original image signal through multiple exposures, where the first original image signal is an image signal generated according to the first preset exposure, and the second original image signal is According to the image signal generated by the second preset exposure, the first preset exposure and the second preset exposure are two of the multiple exposures;

The light supplement 02 includes a first light supplement device 021, which is used to perform near-infrared supplement light in a stroboscopic manner, wherein the near-infrared supplement light is performed at least during a partial exposure period of the first preset exposure , Do not perform near-infrared fill light during the exposure time period of the second preset exposure;

The filter assembly 03 includes a first filter 031, and the first filter 031 passes visible light and part of the near-infrared light;

The image signal processing unit 04 is configured to obtain a grayscale image according to the first original image signal, and obtain a color image according to the second original image signal;

The encoding compression unit 05 is used to compress and encode color images;

The analysis unit 06 is used for intelligent analysis of the gray image.

The image sensor 01 generates and outputs a first original image signal and a second original image signal through multiple exposures, the first original image signal is an image signal generated according to the first preset exposure, and the second original image signal is according to the second preset For the image signal generated by the exposure, the first preset exposure and the second preset exposure are two of the multiple exposures. The first original image signal is processed by the image signal processing unit 04 to output a color image for encoding and compressing the output video stream; the second original image signal is processed by the signal processing unit 04 for image signal processing to generate a gray image, and analyzed Unit 06 analyzes and processes grayscale images and outputs intelligent analysis results; for intelligent analysis, special fill light and image signal acquisition and optimization processing are performed, and the effect of intelligent analysis is further improved without changing the existing video effects.

The first filter 031 is used to pass visible light and part of the near-infrared light, wherein the near-infrared light passing through the first filter 031 in the exposure time period of the first preset exposure includes the first light supplement device 021 for near-infrared light. The near-infrared light reflected by the object into the filter assembly 03 during the fill light, and the near-infrared light passing through the first filter 031 within the exposure time period of the second preset exposure includes the first fill light device 021 not performing near-infrared compensation The near-infrared light that is reflected by the object and enters the filter assembly 03 when light is reflected; wherein, the first light-filling device 021 has a higher intensity of the near-infrared light that passes through the first filter 031 when performing near-infrared light-filling 021 The intensity of the near-infrared light passing through the first filter 031 when the near-infrared light is not applied.

As an example, the image acquisition device may be a video camera, a capture machine, a face recognition camera, a code reading camera, a vehicle-mounted camera, a panoramic detail camera, etc.

As another example, the light supplement 02 may be located in the image acquisition device or outside the image acquisition device. The light supplement 02 can be a part of the image acquisition device, or can be a device independent of the image acquisition device. When the light fill 02 is located outside the image capture, the light fill 02 can communicate with the image capture device, which can ensure the exposure timing of the image sensor 01 in the image capture device and the first fill light device included in the light fill 02 The timing of the near-infrared supplement light of 021 has a certain relationship. For example, the near-infrared supplementary light is performed at least during a partial exposure period of the first preset exposure, and the near-infrared supplementary light is not performed during the exposure period of the second preset exposure.

In addition, it should be noted that the first supplementary light device 021 is a device that can emit near-infrared light, such as a near-infrared supplementary light, which is not limited in the embodiment of the present application. The first light supplement device 021 can perform near-infrared supplement light in a stroboscopic manner, or in other ways similar to stroboscopic. In some examples, when the first light supplement device 021 performs near-infrared supplement light in a stroboscopic manner, the first supplement light device 021 may be manually controlled to perform near-infrared supplement light in a stroboscopic manner, or through a software program Or a specific device controls the first light supplement device 021 to perform near-infrared supplement light in a strobe mode, which is not limited in the embodiment of the present application. The time period during which the first light supplement device 021 performs near-infrared light supplementation may coincide with the exposure time period of the first preset exposure, or may be greater than the exposure time period of the first preset exposure or less than the exposure time period of the first preset exposure. The time period, as long as the near-infrared supplement light is performed during the entire exposure time period or part of the exposure time period of the first preset exposure, and the near-infrared supplement light is not performed during the exposure time period of the second preset exposure.

Referring to FIG. 3, the exposure time period of the first preset exposure and the exposure time period of the second preset exposure appear alternately, and the first light supplement device 021 generates and emits near-infrared light during the exposure time period of the first preset exposure. , To realize near-infrared supplementary light, stop generating near-infrared light during the exposure time period of the second preset exposure, so as to realize no near-infrared supplementary light. The so-called stroboscopic approach for near-infrared supplementary light is to generate and emit near-infrared light during the exposure time period of the first predicted exposure, and stop during the exposure time period of the second preset exposure adjacent to the exposure time period of the first predicted exposure Produce near-infrared light.

Referring to FIG. 3, as an example, the multiple exposures may include odd exposures and even exposures, the first preset exposure may be one of the even exposures, and the second preset exposure may be the odd exposures. Of an exposure.

As an example, the first preset exposure may be one exposure in an odd number of exposures, and the second preset exposure may be one exposure in an even number of exposures; or

As an example, the first preset exposure may be one of the specified odd exposures, and the second preset exposure may be one of the exposures except the specified odd exposures; or

As an example, the first preset exposure may be one exposure in a specified even number of exposures, and the second preset exposure may be one exposure in other exposures except the specified even number of exposures.

As an example, the number of times of supplementary light in the unit time length of the first supplementary light device 021 may be lower than the number of exposures of the image sensor 01 in the unit time length, wherein the supplementary light times of every two adjacent times of supplementary light , Interval one or more exposures.

As an example, the first preset exposure is one exposure in the first exposure sequence, and the second preset exposure is one exposure in the second exposure sequence; or

As an example, the first preset exposure is one exposure in the second exposure sequence, and the second preset exposure is one exposure in the first exposure sequence;

Wherein, the multiple exposure includes multiple exposure sequences, the first exposure sequence and the second exposure sequence are one exposure sequence or two exposure sequences among the multiple exposure sequences, each exposure sequence includes N exposures, and N exposures include 1 first preset exposure and N-1 second preset exposures, or N exposures include 1 second preset exposure and N-1 second preset exposures, and N is a positive integer greater than 2.

As an example, the image sensor 01 may adopt a global exposure method or a rolling shutter exposure method. The global exposure mode means that the exposure start time of each row of effective pixels is the same, and the exposure end time of each row of effective pixels is the same. In other words, the global exposure mode is an exposure mode in which all rows of effective pixels are exposed at the same time and the exposure ends at the same time. Rolling shutter exposure means that the exposure times of different rows of effective pixels do not completely coincide, that is, the exposure start time of a row of effective pixels is later than the exposure start time of the previous row of effective pixels, and the exposure end time of a row of effective pixels is later At the end of the exposure of the effective pixels in the previous row. In addition, in the rolling exposure mode, data can be output after each row of effective pixels is exposed. Therefore, the time from the time when the data of the first row of effective pixels starts to the time when the data of the last row of effective pixels ends can be expressed as reading Time out.

For example, refer to FIG. 4, which is a schematic diagram of a rolling shutter exposure method. It can be seen from Figure 4 that the effective image of the first line begins to be exposed at time T1, and the exposure ends at time T3. The effective image of the second line begins to be exposed at time T2, and the exposure ends at time T4. Time T2 is backward compared to time T1. A period of time has passed, and time T4 has moved a period of time backward compared to time T3. In addition, the effective image of the first line ends exposure at time T3 and begins to output data, and the output of data ends at time T5. The effective image of line n ends exposure at time T6 and begins to output data, and the output of data ends at time T7, then T3 The time between ～T7 is the read time.

In the case of image sensor 01 using global exposure for multiple exposures, for any one near-infrared fill light, the time period of near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure. The time period of fill light is a subset of the exposure time period of the first preset exposure, or the time period of near-infrared fill light and the exposure time period of the first preset exposure overlap, or the time period of the first preset exposure The exposure time period is a subset of the near-infrared fill light. In this way, it can be realized that the near-infrared supplementary light is performed at least during a part of the exposure time period of the first preset exposure, and the near-infrared supplementary light is not performed during the entire exposure time period of the second preset exposure. Set the exposure to affect.

For example, referring to Figure 5, for any one time of near-infrared fill light, the time period of near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure, and the time period of near-infrared fill light is the first preset A subset of the exposure time period for exposure. Referring to Figure 6, for any one near-infrared fill light, the time period of near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure, and the time period of near-infrared fill light is equal to that of the first preset exposure. There is an intersection of exposure time periods. Referring to Figure 7, for any one near-infrared fill light, the time period of near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure, and the exposure time period of the first preset exposure is near-infrared fill light A subset of.

In other embodiments, the image sensor 01 adopts rolling shutter exposure for multiple exposures. For any one near-infrared fill light, the period of near-infrared fill light is the same as the exposure time of the nearest second preset exposure. There is no intersection between segments; the start time of the near-infrared fill light is no earlier than the exposure start time of the last line of the effective image in the first preset exposure, and the end time of the near-infrared fill light is no later than the first line of the first preset exposure. The exposure end time of the image; or, the start time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image of the nearest second preset exposure before the first preset exposure and no later than the first preset exposure The exposure end time of the first line of the effective image in the exposure, the end time of the near-infrared fill light is no earlier than the exposure start time of the last line of the effective image in the first preset exposure and no later than the nearest neighbor after the first preset exposure The exposure start time of the first line of the effective image of the second preset exposure; or the start time of the near-infrared fill light is no earlier than the end of the exposure of the last line of the effective image of the nearest second preset exposure before the first preset exposure Time and no later than the exposure start time of the first line of the effective image in the first preset exposure, and the end time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image in the first preset exposure and no later than the first exposure The exposure start time of the first line of the effective image of the nearest second preset exposure after a preset exposure.

For example, referring to Figure 8, for any one near-infrared fill light, the time period of near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure, and the start time of near-infrared fill light is no earlier than The exposure start time of the last line of the effective image in the first preset exposure, and the end time of the near-infrared fill light is no later than the exposure end time of the first line of the effective image in the first preset exposure. Referring to Figure 9, for any one time of near-infrared fill light, the time period of near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure, and the start time of near-infrared fill light is no earlier than the first The exposure end time of the last effective image line of the nearest second preset exposure before the preset exposure and not later than the exposure end time of the first effective image line in the first preset exposure. The end time of the near-infrared fill light is not It is earlier than the exposure start time of the last line of the effective image in the first preset exposure and not later than the exposure start time of the first line of the effective image of the nearest second preset exposure after the first preset exposure. Referring to FIG. 10, for any one time of near-infrared supplement light, the time period of near-infrared supplement light does not overlap with the exposure time period of the nearest second preset exposure, and the start time of near-infrared supplement light is not earlier than the first The exposure end time of the last line of the effective image of the nearest second preset exposure before the preset exposure and not later than the exposure start time of the first line of the effective image in the first preset exposure, the end time of the near-infrared fill light is not It is earlier than the exposure end time of the last line of the effective image in the first preset exposure and not later than the exposure start time of the first line of the effective image of the nearest second preset exposure after the first preset exposure. 8 to 10 are only an example, and the sorting of the first preset exposure and the second preset exposure may not be limited to these examples.

Referring to FIG. 3, during the exposure time period of the first preset exposure, the first light supplement device 021 generates and emits near-infrared light to the external environment, so the external environment includes the environment during the exposure time period of the first preset exposure Light and the near-infrared light generated by the first light-filling device 021. The ambient light includes visible light, and may also include at least one of near-infrared light and infrared light. In this way, the light is blocked during the exposure time period of the first preset exposure. The light reflected by the object into the first filter 031 includes the near-infrared light and the ambient light generated by the first light-filling device 021, and the first filter 031 passes the near-infrared light and the visible light and near-infrared of the ambient light. Light and transmitted to the image sensor 01. The image sensor 01 performs shooting according to the exposure parameters of the first preset exposure, and can sense the visible light and the near-infrared light in the ambient light and the near-infrared light generated by the first light supplement device 021 to obtain the first original image signal. During the exposure time period of the second preset exposure, the first light supplement device 021 stops generating near-infrared light, so during the exposure time period of the second preset exposure, the external environment includes ambient light, which is reflected by the object into the first filter The light of the light sheet 031 includes the ambient light, and the first filter 031 passes visible light and near-infrared light in the ambient light and transmits it to the image sensor 01, and the image sensor 01 shoots according to the exposure parameters of the second preset exposure. The visible light and near-infrared light in the ambient light in the external environment can be sensed to obtain the second original image signal.

Another point that needs to be explained is that, when the first light supplement device 021 performs near-infrared light supplementation on an external scene, the near-infrared light incident on the surface of the object may be reflected by the object and enter the first filter 031. And because under normal circumstances, the ambient light may include visible light and near-infrared light, and near-infrared light in the ambient light is also reflected by the object when it is incident on the surface of the object, thereby entering the first filter 031. Therefore, the near-infrared light that passes through the first filter 031 when performing near-infrared light supplementation may include the near-infrared light that is reflected by the object and enters the first filter 031 when the first light supplement device 021 performs near-infrared light supplementation. The near-infrared light passing through the first filter 031 when the near-infrared light supplement is not performed may include the near-infrared light reflected by the object into the first filter 031 when the first light supplement device 021 is not performing the near-infrared light supplement. That is, the near-infrared light that passes through the first filter 031 when performing near-infrared supplementary light includes the near-infrared light emitted by the first supplementary light device 021 and reflected by the object, and the ambient light reflected by the object Near-infrared light, the near-infrared light passing through the first filter 031 when the near-infrared supplementary light is not performed includes near-infrared light reflected by an object in the ambient light. In the image acquisition device, the filter assembly 03 may be located between the lens 07 and the image sensor 01, and the image sensor 01 is located on the light-emitting side of the filter assembly 03 as an example. The image acquisition device acquires the first original image signal and the first original image signal. The second process of the original image signal is: when the image sensor 01 performs the first preset exposure, the first light supplement device 021 performs near-infrared supplement light, and at this time the ambient light in the shooting scene and the first light supplement device 021 perform near-infrared light After the near-infrared light reflected by objects in the scene passes through the lens 07 and the first filter 031 when filling light, the image sensor 01 generates the first original image signal through the first preset exposure; the second preset is performed on the image sensor 01 During exposure, the first light-filling device 021 does not perform near-infrared light-filling. At this time, after the ambient light in the shooting scene passes through the lens 07 and the first filter 031, the image sensor 01 generates the second original light through the second preset exposure. For image signals, there can be M first preset exposures and N second preset exposures in one frame period of image acquisition, and there can be multiple combinations of sorts between the first preset exposure and the second preset exposure. In a frame period of image acquisition, the values of M and N and the magnitude relationship between M and N can be set according to actual requirements. For example, the values of M and N may be equal or different.

In addition, since the intensity of the near-infrared light in the ambient light is lower than the intensity of the near-infrared light emitted by the first light supplement device 021, the first light supplement device 021 passes through the first filter 031 when performing near-infrared supplement light. The intensity of the near-infrared light is higher than the intensity of the near-infrared light passing through the first filter 031 when the first light supplement device 021 is not performing near-infrared light supplementation.

Based on the above description, the first light-filling device 021 performs near-infrared light-filling during the partial exposure time period of the first preset exposure, and passes the near-infrared light of the first filter 031 during the exposure time period of the first preset exposure The light may include the near-infrared light reflected by the object into the first filter 031 when the first light supplement device 021 performs near-infrared light supplement light, and the first filter 031 can also make visible light and near-infrared light in the ambient light by. However, since the first light supplement device 021 performs near-infrared light supplementation, the intensity of the near-infrared light reflected by the object and entering the first filter 031 is relatively strong, while the intensity of visible light is relatively weak, the near-infrared light sensed by the image sensor 01 The intensity is strong. Therefore, the first original image signal generated and output according to the first preset exposure mainly includes near-infrared light brightness information. In addition, based on the above description, the first light supplement device 021 does not perform near-infrared supplement light during the entire exposure time period of the second preset exposure, and passes through the first filter 031 during the exposure time period of the second preset exposure. The near-infrared light of is the near-infrared light in the ambient light. Since there is no near-infrared light provided by the first light-filling device 021, the intensity of the near-infrared light that passes through the first filter 031 is weak at this time, while the intensity of visible light is strong, and the visible light that the image sensor 01 can sense The intensity is strong. Therefore, the second original image signal generated and output according to the second preset exposure mainly includes visible light brightness information.

The wavelength range of the first light supplement device 021 for near-infrared supplement light may be the second reference wavelength range, which may be 700 nanometers to 800 nanometers, or 900 nanometers to 1000 nanometers, etc. This is not the case in this application. Make a limit. In addition, the wavelength range of the near-infrared light incident on the first filter 031 may be the first reference wavelength range, and the first reference wavelength range is 650 nm to 1100 nm. As an example, the first reference wavelength range may be 650 nm. Nano ~ 1000 nm. Among them, it should be noted that: the second reference waveband range may be 700 nm to 800 nm, which means that the second reference waveband range is greater than or equal to 700 nm and less than or equal to 800 nm, or is greater than 700 nm and less than 800 nm. The range is a range greater than or equal to 700 nanometers and less than 800 nanometers, or a range greater than 700 nanometers and less than or equal to 800 nanometers. For similar scopes appearing elsewhere in this article, the meanings are the same, so we will not explain them one by one.

Since in the exposure time period of the first preset exposure, the near-infrared light passing through the first filter 031 may include the first light-filling device 021 that is reflected by the object and enters the first filter 031 when the near-infrared light is filled. Near-infrared light, therefore, the center wavelength and/or band width of the near-infrared light passing through the first filter 031 can be matched with the center wavelength of the near-infrared light supplemented by the first light supplement device 021, that is, the first light supplement When the center wavelength of the near-infrared complementary light performed by the optical device 021 is the set characteristic wavelength or falls within the set characteristic wavelength range, the center wavelength and/or the band width of the near-infrared light passing through the first filter 031 meet the constraint conditions.

There are multiple choices for the center wavelength and/or wavelength range of the first light supplement device 021 for near-infrared supplement light. In the embodiment of the present application, in order to make the first light supplement device 021 and the first filter 031 better With cooperation, the center wavelength of the near-infrared supplement light of the first light supplement device 021 can be designed, and the characteristics of the first filter 031 can be selected, so that the center of the first light supplement device 021 for the near-infrared light supplement When the wavelength is the set characteristic wavelength or falls within the set characteristic wavelength range, the center wavelength and/or band width of the near-infrared light passing through the first filter 031 can meet the constraint conditions. This constraint is mainly used to restrict the center wavelength of the near-infrared light passing through the first filter 031 as accurately as possible, and the band width of the near-infrared light passing through the first filter 031 as narrow as possible, so as to avoid The infrared light band width is too wide and introduces wavelength interference.

Wherein, the center wavelength of the near-infrared light supplemented by the first light-filling device 021 may be the average value in the wavelength range of the highest energy in the spectrum of the near-infrared light emitted by the first light-filling device 021, or it may be understood as the first light supplement The wavelength of the near-infrared light emitted by the device 021 at the middle position in the wavelength range where the energy exceeds a certain threshold.

Among them, the set characteristic wavelength or the set characteristic wavelength range can be preset. As an example, the center wavelength of the first light supplement device 021 for near-infrared supplement light may be any wavelength within the wavelength range of 750±10 nanometers; or, the center wavelength of the first light supplement device 021 for near-infrared supplement light It is any wavelength within the wavelength range of 780±10 nanometers; or, the center wavelength of the first light supplement device 021 for near-infrared supplement light is any wavelength within the wavelength range of 940±10 nanometers. That is, the set characteristic wavelength range may be a wavelength range of 750±10 nanometers, or a wavelength range of 780±10 nanometers, or a wavelength range of 940±10 nanometers. Exemplarily, the center wavelength of the near-infrared supplement light performed by the first light supplement device 021 is 940 nanometers, and the relationship between the wavelength and the relative intensity of the near-infrared supplement light performed by the first light supplement device 021 is shown in FIG. 11. It can be seen from FIG. 11 that the wavelength range of the first light supplement device 021 for near-infrared supplement light is 900 nanometers to 1000 nanometers, and between 940 nanometers and 960 nanometers, the relative intensity of near-infrared light is the highest.

Since in the exposure time period of the first preset exposure, most of the near-infrared light passing through the first filter 031 is the near-infrared light of the first light supplement device 021 when it is reflected by the object and enters the near-infrared light of the first filter 031. Infrared light, therefore, in some embodiments, the above constraint conditions may include: the difference between the center wavelength of the near-infrared light passing through the first filter 031 and the center wavelength of the near-infrared light that the first light supplement device 021 performs The value lies in the wavelength fluctuation range. As an example, the wavelength fluctuation range may be 0-20 nanometers.

Among them, the center wavelength of the near-infrared supplement light passing through the first filter 031 can be the wavelength at the peak position in the near-infrared band in the near-infrared light pass rate curve of the first filter 031, or it can be understood as the first A filter 031 is the wavelength at the middle position in the near-infrared waveband whose pass rate exceeds a certain threshold in the near-infrared light pass rate curve.

In order to avoid the introduction of wavelength interference due to the excessively wide band width of the near-infrared light passing through the first filter 031, in some embodiments, the above constraint conditions may include: the first band width may be smaller than the second band width. The first waveband width refers to the waveband width of the near-infrared light passing through the first filter 031, and the second waveband width refers to the waveband width of the near-infrared light blocked by the first filter 031. It should be understood that the wavelength band width refers to the width of the wavelength range in which the wavelength of light lies. For example, if the wavelength of the near-infrared light passing through the first filter 031 is in the wavelength range of 700 nanometers to 800 nanometers, then the first wavelength band width is 800 nanometers minus 700 nanometers, that is, 100 nanometers. In other words, the wavelength band width of the near-infrared light passing through the first filter 031 is smaller than the wavelength band width of the near-infrared light blocked by the first filter 031.

It can be seen from the above description that the second original image signal generated and output by the image sensor 01 during the exposure time period of the second preset exposure mainly includes visible light brightness information. However, due to the exposure time period of the second preset exposure, the first filter 031 can not only pass the visible light in the ambient light, but also can make the ambient light reflected by the object enter the near infrared of the first filter 031 The light passes. Therefore, in order to prevent the second original image signal generated and output by the image sensor 01 from containing too much near-infrared light brightness information and to make the quality of the second original image signal higher, it is necessary to reduce the effect of the first filter 031 on visible light. For this reason, the first waveband width of the first filter 031 can be smaller than the second waveband width of the second filter 031. The first waveband width refers to the waveband width of the near-infrared light passing through the first filter 031, and the second waveband width refers to the waveband width of the near-infrared light blocked by the first filter 031. It should be understood that the wavelength band width refers to the width of the wavelength range in which the wavelength of light lies. For example, if the wavelength of the near-infrared light passing through the first filter 031 is in the wavelength range of 700 nanometers to 800 nanometers, the first wavelength band width is 800 nanometers minus 700 nanometers, that is, 100 nanometers. In other words, the wavelength band width of the near-infrared light passing through the first filter 031 is smaller than the wavelength band width of the near-infrared light blocked by the first filter 031. In this way, during the exposure time period of the second preset exposure, the intensity of the near-infrared light passing through the first filter 031 is relatively weak, and the intensity of the visible light passing through is relatively strong, and the near-infrared light that the image sensor 01 can sense The intensity of the light is relatively weak, so that the generated and output second original image signal does not contain too much near-infrared light brightness information, thereby making the quality of the second original image signal higher.

For example, referring to FIG. 12, FIG. 12 is a schematic diagram of the relationship between the wavelength of the light that can pass through the first filter 031 and the pass rate. The wavelength range of the near-infrared light incident on the first filter 031 ranges from 650 nanometers to 1100 nanometers. The first filter 031 can pass visible light with a wavelength between 380 nanometers and 650 nanometers, as well as those with a wavelength between 900 nanometers and 1000 nanometers. Near-infrared light passes through and blocks near-infrared light with a wavelength between 650 nanometers and 900 nanometers. That is, the width of the first band is 1000 nanometers minus 900 nanometers, that is, 100 nanometers. The second band width is 900 nm minus 650 nm, plus 1100 nm minus 1000 nm, or 350 nm. 100 nanometers are smaller than 350 nanometers, that is, the wavelength band width of the near-infrared light passing through the first filter 031 is smaller than the wavelength band width of the near-infrared light blocked by the first filter 031. The above relationship curve is just an example. For different filters, the wavelength range of the near-red light that can pass through the filter can be different, and the wavelength range of the near-infrared light blocked by the filter can also be different. different.

In order to prevent the second original image signal generated and output by the image sensor 01 from containing too much near-infrared light brightness information, and to make the quality of the second original image signal higher, the constraints include: the first filter 031 The half-bandwidth of near-infrared light is less than or equal to 50 nanometers. Among them, the half bandwidth refers to the band width of near-infrared light with a pass rate greater than 50%.

In order to prevent the second original image signal generated and output by the image sensor 01 from containing too much near-infrared light brightness information and to make the quality of the second original image signal higher, the constraint condition is: the first filter 031 The three-band width can be smaller than the reference band width. The third waveband width refers to the waveband width of near-infrared light with a pass rate greater than a set ratio. As an example, the reference waveband width may be any waveband width in the range of 50 nm to 150 nm. The set ratio can be any ratio from 30% to 50%. Of course, the set ratio can also be set to other ratios according to usage requirements, which is not limited in the embodiment of the present application. In other words, the band width of the near-infrared light whose pass rate is greater than the set ratio can be smaller than the reference band width. In this way, during the exposure period of the second preset exposure, the intensity of the near-infrared light that the image sensor 01 can sense is relatively weak, so that the second original image signal generated and output will not contain too much near-infrared light. The brightness information in turn makes the quality of the second original image signal higher.

For example, referring to FIG. 12, the wavelength band of the near-infrared light incident on the first filter 031 is 650 nanometers to 1100 nanometers, the setting ratio is 30%, and the reference wavelength band width is 100 nanometers. It can be seen from FIG. 12 that in the wavelength band of near-infrared light from 650 nanometers to 1100 nanometers, the band width of near-infrared light with a pass rate greater than 30% is significantly less than 100 nanometers.

Since the first light supplement device 021 provides near-infrared supplementary light at least during a partial exposure period of the first preset exposure, it does not provide near-infrared supplementary light during the entire exposure period of the second preset exposure, and the first preset exposure The exposure and the second preset exposure are two of the multiple exposures of the image sensor 01, that is, the first light supplement device 021 provides near-infrared supplement light during the exposure period of the partial exposure of the image sensor 01, The near-infrared supplementary light is not provided during the exposure time period when another part of the image sensor 01 is exposed. Therefore, the number of times of supplementary light in the unit time length of the first supplementary light device 021 may be lower than the number of exposures of the image sensor 01 in the unit time length, wherein, within the interval of two adjacent times of supplementary light, there is one interval. Or multiple exposures. In this way, the number of light supplements of the first light supplement device 021 can be reduced, thereby prolonging the service life of the first light supplement device 021.

It should be noted that, since the human eye can easily confuse the color of the near-infrared light of the first light supplement device 021 with the color of the red light in the traffic light, referring to FIG. 13, the light supplement 02 may also include a second The light supplement device 022, and the second light supplement device 022 is used to supplement light with visible light. In this way, if the second light supplement device 022 provides visible light supplement light at least during part of the exposure time of the first preset exposure, that is, it performs near-infrared supplement light and visible light supplement light at least during the partial exposure time period of the first preset exposure. Light, the mixed color of the two lights can be distinguished from the color of the red light in the traffic light, so as to avoid the human eye from confusing the color of the light fill 02 for near-infrared fill light with the color of the red light in the traffic light. In addition, if the second light supplement device 022 provides visible light supplement light during the exposure time period of the second preset exposure, since the intensity of visible light is not particularly high during the exposure time period of the second preset exposure, When the visible light supplement is performed during the exposure time period of the exposure, the brightness of the visible light in the second original image signal can also be increased, thereby ensuring the quality of image collection.

In some embodiments, the second light supplement device 022 may be used to perform visible light supplement light in a constant light mode, wherein the visible light supplement light is performed at least during a part of the exposure time period of the first preset exposure, or it may be performed during the second preset exposure. It is assumed that the visible light fill light is performed during the entire exposure time period of the exposure. When the second light supplement device 022 performs visible light supplement light in a constant light mode, it can not only prevent human eyes from confusing the color of the first light supplement device 021 for near-infrared supplement light with the color of the red light in the traffic light, but also can improve the 2. The brightness of the visible light in the original image signal to ensure the quality of image acquisition. Alternatively, the second light supplement device 022 can be used to perform visible light supplement light in a stroboscopic manner, where visible light supplement light is not performed at least during the entire exposure time period of the first preset exposure, and the second preset exposure time Visible light fill light in the segment. Alternatively, the second light supplement device 022 is used to perform visible light supplement light in a stroboscopic manner, wherein the visible light supplement light is performed at least during a partial exposure time period of the first preset exposure, and the visible light supplement light is performed during the entire exposure time period of the second preset exposure. There is no visible light fill light inside.

When the second light supplement device 022 performs visible light supplement light in a stroboscopic manner, it can prevent human eyes from confusing the color of the first light supplement device 021 for near-infrared supplement light with the color of the red light in the traffic light, or can improve The brightness of the visible light in the second original image signal in turn ensures the quality of image acquisition, and can also reduce the number of times of supplementary light of the second supplementary light device 022, thereby prolonging the service life of the second supplementary light device 022.

As an example, for the aforementioned lens 07, the lens 07 is used to focus the light reflected by the object;

The first filter 031 in the filter assembly 03 is used to filter out the near-infrared light generated by the first light supplement device 021 and the visible light in the ambient light from the focused light within the exposure time period of the first preset exposure And near-infrared light, the visible light and near-infrared light in the ambient light are filtered out from the focused light within the exposure time period of the second preset exposure.

Referring to FIG. 2, the filter assembly 03 includes a first filter 031, a second filter 032 and a switching component 033; both the first filter 031 and the second filter 032 are connected to the switching component 033.

The first filter 031 is used to pass light in the visible light waveband and the near-infrared light waveband, and the light in the near-infrared light waveband is the light in the first reference filter range;

The second filter 032 is used to pass light in the visible light band;

In an external environment with strong light, the first light supplement device 021 can stop performing near-infrared light supplement light. For example, in the daytime when the light is strong, the first light supplement device 021 may stop performing near-infrared light supplement light. In an external environment with weak light, the first light supplement device 021 performs near-infrared supplement light in a strobe mode. For example, when the light is weak at night, the first supplementary light device 021 performs near-infrared supplementary light in a strobe mode.

Therefore, the switching component 033 is used to switch the first filter 031 to the light incident side of the image sensor 01 in an external environment with weak light intensity, and to switch the second filter 032 in an external environment with strong light intensity The light incident side of the image sensor 01.

After the second filter 032 is switched to the light incident side of the image sensor 01, the second filter 032 allows light in the visible light band to pass and blocks light in the near-infrared light band. The image sensor 01 is used to generate and output through exposure The third image signal.

As an example, referring to FIGS. 1 and 2, the device may further include a lens 07. In this case, the filter assembly 03 may be located between the lens 07 and the image sensor 01, and the image sensor 01 is located on the light exit side of the filter assembly 03. Alternatively, the lens 07 is located between the filter assembly 03 and the image sensor 01, and the image sensor 01 is located on the light exit side of the lens 07. As an example, the first filter 031 can be a filter film. In this way, when the filter assembly 03 is located between the lens 07 and the image sensor 01, the first filter 031 can be attached to the light emitting side of the lens 07 The surface, or, when the lens 07 is located between the filter assembly 03 and the image sensor 01, the first filter 031 may be attached to the surface of the lens 07 on the light incident side.

The switching component 03 switches the second filter 032 to the light incident side of the image sensor 01 or switches the first filter 031 to the light incident side of the image sensor 01; in the second filter 032, it switches to the light incident side of the image sensor. After the light side, the first supplementary light device 021 is in the closed state; the second filter 032 is used to block the light in the near-infrared light band and pass the light in the visible light band; the image sensor 01 is used to generate and output the third light through exposure Original image signal.

It should be noted that the switching component 033 is used to switch the second filter 032 to the light incident side of the image sensor 01, and can also be understood as the second filter 032 replacing the first filter 031 in the image sensor 01. Position on the light side. After the second light filter 032 is switched to the light incident side of the image sensor 01, the first light supplement device 021 may be in the off state or in the on state.

An external environment with strong light intensity may mean that the light intensity of the external environment is greater than or equal to a preset intensity threshold, and an external environment with weak light intensity may mean that the light intensity of the external environment is less than the preset intensity threshold.

As an example, the switching component 033 switches in an external environment with weak light intensity (such as night). The first filter 031 is located between the lens 07 and the image sensor 01, and the light transmission path reflected by the object passes through the lens 07 and the first filter. Filter 031 and image sensor 01. Since the first filter 031 can pass the visible light and the near-infrared light in the ambient light from the light, and the near-infrared light generated by the first light-filling device 021 reflected by the object, it is within the exposure time period of the first preset exposure The first filter 031 can pass the visible light and near-infrared light reflected by the object in the ambient light from the light focused by the lens 07, and the near-infrared light generated by the first light-filling device 021 reflected by the object, and will pass through Visible light and near-infrared light are transmitted to the image sensor 01. In the exposure time period of the second preset exposure, the first filter 031 can pass the visible light and near-infrared light reflected by the object in the ambient light from the light focused by the lens 07, and filter out the visible light and the near-infrared light in the ambient light. The near infrared light is transmitted to the image sensor 01.

As an example, the switching component 033 switches in an external environment with strong light intensity (such as daytime). The second filter 032 is located between the lens 07 and the image sensor 01, and the light transmission path reflected by the object passes through the lens 07 and the first filter. Two filters 032 and image sensor 01. Since the second filter 032 can pass visible light from the light, the second filter 032 can pass the visible light from the light focused by the lens 07 and transmit the passed visible light to the image sensor 01.

In some embodiments, the multiple exposure performed by the image sensor refers to multiple exposures in one frame period, that is, the image sensor 01 performs multiple exposures in one frame period, thereby generating and outputting at least one frame of the first original The image signal and at least one frame of the second original image signal. For example, 1 second includes 25 frame periods, and the image sensor 01 performs multiple exposures in each frame period, thereby generating at least one frame of the first original image signal and at least one frame of the second original image signal. The generated first original image signal and second original image signal are called a group of image signals, so that 25 groups of image signals are generated within 25 frame periods. Among them, the first preset exposure and the second preset exposure can be two adjacent exposures in multiple exposures in one frame period, or two non-adjacent exposures in multiple exposures in one frame period. The application embodiment does not limit this.

The first original image signal is generated and output by the first preset exposure, and the second original image signal is generated and output by the second preset exposure. After the first original image signal and the second original image signal are generated and output, The first original image signal and the second original image signal are processed. In some cases, the purposes of the first original image signal and the second original image signal may be different, so in some embodiments, at least one exposure parameter of the first preset exposure and the second preset exposure may be different. As an example, the at least one exposure parameter may include but is not limited to one or more of exposure time, analog gain, digital gain, and aperture size. Wherein, the exposure gain includes analog gain and/or digital gain.

In some embodiments. It can be understood that, compared with the second preset exposure, the intensity of the near-infrared light sensed by the image sensor 01 is stronger when the near-infrared light is supplemented, and the first original image signal generated and output accordingly includes the near-infrared light. The brightness of infrared light will also be higher. However, near-infrared light with higher brightness is not conducive to the acquisition of external scene information. Moreover, in some embodiments, the greater the exposure gain, the higher the brightness of the image signal output by the image sensor 01, and the smaller the exposure gain, the lower the brightness of the image signal output by the image sensor 01. Therefore, in order to ensure the first original image signal The brightness of the included near-infrared light is within a suitable range. In the case where at least one exposure parameter of the first preset exposure and the second preset exposure are different, as an example, the exposure gain of the first preset exposure may be less than Exposure gain for the second preset exposure. In this way, when the first light supplement device 021 performs near-infrared supplement light, the brightness of the near-infrared light contained in the first original image signal generated and output by the image sensor 01 will not be caused by the first light supplement device 021. And too high.

In other embodiments, the longer the exposure time, the higher the brightness included in the image signal obtained by the image sensor 01, and the longer the motion trailing of the moving objects in the external scene in the image signal; the shorter the exposure time, the longer the image The image signal obtained by the sensor 01 includes the lower the brightness, and the shorter the motion trail of the moving object in the external scene is in the image signal. Therefore, in order to ensure that the brightness of the near-infrared light contained in the first original image signal is within a proper range, and the moving objects in the external scene have a shorter motion trail in the first original image signal. In the case where at least one exposure parameter of the first preset exposure and the second preset exposure are different, as an example, the exposure time of the first preset exposure may be less than the exposure time of the second preset exposure. In this way, when the first light supplement device 021 performs near-infrared supplement light, the brightness of the near-infrared light contained in the first original image signal generated and output by the image sensor 01 will not be caused by the first light supplement device 021. And too high. In addition, the shorter exposure time makes the motion trailing of the moving object in the external scene appear shorter in the first original image signal, thereby facilitating the recognition of the moving object. Exemplarily, the exposure time of the first preset exposure is 40 milliseconds, the exposure time of the second preset exposure is 60 milliseconds, and so on.

It is worth noting that, in some embodiments, when the exposure gain of the first preset exposure is less than the exposure gain of the second preset exposure, the exposure time of the first preset exposure may not only be less than the exposure time of the second preset exposure , Can also be equal to the exposure time of the second preset exposure. Similarly, when the exposure time of the first preset exposure is less than the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be less than the exposure gain of the second preset exposure, or may be equal to the second preset exposure The exposure gain.

In other embodiments, the purpose of the first original image signal and the second original image signal may be the same. For example, when both the first original image signal and the second original image signal are used for intelligent analysis, in order to enable the person performing the intelligent analysis The face or target can have the same sharpness when moving, and at least one exposure parameter of the first preset exposure and the second preset exposure can be the same. As an example, the exposure time of the first preset exposure may be equal to the exposure time of the second preset exposure. If the exposure time of the first preset exposure and the exposure time of the second preset exposure are different, the exposure time will be longer. There is a motion trailing in the image signal of one channel, resulting in different definitions of the two image signals. Similarly, as another example, the exposure gain of the first preset exposure may be equal to the exposure gain of the second preset exposure.

It is worth noting that, in some embodiments, when the exposure time of the first preset exposure is equal to the exposure time of the second preset exposure, the exposure gain of the first preset exposure may be less than the exposure gain of the second preset exposure. It can also be equal to the exposure gain of the second preset exposure. Similarly, when the exposure gain of the first preset exposure is equal to the exposure gain of the second preset exposure, the exposure time of the first preset exposure may be less than the exposure time of the second preset exposure, or may be equal to the second preset exposure The exposure time.

As an example, another implementation manner for generating the first original image signal and the second original image signal is also provided in the embodiment of the present application, and the other implementation manner is:

In an external environment with weak light intensity (such as at night), the first light supplement device 021 is used to continuously generate and emit near-infrared light. The switching component 033 drives the first filter 031 between the lens 07 and the image sensor 01 within the exposure time period of the first preset exposure. The lens 07 focuses the light reflected by the object, and the first filter 031 focuses The light passes through the visible light and near-infrared light in the ambient light and the near-infrared light generated by the first light-filling device 021 reflected by the object, and the image sensor 01 passes the first filter 031 according to the exposure parameters of the first preset exposure The visible light and the near-infrared light are sensed to obtain the first original image signal. And the switching component 033 drives the second filter 032 between the lens 07 and the image sensor 01 within the exposure time period of the second preset exposure, the lens 07 focuses the light reflected by the object, and the second filter 032 focuses The visible light is filtered out of the light, and the image sensor 01 senses the visible light according to the exposure parameter of the second preset exposure to obtain the second original image signal.

Since in the exposure time period of the second preset exposure, the second filter 032 only allows visible light to pass and blocks near-visible light, so that the second original image signal obtained by the image sensor 01 sensing the visible light will not be overexposed This phenomenon prevents the color image obtained based on the second original image signal from being whitened.

In an external environment with strong light intensity (such as during the day), the switching component 033 drives the second filter 032 to be located between the lens 07 and the image sensor 01.

For the image sensor 01 described above, the image sensor 01 includes a plurality of photosensitive channels, and the plurality of photosensitive channels are used to sense at least two different visible light wavelength bands. Each light-sensitive channel is used to sense at least one color of light in the visible light band and light in the near-infrared band, and the at least one type of visible light includes red light, green light, blue light, yellow light, and the like.

As an example, each photosensitive channel corresponds to a visible light wavelength range of one color, that is, each photosensitive channel is used to sense visible light and near-infrared light in the visible wavelength range of a corresponding color. For example, the multiple photosensitive channels include at least two of the R photosensitive channel, G photosensitive channel, B photosensitive channel, Y photosensitive channel, W photosensitive channel, and C photosensitive channel; wherein, the R photosensitive channel is used to sense the red light band and near Infrared light, G photosensitive channel is used to sense light in the green and near-infrared bands, B photosensitive channel is used to sense blue and near-infrared light, and Y photosensitive channel is used to sense yellow and near-infrared light. For light, the W photosensitive channel is used to sense the full-wavelength light, and the C photosensitive channel is used to sense the full-wavelength light.

Since in some embodiments, W can be used to represent the light-sensing channel used to sense full-wavelength light, in other embodiments, C can be used to represent the light-sensing channel used to sense full-wavelength light, so when there is more When a photosensitive channel includes a photosensitive channel for sensing full-wavelength light, this photosensitive channel may be a W photosensitive channel or a C photosensitive channel. That is, in practical applications, the photosensitive channel for sensing the light of the whole waveband can be selected according to the use requirements. 14 to 17, the image sensor 01 is a channel array, and the channel array includes a plurality of photosensitive channels. Referring to Figure 14, the image sensor 01 may be an RGBW sensor. The RGBW sensor includes at least two of the R photosensitive channel, G photosensitive channel, B photosensitive channel, and W photosensitive channel. The W photosensitive channel is used for full-band white light and near infrared. Or, see Figure 15, the image sensor 01 can be an RCCB sensor, the RCCB sensor includes at least two of the R photosensitive channel, the C photosensitive channel and the B photosensitive channel, and the C photosensitive channel is used for full-band white light and Near-infrared light; or, see Figure 16, the image sensor 01 may be an RGB sensor, the RGB sensor includes at least two of the R photosensitive channel, G photosensitive channel and B photosensitive channel; or, see Figure 17, the image sensor 01 can be a RYYB sensor. The RYYB sensor includes at least two of the R photosensitive channel, the Y photosensitive channel and the B photosensitive channel. The Y photosensitive channel is used to sense light in the yellow light band and the near-infrared band.

In other embodiments, some photosensitive channels may only sense light in the near-infrared waveband, but not light in the visible light waveband. As an example, the plurality of photosensitive channels may include at least two of R photosensitive channels, G photosensitive channels, B photosensitive channels, and IR photosensitive channels. Among them, the R photosensitive channel is used to sense red light and near-infrared light, the G photosensitive channel is used to sense green light and near-infrared light, and the B photosensitive channel is used to sense blue light and near-infrared light. IR The photosensitive channel is used to sense light in the near-infrared band.

For example, the image sensor 01 may be an RGBIR sensor, where each IR photosensitive channel in the RGBIR sensor can sense light in the near-infrared waveband, but not light in the visible light waveband.

Among them, when the image sensor 01 is an RGB sensor, compared to other image sensors, such as RGBIR sensors, the RGB information collected by the RGB sensor is more complete. Some of the photosensitive channels of the RGBIR sensor cannot collect visible light, so the RGB sensor collects The color details of the image are more accurate.

It should be noted that the multiple photosensitive channels included in the image sensor 01 may correspond to multiple sensing curves. Exemplarily, referring to FIG. 18, the R curve in FIG. 18 represents the sensing curve of the image sensor 01 to light in the red light band, the G curve represents the sensing curve of the image sensor 01 to light in the green light band, and the B curve represents the image sensor 01 For the sensing curve of light in the blue band, the W (or C) curve represents the sensing curve of the image sensor 01 sensing the light in the full band, and the NIR (Near infrared) curve represents the sensing of the image sensor 01 sensing the light in the near infrared band. curve.

For each photosensitive channel, the sensing channel corresponds to the visible light wavelength range of a color, and the photosensitive channel has a high sensing quantum efficiency for the color light in the corresponding visible light wavelength range, so that the photosensitive channel can sense its corresponding visible light Color light in the wavelength range. For example, referring to Figure 18, for the RGBW sensor, the channel array of the RGBW sensor includes four color photosensitive channels of red, green, blue, and white. The four color photosensitive channels of red, green, blue, and white are respectively R photosensitive channel and G photosensitive channel. , B photosensitive channel and W photosensitive channel. Referring to Figure 18, the R photosensitive channel Red has a higher quantum efficiency for sensing red light in the red light band, so the R photosensitive channel can be used to sense red light in the red light band; the G photosensitive channel Green is for green in the green light band The light sensing quantum efficiency is high, so the G photosensitive channel can be used to sense green light in the green light band; the B photosensitive channel Blue has a higher sensing quantum efficiency for blue light in the blue light band, so the B photosensitive channel can be used Sensing blue light in the blue light band; W photosensitive channel W has a high quantum efficiency for sensing white light in the full band, so W photosensitive channel can be used to sense white light in the full band.

For the image signal processing unit 04, the image signal processing unit 04 is used to perform at least a first sharpening process on the first original image signal to obtain a grayscale image, and perform at least a second sharpening process on the second original image signal to obtain a color image. The intensity of one sharpening process is less than the intensity of the second sharpening process.

Among them, the sharpening processing of the image will result in the loss of information in the image. The intensity of the first sharpening processing is less than the intensity of the second sharpening processing, so that the intensity of the first sharpening processing can be reduced, and the first original image signal Performing at least the first sharpening process to obtain a grayscale image can reduce the amount of information loss in the grayscale image, so that the accuracy of intelligent analysis can be improved when performing intelligent analysis based on the grayscale image.

The first processing further includes at least one of black level correction, gamma correction, color correction, demosaicing, or noise reduction. The second processing further includes at least one of black level correction, gamma correction, color correction, demosaicing, or noise reduction.

Optionally, referring to FIG. 19, the image signal processing unit 04 is configured to fuse the image signal after preprocessing the first original image signal and the image signal after preprocessing the second original image signal to obtain a fused image, The merged image is determined to be a color image.

Wherein, preprocessing the first original image signal and the second original image signal can eliminate noise in the image signal. The first original image signal is exposed in the presence of near-infrared supplementary light, so the noise of the first original image signal is lower than that of the second original image signal, and the brightness of the image corresponding to the first original image signal is higher than that of the second original image signal. The brightness of the image, by fusing the two image signals after preprocessing, can reduce the noise in the color image and increase the brightness of the color image.

The first original image signal and the second original image signal are two images respectively. The image signal processing unit 04 may extract the first edge image from the image signal after the first original image signal is preprocessed, and the first edge image includes the first edge image. An original image signal corresponds to the edge of each object image in the image, and a second edge image is extracted from the image signal after the second original image signal is preprocessed, and the second edge image includes each object in the image corresponding to the second original image signal The edge of the image. Then through the following first formula, the fusion is performed to obtain the fused image.

The first formula is:

In the first formula: K _i,j is the pixel point in the image signal after the first original image signal is preprocessed, and Z _i,j is the pixel point in the image signal after the second original image signal is preprocessed, H _i,j are the pixels in the fused image, _mi,j are the pixels in the first edge image, n _i,j are the pixels in the second edge image, and (i,j) are the pixels s position.

For the above analysis unit 06, the analysis unit 06 includes an intelligent analysis algorithm, and the analysis unit 06 uses the intelligent analysis algorithm to perform intelligent analysis on the gray image to obtain an analysis result or target sub-image. For example, the intelligent analysis algorithm may be a face detection algorithm. The analysis unit 06 uses the face detection algorithm to intelligently analyze the gray image to obtain an analysis result. The analysis result is whether the gray image is a human face image, or the analysis result is The detected face image.

The intelligent analysis algorithm can be obtained by training a neural network. For example, FastRCNN is a neural network, and face detection algorithms can be obtained by training FastRCNN.

The encoding compression unit 05 may use the H.264 standard or the H.265 standard to compress and encode the color image sequence generated by the image signal processing unit 2 to obtain a video code stream.

Next, the relevant components in this embodiment are briefly described as follows:

The light fill 021 performs near-infrared fill light in a stroboscopic manner, and does not exclude the generation of visible light fill light in a certain way at the same time, specifically: the fill light does not perform near-infrared fill light during the exposure period of the second preset exposure. Perform near-infrared fill light in the exposure time period of the first preset exposure.

The filter assembly 03 can pass the near-infrared supplement light generated by the light supplement 02, while allowing visible light to pass, while blocking other light.

Each pixel in the image sensor 01 can sense near-infrared light to ensure that the near-infrared images collected under supplemental light have clear details, and help improve the effect of intelligent analysis.

Each pixel in the image sensor 01 can sense at least one visible light of red light, green light, and blue light, or can sense all the above three types of visible light at the same time, to ensure that the captured color video has sufficient resolution.

Image sensor 01 uses different exposure parameters for the first preset exposure and the second preset exposure. The exposure parameters include but are not limited to exposure time, analog gain, digital gain, and aperture size to ensure that the image collected for intelligent analysis is appropriate Of exposure.

Filler 02 near-infrared fill light, the energy is concentrated in the range of 700nm ~ 800nm, or concentrated in the range of 900nm ~ 1000nm, avoiding 800nm ~ 900nm, in order to reduce the interference caused by common 850nm infrared lamps.

In addition to the near-infrared supplementary light, the light supplement 02 also generates visible light supplementary light, which can make the near-infrared supplementary light look less reddish and avoid confusion with traffic lights.

The filter component 03 can pass the near-infrared supplementary light. Specifically, in the near-infrared band of 650nm~1000nm, the passed band width is less than the sum of the blocked band widths, so as to ensure the effective use of the infrared supplementary light, as little as possible Interference from other light sources.

The filter component 03 can pass the near-infrared supplementary light. Specifically, in the near-infrared band from 650nm to 1000nm, the width of the band whose pass rate is greater than 30% is less than 100nm, so as to ensure the effective use of the infrared supplementary light, as little as possible. Light source interference.

The filter assembly 03 has a switching component 033. In addition to the above-mentioned first state, it can also be switched to a second state that blocks near-infrared light and allows visible light to pass, so that the image capture device can be easily switched to the image capture mode of the existing camera, which is compatible Existing usage.

The image signal processing unit 04, when processing the first original image signal and the second original image signal, the processing steps performed are different; and when the sharpening processing is performed, the processing intensity used is different. Specifically, the processing steps are different. 2. The original image signal adopts weaker sharpening intensity, which is more suitable for the needs of intelligent analysis.

The image signal processing unit 04, when generating a color image, can fuse the first original image signal and the second original image signal to generate a higher quality color image, which is more suitable for the needs of the video stream for human eyes to watch.

In the implementation column of this application, the image acquisition device includes: an image sensor, a light supplement and a filter component, the image sensor is located on the light exit side of the filter component; it also includes an image signal processing unit, an encoding compression unit and an analysis unit; It includes a first light-filling device. The near-infrared light-filling is performed by the first light-filling device in a strobe mode, that is, the near-infrared light-filling is performed during the exposure time period of the first preset exposure of the image sensor. The near-infrared light supplement is not performed during the exposure time period of the second preset exposure of the second exposure; the filter assembly includes a first filter, and the first filter is used to pass visible light and part of the near-infrared light. The near-infrared light that passes through the first filter during the exposure time period includes the near-infrared light that is reflected by the object and enters the filter assembly when the first light supplement device performs near-infrared light supplementation, and is in the second preset exposure time period The near-infrared light passing through the first filter includes the near-infrared light reflected by the object and entering the filter assembly when the first light supplement device is not performing near-infrared light supplement; the image sensor is used to generate and output the first light through multiple exposures. The original image signal and the second original image signal, where the first original image signal is an image signal generated according to a first preset exposure, the second original image signal is an image signal generated according to a second preset exposure, and the first preset The exposure and the second preset exposure are two of the multiple exposures; the image signal processing unit is used to perform first processing on the first original image signal to obtain a grayscale image, and to perform the second processing on the second original image signal to obtain a color Image: Use the encoding compression unit to compress and encode the color image to obtain the video stream; use the analysis unit to perform intelligent analysis on the gray image to obtain the analysis result. Since during the exposure time period of the first preset exposure, the first light-filling device performs near-infrared light-filling, so that the image sensor is exposed to the outside world according to the exposure parameters of the first preset exposure during the exposure time period of the first preset exposure. The ambient light in the environment and the near-infrared light are sensed to obtain the first original image signal, which can reduce the noise in the first original image signal, and process the first original image signal to obtain a gray image. The noise of the gray image also varies with The reduction, analysis based on gray image, can improve the accuracy of the analysis results.

Referring to FIG. 20, an embodiment of the present application provides a camera method, which is applied to an image acquisition device. The image acquisition device may be any image acquisition device in the embodiment shown in FIG. 1, that is, the image acquisition device includes an image sensor 01. The light fill 02 and the filter assembly 03, the image sensor 01 is located on the light exit side of the filter assembly 03, the light fill 02 includes a first light fill device, and the filter assembly 03 includes a first filter sheet. The method includes:

Step 201: The image acquisition device performs near-infrared light supplementation through the first light-filling device, where the first light-filling device performs near-infrared light supplementation at least during a partial exposure period of the first preset exposure of the image sensor multiple exposures, and The near-infrared fill light is not performed during the exposure time period of the second preset exposure of the multiple exposure, and the first preset exposure and the second preset exposure are two exposures in the multiple exposures.

The first supplementary light device 021 is a device that can emit near-infrared light, such as a near-infrared supplementary light, which is not limited in the embodiment of the present application. The first light supplement device 021 can perform near-infrared supplement light in a stroboscopic manner, or in other ways similar to stroboscopic. In some examples, when the first light supplement device 021 performs near-infrared supplement light in a stroboscopic manner, the first supplement light device 021 may be manually controlled to perform near-infrared supplement light in a stroboscopic manner, or through a software program Or a specific device controls the first light supplement device 021 to perform near-infrared supplement light in a strobe mode, which is not limited in the embodiment of the present application. The time period during which the first light supplement device 021 performs near-infrared light supplementation may coincide with the exposure time period of the first preset exposure, or may be greater than the exposure time period of the first preset exposure or less than the exposure time period of the first preset exposure. The time period, as long as the near-infrared supplement light is performed during the entire exposure time period or part of the exposure time period of the first preset exposure, and the near-infrared supplement light is not performed during the exposure time period of the second preset exposure.

For example, referring to FIG. 3, the exposure time period of the first preset exposure and the exposure time period of the second preset exposure cyclically alternate, and the image capture device generates and emits near-infrared light during the exposure time period of the first preset exposure, In order to realize near-infrared supplementary light, the generation of near-infrared light is stopped during the exposure time period of the second preset exposure, so that near-infrared supplementary light is not performed. The so-called stroboscopic approach for near-infrared supplementary light is to generate and emit near-infrared light during the exposure time period of the first predicted exposure, and stop during the exposure time period of the second preset exposure adjacent to the exposure time period of the first predicted exposure Produce near-infrared light.

The first light supplement device 021 can be manually controlled to perform near-infrared supplement light in a stroboscopic manner, or the first light supplement device 021 can be controlled to perform near-infrared supplement light in a stroboscopic manner through a software program or a specific device. The embodiment does not limit this. The time period during which the first light supplement device 021 performs near-infrared light supplementation may coincide with the exposure time period of the first preset exposure, or may be greater than the exposure time period of the first preset exposure or less than the exposure time period of the first preset exposure. The time period, as long as the near-infrared supplement light is performed during the entire exposure time period or part of the exposure time period of the first preset exposure, and the near-infrared supplement light is not performed during the exposure time period of the second preset exposure.

During the exposure time period of the first preset exposure, the first supplementary light device 021 generates and emits near-infrared light to the external environment, so the external environment includes ambient light and first light during the exposure time period of the first preset exposure. The near-infrared light generated by the light supplement device 021, the ambient light includes visible light, and may also include at least one of near-infrared light and infrared light, etc., so that within the exposure time period of the first preset exposure, the object is reflected into the image The light of the first filter 031 in the collection device includes the near-infrared light and the ambient light generated by the first light supplement device 021, and the first filter 031 passes the near-infrared light and the visible light and near-infrared light in the ambient light. Infrared light. During the exposure time period of the second preset exposure, the first light supplement device 021 stops generating near-infrared light, so during the exposure time period of the second preset exposure, the external environment includes ambient light, which is reflected by the object into the first filter The light of the light sheet 031 includes the ambient light, and the first filter 031 passes visible light and near-infrared light in the ambient light.

The wavelength range of the first light supplement device 021 for near-infrared supplement light may be the second reference wavelength range, and the second reference wavelength range may be 700 nanometers to 800 nanometers, or 900 nanometers to 1000 nanometers, etc. This is not the case in this embodiment. Make a limit. In addition, the wavelength range of the near-infrared light incident on the first filter 031 may be the first reference wavelength range, and the first reference wavelength range is 650 nm to 1100 nm. As an example, the first reference wavelength range may be 650 nm. Nano ~ 1000 nm. Among them, it should be noted that: the second reference waveband range may be 700 nm to 800 nm, which means that the second reference waveband range is greater than or equal to 700 nm and less than or equal to 800 nm, or is greater than 700 nm and less than 800 nm. The range is a range greater than or equal to 700 nanometers and less than 800 nanometers, or a range greater than 700 nanometers and less than or equal to 800 nanometers. For similar scopes appearing elsewhere in this article, the meanings are the same, so we will not explain them one by one.

Since in the exposure time period of the first preset exposure, the near-infrared light passing through the first filter 031 may include the first light-filling device 021 that is reflected by the object and enters the first filter 031 when the near-infrared light is filled. Near-infrared light, therefore, the center wavelength and/or band width of the near-infrared light passing through the first filter 031 can be matched with the center wavelength of the near-infrared light supplemented by the first light supplement device 021, that is, the first light supplement When the center wavelength of the near-infrared complementary light performed by the optical device 021 is the set characteristic wavelength or falls within the set characteristic wavelength range, the center wavelength and/or band width of the near-infrared light passing through the first filter 031 meets the constraint conditions.

There are multiple choices for the center wavelength and/or wavelength range of the first light supplement device 021 for near-infrared supplement light. In the embodiment of the present application, in order to make the first light supplement device 021 and the first filter 031 better In cooperation, the center wavelength of the near-infrared supplement light of the first light supplement device 021 can be designed, and the characteristics of the first filter 031 can be selected, so that the center of the first light supplement device 021 for the near-infrared light supplement When the wavelength is the set characteristic wavelength or falls within the set characteristic wavelength range, the center wavelength and/or band width of the near-infrared light passing through the first filter 031 can meet the constraint conditions. This constraint is mainly used to restrict the center wavelength of the near-infrared light passing through the first filter 031 as accurate as possible, and the band width of the near-infrared light passing through the first filter 031 is as narrow as possible, so as to avoid The infrared light band width is too wide and introduces wavelength interference.

Among them, the set characteristic wavelength or the set characteristic wavelength range can be preset. As an example, the center wavelength of the first light supplement device 021 for near-infrared supplement light may be any wavelength within the wavelength range of 750±10 nanometers; or, the center wavelength of the first light supplement device 021 for near-infrared supplement light It is any wavelength in the wavelength range of 780±10 nanometers; or, the center wavelength of the first light supplement device 021 for near-infrared supplement light is any wavelength in the wavelength range of 940±10 nanometers. That is, the set characteristic wavelength range may be a wavelength range of 750±10 nanometers, or a wavelength range of 780±10 nanometers, or a wavelength range of 940±10 nanometers. Exemplarily, the center wavelength of the near-infrared supplement light performed by the first light supplement device 021 is 940 nanometers, and the relationship between the wavelength and the relative intensity of the near-infrared supplement light performed by the first light supplement device 021 is shown in FIG. 11. It can be seen from FIG. 11 that the wavelength range of the first light supplement device 021 for near-infrared supplement light is 900 nanometers to 1000 nanometers, and between 940 nanometers and 960 nanometers, the relative intensity of near-infrared light is the highest.

Since in the exposure time period of the first preset exposure, most of the near-infrared light passing through the first filter 031 is the near-infrared light of the first light supplement device 021 when it is reflected by the object and enters the near-infrared light of the first filter 031. Infrared light, therefore, in some embodiments, the above constraint conditions may include: the difference between the center wavelength of the near-infrared light that passes through the first filter 031 and the center wavelength of the near-infrared light that the first light supplement device 021 performs The value lies within the wavelength fluctuation range. As an example, the wavelength fluctuation range may be 0-20 nanometers.

Step 202: The image acquisition device passes the first filter 031 to pass visible light and part of the near-infrared light.

The filter assembly 03 in the image acquisition device includes a first filter 032, a second filter 032, and a switching component 033, and the first filter 031 and the second filter 032 are both connected to the switching component 033;

The first filter 031 is switched to the light incident side of the image sensor 01 through the switching component 033; the first filter 031 is used to pass visible light and part of the near-infrared light, wherein, within the exposure time period of the first preset exposure The near-infrared light passing through the first filter 031 includes the near-infrared light reflected by the object and entering the filter assembly 03 when the first light-filling device 021 performs near-infrared light-filling, and passes through the second preset exposure time period. The near-infrared light of a filter 031 includes the near-infrared light reflected by the object into the filter assembly 03 when the first light-filling device 021 is not performing near-infrared light-filling; wherein, the first light-filling device 021 performs near-infrared light-filling The intensity of the near-infrared light passing through the first filter 031 is higher than the intensity of the near-infrared light passing through the first filter 031 when the first light supplement device 021 does not perform near-infrared light supplementation.

The second filter 032 is switched to the light incident side of the image sensor 01 through the switching component 033; the second filter 032 allows the light in the visible light band to pass and blocks the light in the near-infrared light band, and the image capture device exposes the image sensor 01 The third image signal is generated and output.

Step 203: The image acquisition device generates a first original image signal and a second original image signal through multiple exposures, where the first original image signal is an image signal generated according to the first preset exposure, and the second original image signal is an image signal generated according to the first preset exposure. 2. The image signal generated by the preset exposure.

Since the first light supplement device 021 performs near-infrared light supplement light within the exposure time period of the first preset exposure, the external environment includes the near-infrared light and ambient light supplemented by the first light supplement device 021, and the ambient light Including visible light and near-infrared light. Objects in the external environment can reflect the near-infrared light supplemented by the first light-filling device 021 to the image acquisition device, and reflect ambient light to the image acquisition device. The image acquisition device can reflect the reflected near-infrared light through the image sensor 01 Exposing with visible light and near-infrared light in the ambient light generates a first original image signal.

The first light supplement device 021 does not perform near-infrared light supplement light during the exposure time period of the second preset exposure, so the external environment includes ambient light, and the ambient light includes visible light and near-infrared light. Objects in the external environment can reflect ambient light to the image acquisition device, and the image acquisition device can expose visible light and near-infrared light in the reflected ambient light through the image sensor 01 to generate a second original image signal.

The image sensor 01 includes a plurality of photosensitive channels, and the plurality of photosensitive channels are used to sense at least two different visible light wavelength bands. Each light-sensitive channel is used to sense at least one color of light in the visible light band and light in the near-infrared band, and the at least one type of visible light includes red light, green light, blue light, yellow light, and the like.

As an example, each photosensitive channel corresponds to a visible light wavelength range of one color, that is, each photosensitive channel is used to sense visible light and near-infrared light in the visible wavelength range of a corresponding color. For example, the multiple photosensitive channels include at least two of R photosensitive channels, G photosensitive channels, B photosensitive channels, Y photosensitive channels, W photosensitive channels, and C photosensitive channels; wherein, the R photosensitive channel is used to sense the red light band and near Infrared light, G photosensitive channel is used to sense light in the green and near-infrared bands, B photosensitive channel is used to sense blue and near-infrared light, and Y photosensitive channel is used to sense yellow and near-infrared light. Light, the W photosensitive channel is used to sense the full-wavelength light, and the C photosensitive channel is used to sense the full-wavelength light.

Since in some embodiments, W can be used to represent the light-sensing channel used to sense full-wavelength light, in other embodiments, C can be used to represent the light-sensing channel used to sense full-wavelength light, so when there is more When a photosensitive channel includes a photosensitive channel for sensing light of a full waveband, this photosensitive channel may be a W photosensitive channel or a C photosensitive channel. That is, in practical applications, the photosensitive channel used for sensing the light of the full waveband can be selected according to the use requirements. 14 to 17, the image sensor 01 is a channel array, and the channel array includes a plurality of photosensitive channels. Referring to Figure 14, the image sensor 01 may be an RGBW sensor. The RGBW sensor includes at least two of the R photosensitive channel, G photosensitive channel, B photosensitive channel and W photosensitive channel. The W photosensitive channel is used for full-band white light and near infrared. Or, see Figure 15, the image sensor 01 can be an RCCB sensor. The RCCB sensor includes at least two of the R photosensitive channel, the C photosensitive channel and the B photosensitive channel. The C photosensitive channel is used for full-band white light and Light in the near-infrared band; or, see FIG. 16, the image sensor 01 may be an RGB sensor, and the RGB sensor includes at least two of the R photosensitive channel, the G photosensitive channel, and the B photosensitive channel; or, see FIG. 17, the image sensor 01 can be a RYYB sensor. The RYYB sensor includes at least two of the R photosensitive channel, the Y photosensitive channel, and the B photosensitive channel. The Y photosensitive channel is used to sense light in the yellow light band and the near-infrared band.

It should be noted that the multiple photosensitive channels included in the image sensor 01 may correspond to multiple sensing curves. Exemplarily, referring to FIG. 18, the R curve in FIG. 18 represents the sensing curve of the image sensor 01 to light in the red light band, the G curve represents the sensing curve of the image sensor 01 to light in the green light band, and the B curve represents the image sensor 01 For the sensing curve of light in the blue band, the W (or C) curve represents the sensing curve of the image sensor 01 sensing the light in the full band, and the NIR (Near infrared) curve represents the sensing of the image sensor 01 sensing light in the near infrared band. curve.

For each photosensitive channel, the sensing channel corresponds to the visible light wavelength range of a color, and the photosensitive channel has a high sensing quantum efficiency for the color light in the corresponding visible light wavelength range, so that the photosensitive channel can sense its corresponding visible light Color light in the wavelength range. For example, referring to Figure 18, for the RGBW sensor, the channel array of the RGBW sensor includes four color photosensitive channels of red, green, blue, and white. The four color photosensitive channels of red, green, blue and white are respectively R photosensitive channel and G photosensitive channel. , B photosensitive channel and W photosensitive channel. Referring to Fig. 18, the R photosensitive channel Red has a high sensing quantum efficiency for red light in the red light band, so the R photosensitive channel can be used to sense red light in the red light band; G photosensitive channel Green is for green in the green light band The sensing quantum efficiency of light is high, so the G photosensitive channel can be used to sense green light in the green light band; the B photosensitive channel Blue has a higher quantum efficiency for sensing blue light in the blue light band, so the B photosensitive channel can be used Sensing blue light in the blue light band; W photosensitive channel W has a higher quantum efficiency for sensing white light in the full band, so the W photosensitive channel can be used to sense white light in the full band.

As an example, during the exposure time period of the first preset exposure, the image capture device performs exposure based on the exposure parameters of the first preset exposure to generate the first original image signal; during the exposure time period of the second preset exposure, The image acquisition device performs exposure based on the exposure parameters of the second preset exposure to generate a second original image signal.

For the exposure parameter of the first preset exposure and the exposure parameter of the second preset exposure, the exposure parameter includes at least one of exposure time, analog gain, digital gain, and aperture size.

The exposure parameter of the first preset exposure and the exposure parameter of the second preset exposure may be the same, or the exposure parameter of the first preset exposure and at least one exposure parameter of the second preset exposure may be different.

In the case that the exposure parameter of the first preset exposure and the at least one exposure parameter of the second preset exposure are different, the at least one exposure parameter is at least one of exposure time, exposure gain, and aperture size, and the exposure gain includes analog gain , And/or, digital gain.

At least one parameter in the exposure parameter of the first preset exposure is different from at least one parameter in the exposure parameter of the second preset exposure. As an example, the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure. During the exposure time period of the first preset exposure, the intensity of the near-infrared light sensed by the image acquisition device is relatively strong, and accordingly the intensity of the near-infrared light included in the first original image signal generated and output will also be relatively high. However, near-infrared light with higher brightness is not conducive to the acquisition of external scene information. The greater the exposure gain, the higher the brightness of the image signal generated by the exposure of the image capture device, and the smaller the exposure gain, the lower the brightness of the image signal generated by the exposure of the image capture device. Therefore, in order to ensure that the first original image signal contains the near infrared The brightness of the light is within a suitable range, and the exposure gain of the first preset exposure may be smaller than the exposure gain of the second preset exposure. In this way, when the image acquisition device performs near-infrared supplementary light, the brightness of the near-infrared light contained in the first original image signal generated by the image acquisition device exposure will not be too high due to the near-infrared supplementary light performed by the image acquisition device. and / or,

As an example, the exposure time of the first preset exposure is less than the exposure time of the second preset exposure. The longer the exposure time, the higher the brightness of the image generated by the exposure of the image capture device, and the shorter the exposure time, the lower the brightness included in the image signal generated by the exposure of the image capture device. Therefore, in order to ensure that the brightness of the near-infrared light contained in the first original image signal is within an appropriate range, the exposure time of the first preset exposure may be shorter than the exposure time of the second preset exposure. In this way, when the image acquisition device performs near-infrared supplementary light, the brightness of the near-infrared light contained in the first original image signal generated by the image acquisition device exposure will not be too high due to the near-infrared supplementary light performed by the image acquisition device. In addition, when shooting moving objects, a shorter exposure time is not easy to cause motion tailing in the image signal, thereby ensuring a higher imaging quality.

As an example, when the exposure gain of the first preset exposure is less than the exposure gain of the second preset exposure, the exposure time of the first preset exposure may not only be less than the exposure time of the second preset exposure, but may also be equal to the second preset exposure. Exposure time for preset exposure. Similarly, when the exposure time of the first preset exposure is less than the exposure time of the second preset exposure, the exposure gain of the first preset exposure may not only be less than the exposure gain of the second preset exposure, but also be equal to the second preset exposure. Exposure gain for exposure.

As an example, the number of times of supplementary light in the unit time length of the image acquisition device is lower than the number of exposures of the image acquisition device in the unit time length, wherein, within the interval of two adjacent supplementary times, there is one interval Or multiple exposures.

In this step, the multiple exposure operation of the image capture device can include the following two methods:

In the first method, the image acquisition device uses a global exposure method to perform multiple exposures. For any one near-infrared fill light, the time period of the near-infrared fill light does not overlap with the exposure time period of the nearest second preset exposure , The time period of near-infrared supplement light is a subset of the exposure time period of the first preset exposure, or the time period of near-infrared supplement light and the exposure time period of the first preset exposure overlap, or the first preset exposure The exposure time period is a subset of the near-infrared fill light.

In the second way, the image capture device uses rolling shutter exposure for multiple exposures. For any one near-infrared fill light, the time period of near-infrared fill light does not exist with the exposure time period of the nearest second preset exposure Intersection; the start time of the near-infrared fill light is no earlier than the exposure start time of the last line of the effective image in the first preset exposure, and the end time of the near-infrared fill light is no later than the exposure of the first line of the effective image in the first preset exposure End time; or, the start time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image of the nearest second preset exposure before the first preset exposure and no later than the first preset exposure The exposure end time of one line of effective image, the end time of near-infrared fill light is no earlier than the exposure start time of the last line of effective image in the first preset exposure and no later than the nearest second preset after the first preset exposure Set the exposure start time of the first line of the effective image of the exposure; or the start time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image of the nearest second preset exposure before the first preset exposure. Later than the exposure start time of the first line of the effective image in the first preset exposure, the end time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image in the first preset exposure and no later than the The exposure start time of the first line of the effective image of the nearest second preset exposure after the first preset exposure.

The multiple exposures include odd exposures and even exposures;

As an example, the first preset exposure is one exposure in odd-numbered exposures, and the second preset exposure is one exposure in even-numbered exposures.

As an example, the first preset exposure is one exposure in an even number of exposures, and the second preset exposure is one exposure in an odd number of exposures.

As an example, the first preset exposure is one exposure in the specified odd-numbered exposures, and the second preset exposure is one exposure in other exposures except the specified odd-numbered exposures.

As an example, the first preset exposure is one exposure in the specified even number of exposures, and the second preset exposure is one exposure in other exposures except the specified even number of exposures.

As an example, the first preset exposure is one exposure in the first exposure sequence, and the second preset exposure is one exposure in the second exposure sequence.

Wherein, multiple exposures include multiple exposure sequences, the first exposure sequence and the second exposure sequence are one exposure sequence or two exposure sequences among the multiple exposure sequences, each exposure sequence includes N exposures, and N exposures include 1. First preset exposure and N-1 second preset exposure, or N exposures include 1 second preset exposure and N-1 second preset exposure, and N is a positive integer greater than 2.

It should be noted that, since the human eye can easily confuse the color of the near-infrared light supplemented by the image acquisition device with the color of the red light in the traffic light, the image acquisition device also performs visible light supplementation. In this way, if the image capture device provides visible light supplemental light at least during a partial exposure time of the first preset exposure, that is, it performs near-infrared supplementary light and visible light supplementary light at least during a partial exposure time period of the first preset exposure. The mixed color of the two lights can be distinguished from the color of the red light in the traffic light, thereby avoiding the human eye from confusing the color of the near-infrared fill light of the image acquisition device with the color of the red light in the traffic light. In addition, if the image capture device provides visible light supplementary light during the exposure time period of the second preset exposure, since the intensity of visible light is not particularly high during the exposure time period of the second preset exposure, the exposure time of the second preset exposure When the visible light supplement is performed within the time period, the brightness of the visible light in the second original image signal can also be increased, thereby ensuring the quality of image collection.

As an example, the implementation of visible light supplemental light by the image acquisition device may be:

The image acquisition device performs visible light supplementation in a constant light mode; or

The image acquisition device performs visible light supplementation in a stroboscopic manner, wherein visible light supplementation is performed at least during a partial exposure time period of the first preset exposure, and visible light supplementation is not performed during the entire exposure time period of the second preset exposure; or

The image acquisition device performs visible light supplementation in a stroboscopic manner, where visible light supplementation is not performed at least during the entire exposure time period of the first preset exposure, and visible light supplementation is performed during the partial exposure time period of the second preset exposure.

Step 204: The image acquisition device obtains a grayscale image according to the first original image signal, and obtains a color image according to the second original image signal.

In this step, the image acquisition device performs first signal processing on the first original image signal to obtain a grayscale image, and performs second signal processing on the second original image signal to obtain a color image.

The first signal processing includes at least a first sharpening process, and the second signal processing includes at least a second sharpening process, and the intensity of the first sharpening process is less than the intensity of the second sharpening process.

The first signal processing further includes at least one of black level correction, gamma correction, color correction, demosaicing, or noise reduction, and the second signal processing further includes black level correction, gamma correction, and color correction. At least one of processing such as demosaicing or noise reduction.

As an example, the image acquisition device may fuse the image signal after preprocessing the first original image signal and the image signal after preprocessing the second original image signal to obtain a fused image, and perform the first fusion image on the fused image. Second, signal processing to obtain a color image.

The first original image signal and the second original image signal are two images respectively. The image acquisition device can extract the first edge image from the image signal after the first original image signal is preprocessed, and the first edge image includes the first original image. The image signal corresponds to the edge of each object image in the image, and the second edge image is extracted from the image signal after the second original image signal is preprocessed. The second edge image includes the image of each object in the image corresponding to the second original image signal. edge. Then, the image acquisition device performs fusion through the above-mentioned first formula to obtain a fused image, and performs a second signal processing on the fused image to obtain a color image.

Step 205: Perform compression coding on the color image, and perform intelligent analysis on the grayscale image.

In this step, the neural network computing unit in the SoC chip can be used, and a deep learning network, such as FastRCNN, can be used to perform intelligent analysis on grayscale images, such as face detection.

In this step, the encoding compression module in the SoC chip can be used, for example, the H.264 standard is used to compress the image sequence into a video stream and output.

In this step, the color image is compressed and encoded to obtain a video code stream, and the gray image is intelligently analyzed to obtain an analysis result or a target sub-image.

When intelligent analysis is target detection or face detection, the obtained target sub-image can be a target image or a face image. For example, when performing license plate detection, the target sub-image obtained is a license plate image.

In the implementation column of this application, the image acquisition device performs near-infrared supplementary light during the exposure time period of the first preset exposure, and does not perform near-infrared supplementary light during the exposure time period of the second preset exposure; Suppose the image acquisition device exposes the near-infrared light reflected by the object and the visible light and near-infrared light in the ambient light to obtain the first original image signal during the exposure time period of the exposure, and the image is in the exposure time period of the second preset exposure The acquisition device exposes the visible light and near-infrared light in the ambient light reflected by the object to obtain the second original image signal; then the image acquisition device performs the first signal processing on the first original image signal to obtain the gray image, and the second original image The second signal processing is performed on the signal to obtain a color image, the color image is compressed and encoded to obtain a video code stream, and the gray image is intelligently analyzed to obtain an analysis result. Since during the exposure time period of the first preset exposure, the image capture device performs near-infrared fill light, so that the image capture device obtains the first exposure according to the exposure parameters of the first preset exposure within the exposure time period of the first preset exposure. The original image signal can reduce the noise in the first original image signal. The first original image signal is processed to obtain a gray image. The noise of the gray image is also reduced. The analysis based on the gray image can improve the analysis result. Accuracy.

After considering the specification and practicing the application disclosed herein, those skilled in the art will easily think of other embodiments of the application. This application is intended to cover any variations, uses, or adaptive changes of this application. These variations, uses, or adaptive changes follow the general principles of this application and include common knowledge or customary technical means in the technical field not disclosed in this application. . The description and embodiments are only regarded as exemplary, and the true scope and spirit of the application are pointed out by the following claims.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the application is only limited by the appended claims.

Claims

An image acquisition device, which is characterized by comprising: an image sensor (01), a light supplement (02) and a filter component (03), the image sensor (01) is located at the light output of the filter component (03) Side; also includes an image signal processing unit (04), a coding compression unit (05), and an analysis unit (06);

The image sensor (01) is used to generate and output a first original image signal and a second original image signal through multiple exposures, wherein the first original image signal is an image signal generated according to a first preset exposure, The second original image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two of the multiple exposures;

The light fill device (02) includes a first light fill device (021), and the first light fill device (021) is used to perform near-infrared fill light in a stroboscopic manner, wherein, at least in the first preset Performing near-infrared supplemental light during a partial exposure time period of the exposure, and not performing near-infrared supplementary light during the exposure time period of the second preset exposure;

The filter assembly (03) includes a first filter (031), and the first filter (031) passes visible light and part of near-infrared light;

The image signal processing unit (04) is configured to obtain a gray image according to the first original image signal, and obtain a color image according to the second original image signal;

The encoding compression unit (05) is configured to compress and encode the color image;

The analysis unit (06) is used for intelligent analysis of the gray image.
The image acquisition device according to claim 1, wherein:

At least one exposure parameter of the first preset exposure and the second preset exposure is different, the at least one exposure parameter is one or more of exposure time, exposure gain, and aperture size, and the exposure gain includes Analog gain, and/or, digital gain.
3. The image capture device of claim 2, wherein the exposure gain of the first preset exposure is smaller than the exposure gain of the second preset exposure.
The image acquisition device according to claim 1, wherein at least one exposure parameter of the first preset exposure and the second preset exposure is the same, and the at least one exposure parameter includes exposure time, exposure gain, One or more of the aperture size, the exposure gain includes analog gain, and/or, digital gain.
5. The image acquisition device of claim 4, wherein the exposure time of the first preset exposure is equal to the exposure time of the second preset exposure.
8. The image acquisition device of claim 1, wherein the obtaining a grayscale image according to the first original image signal and obtaining a color image according to the second original image signal comprises:

Perform at least a first sharpening process on the first original image signal to obtain a grayscale image, perform at least a second sharpening process on the second original image signal to obtain a color image, and the intensity of the first sharpening process is less than The intensity of the second sharpening process.
8. The image acquisition device according to claim 1, wherein said obtaining a color image according to the second original image signal comprises:

The image after the preprocessing of the first original image signal and the image after the preprocessing of the second original image signal are fused to obtain a fused image, and the fused image is determined as the color image.
The image acquisition device according to claim 1, wherein the filter assembly (03) further comprises a second filter (032) and a switching component (033), and the first filter (031) And the second filter (032) are both connected to the switching component (033);

The switching component (033) is used to switch the second filter (032) to the light incident side of the image sensor (01);

After the second filter (032) is switched to the light incident side of the image sensor (01), the second filter (032) allows light in the visible light band to pass and blocks light in the near-infrared light band , The image sensor (01) is used to generate and output a third image signal through exposure.
The image acquisition device according to claim 1, wherein the wavelength range of the near-infrared light incident on the first filter (031) is a first reference wavelength range, and the first reference wavelength range is 650 Nanometer ~ 1100 nm.
The image acquisition device according to claim 1, wherein:

When the center wavelength of the near-infrared supplement light performed by the first light supplement device (021) is the set characteristic wavelength or falls within the set characteristic wavelength range, the center of the near-infrared light passing through the first filter (031) The wavelength and/or band width meet the constraint conditions.
The image acquisition device according to claim 10, characterized in that the center wavelength of the near-infrared supplement light performed by the first light supplement device (021) is any wavelength within the wavelength range of 750±10 nanometers;

The center wavelength of the first light supplement device (021) for near-infrared supplement light is any wavelength within the wavelength range of 780±10 nanometers; or

The center wavelength of the first light supplement device (021) for near-infrared supplement light is any wavelength within the wavelength range of 940±10 nanometers.
9. The image acquisition device according to claim 10, wherein the constraint condition comprises:

The difference between the center wavelength of the near-infrared light passing through the first filter (031) and the center wavelength of the near-infrared light supplemented by the first light-filling device (021) lies within the wavelength fluctuation range, and The wavelength fluctuation range is 0-20 nanometers; or,

The restriction conditions include: the half bandwidth of the near-infrared light passing through the first filter (031) is less than or equal to 50 nanometers; or,

The restriction conditions include: the width of the first waveband is smaller than the width of the second waveband; wherein, the first waveband width refers to the waveband width of the near-infrared light passing through the first filter (031), and the second waveband The width refers to the wavelength band width of the near infrared light blocked by the first filter (031); or,

The restriction conditions include: the third waveband width is less than the reference waveband width, the third waveband width refers to the waveband width of near-infrared light whose pass rate is greater than a set ratio, and the reference waveband width is a waveband of 50 nanometers to 150 nanometers For any band width within the range, the set ratio is any ratio within the ratio range of 30% to 50%.
The image acquisition device according to claim 1, wherein the image sensor (01) comprises a plurality of photosensitive channels, and each photosensitive channel is used to sense at least one light in the visible light band, and to sense light in the near-infrared band. .
The image acquisition device according to claim 13, wherein the plurality of photosensitive channels are used to sense at least two different visible light wavelength bands.
The image acquisition device according to any one of claims 1 to 3, wherein the light supplement device (02) further comprises a second light supplement device (022);

The second light supplement device (022) is used to perform visible light supplement light in a constant light mode; or

The second light supplement device (022) is used to perform visible light supplement light in a stroboscopic manner, wherein the visible light supplement light is performed at least during a part of the exposure time period of the first preset exposure, and the visible light supplement is performed in the second preset exposure. No visible light fill is performed during the entire exposure time period of the exposure; or

The second light supplement device (022) is used to perform visible light supplement light in a stroboscopic manner, wherein at least visible light supplement light is not performed during the entire exposure time period of the first preset exposure. It is assumed that the visible light fill light is performed during a part of the exposure time period of the exposure.
The image acquisition device according to claim 1, characterized in that, the number of times of supplementary light in the unit time length of the first light supplement device (021) is lower than that of the image sensor (01) in the unit time length The number of exposures is one or more exposures within the interval between two adjacent fills.
The image acquisition device according to claim 1, wherein:

The image sensor adopts a global exposure mode for multiple exposures. For any one near-infrared supplementary light, the time period of near-infrared supplementary light does not overlap with the exposure time period of the nearest second preset exposure. The time period of light is a subset of the exposure time period of the first preset exposure, or the time period of near-infrared fill light and the exposure time period of the first preset exposure overlap, or the first preset exposure Let the exposure period of exposure be a subset of the near-infrared fill light.
The image acquisition device according to claim 1, wherein:

The image sensor adopts a rolling shutter exposure method to perform multiple exposures. For any one near-infrared supplement light, the time period of the near-infrared supplement light does not overlap with the exposure time period of the nearest second preset exposure;

The start time of the near-infrared fill light is no earlier than the exposure start time of the last line of the effective image in the first preset exposure, and the end time of the near-infrared fill light is no later than the first line of the effective image in the first preset exposure The end of the exposure;

or,

The start time of the near-infrared fill light is not earlier than the exposure end time of the last line of the effective image of the nearest second preset exposure before the first preset exposure and not later than the first preset exposure. The exposure end time of the line effective image, the end time of the near-infrared fill light is no earlier than the exposure start time of the last line of the effective image in the first preset exposure and no later than the nearest neighbor after the first preset exposure The exposure start time of the first line of the effective image of the second preset exposure; or

The start time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image of the nearest second preset exposure before the first preset exposure and no later than the first preset exposure. The exposure start time of the line effective image, the end time of the near-infrared fill light is no earlier than the exposure end time of the last line of the effective image in the first preset exposure and no later than the nearest neighbor after the first preset exposure The exposure start time of the first line of the effective image of the second preset exposure.
A photographing method, characterized in that it is applied to an image acquisition device, the image acquisition device includes an image sensor (01), a light supplement (02) and a filter component (03), the image sensor (01) is located in the On the light exit side of the light filter assembly (03), the light fill device (02) includes a first light fill device, the light filter assembly (03) includes a first filter, and the method includes:

The image acquisition device performs near-infrared light supplementation through the first light-filling device, wherein the first light-filling device performs near-infrared light supplementation at least within a partial exposure time period of the first preset exposure of the multiple exposures of the image sensor. Infrared supplementary light, no near-infrared supplementary light is performed during the exposure time period of the second preset exposure of the multiple exposures, and the first preset exposure and the second preset exposure are two of the multiple exposures;

The image acquisition device passes through the first filter to pass visible light and part of the near-infrared light;

The image acquisition device generates a first original image signal and a second original image signal through the multiple exposures, wherein the first original image signal is an image signal generated according to the first preset exposure, and the second The original image signal is an image signal generated according to the second preset exposure;

The image acquisition device obtains a grayscale image according to the first original image signal, and obtains a color image according to the second original image signal;

The image acquisition device compresses and encodes the color image, and performs intelligent analysis on the gray image.
The method of claim 19, wherein the image acquisition device obtains a grayscale image according to the first original image signal and obtains a color image according to the second original image signal, comprising:

Perform at least a first sharpening process on the first original image signal to obtain a grayscale image, perform at least a second sharpening process on the second original image signal to obtain a color image, and the intensity of the first sharpening process is less than The intensity of the second sharpening process.
The method of claim 19, wherein the image acquisition device obtaining a color image according to the second original image signal comprises:

The image after the preprocessing of the first original image signal and the image after the preprocessing of the second original image signal are fused to obtain a fused image, and the fused image is determined as the color image.