WO2020119504A1

WO2020119504A1 - Image processing method and system

Info

Publication number: WO2020119504A1
Application number: PCT/CN2019/122437
Authority: WO
Inventors: 范蒙; 俞海; 浦世亮
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2018-12-12
Filing date: 2019-12-02
Publication date: 2020-06-18
Also published as: CN110493506B; CN110493506A

Abstract

Provided are an image processing method and system. The system comprises: an image sensor for generating and outputting a first image signal and a second image signal by means of multiple instances of exposure, wherein the first image signal is an image signal generated according to a first preset exposure, and the second image signal is an image signal generated according to a second preset exposure; a light supplementing apparatus for performing near-infrared light supplementation within an exposure time period of the first preset exposure, and not performing near-infrared light supplementation within an exposure time period of the second preset exposure; an image processor for generating a first target image according to the first image signal, and generating a second target image according to the second image signal; and an intelligent analysis apparatus for acquiring images to be analyzed from the first target image and the second target image, and intelligently analyzing said images to obtain intelligent analysis results corresponding to the images to be analyzed. Therefore, the quality of images to be analyzed used for output or intelligent analysis can be improved by means of the present solution.

Description

Image processing method and system

This application claims the priority of the Chinese patent application filed on December 12, 2018 in the Chinese Patent Office with the application number 201811517428.8 and the invention titled "An Image Processing Method and System", the entire contents of which are incorporated by reference in this application.

Technical field

The present application relates to the field of image processing technology, in particular to an image processing method and system.

Background technique

In order to better obtain the information in the environment, the information in the environment can usually be recognized based on the image taken by the camera. However, in the related art, due to the variability of light, it is difficult for the camera to output high-quality images according to different ambient lights. There will always be cases where the image quality is good when the light is good and the image quality is poor when the light is poor. Therefore, In the above related technologies, the obtained image captured by the camera cannot be applied to all environments, resulting in poor information perception effect of the environment.

Summary of the invention

The purpose of the embodiments of the present application is to provide an image processing method and system to improve the quality of an image to be analyzed for output or intelligent analysis. The specific technical solutions are as follows:

In a first aspect, an embodiment of the present application provides an image processing system, including:

An image sensor for generating and outputting a first image signal and a second image signal through multiple exposures, wherein the first image signal is an image signal generated according to a first preset exposure, and the second image signal is based on An image signal generated by a second preset exposure, the first preset exposure and the second preset exposure are two of the multiple exposures;

The fill light device is used to perform near-infrared fill light in a strobe manner, specifically: the fill light device performs near-infrared fill light during the exposure period of the first preset exposure, and the second preset light No near-infrared fill light is applied during the exposure period of exposure;

An image processor for receiving the first image signal and the second image signal output by the image sensor, generating a first target image based on the first image signal, and generating a second target image based on the second image signal Target image

An intelligent analysis device is configured to obtain an image to be analyzed from the first target image and the second target image, and perform an intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

In a second aspect, an embodiment of the present application provides an image processing method, including:

Obtaining the first image signal and the second image signal output by the image sensor, wherein the image sensor generates and outputs the first image signal and the second image signal through multiple exposures, wherein the first image signal is based on the first An image signal generated by a preset exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are among the multiple exposures Double exposure; during the exposure period of the first preset exposure, the fill light device performs near infrared fill light, and during the exposure period of the second preset exposure, the fill light device does not perform near infrared fill light ;

Generating a first target image based on the first image signal, and generating a second target image based on the second image signal;

Obtaining an image to be analyzed from the first target image and the second target image;

Perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

In a third aspect, an embodiment of the present application provides an image processing apparatus, including:

An image signal obtaining module for obtaining the first image signal and the second image signal output by the image sensor, wherein the image sensor generates and outputs the first image signal and the second image signal through multiple exposures, wherein the first An image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure and the second preset exposure are Two of the multiple exposures; the fill light device performs near-infrared fill light during the exposure period of the first preset exposure, and the fill light during the exposure period of the second preset exposure The device does not perform near infrared fill light;

An image generating module, configured to generate a first target image based on the first image signal, and generate a second target image based on the second image signal;

An image selection module, configured to obtain an image to be analyzed from the first target image and the second target image;

The image analysis module is configured to perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

According to a fourth aspect, an embodiment of the present application provides a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

Memory, used to store computer programs;

The processor, when used to execute the program stored in the memory, implements the steps of an image processing method provided by the embodiments of the present application.

It can be seen that this solution uses near infrared fill light to the target scene to regulate the light environment of the target scene, so that the quality of the image signal received by the image sensor can be guaranteed, which in turn can be guaranteed for output or intelligent analysis. The image quality of the image. Therefore, this solution can improve the quality of the image to be analyzed for output or intelligent analysis.

Of course, implementing any of the products or methods of this application does not necessarily need to achieve all the advantages described above at the same time.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present application and the technical solutions of the prior art, the following briefly introduces the drawings required in the embodiments and the prior art. Obviously, the drawings in the following description are only For some embodiments of the application, for those of ordinary skill in the art, without paying any creative labor, other drawings may be obtained based on these drawings.

FIG. 1 is a schematic structural diagram of an image processing system provided by an embodiment of the present application;

2 is another schematic structural diagram of an image processing system provided by an embodiment of the present application;

FIG. 3(a) is a schematic diagram of the principle when the image processing system provided by the embodiment of the present application completes image processing through multiple units;

FIG. 3(b) is another schematic diagram of the image processing system provided by the embodiment of the present application when image processing is completed by multiple units;

FIG. 3(c) is another schematic diagram of the image processing system provided by the embodiment of the present application when image processing is completed by multiple units together;

Figure 4 is a schematic diagram of the array corresponding to the RGBIR image sensor;

5(a) is a schematic diagram illustrating the relationship between exposure and near-infrared fill light according to an embodiment of the present application;

FIG. 5(b) is another schematic diagram embodying the relationship between exposure and near-infrared fill light according to an embodiment of the present application;

Figure 6 is a schematic diagram of the principle of spectral blocking;

7 is a spectral diagram of a near-infrared light source;

8 is a flowchart of an image processing method provided by an embodiment of the present application;

9 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solutions and advantages of the present application more clear, the following describes the present application in further detail with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

The following first briefly introduces the technical terms involved in this application document.

Visible light is an electromagnetic wave that human eyes can perceive. The visible spectrum has no precise range. The wavelength of electromagnetic waves that the human eye can perceive is between 400 and 760 nm (nanometer), but some people can perceive that the wavelength is about 380 to 780 nm Between electromagnetic waves.

Near-infrared light refers to electromagnetic waves with a wavelength in the range of 780-2526nm.

The visible light image refers to a color image that only perceives visible light signals, and the color image is only sensitive to the visible light band.

Infrared-sensitive image refers to a brightness image that perceives near-infrared light signals. It should be noted that the infrared-sensing image is not limited to a brightness image that only perceives near-infrared light signals, but it may also be a brightness image that perceives near-infrared light signals and other band light signals.

In the first aspect, in order to improve the quality of the image to be analyzed for output or intelligent analysis, embodiments of the present application provide an image processing system.

As shown in FIG. 1, an image processing system provided by an embodiment of the present application may include:

The image sensor 110 is configured to generate and output a first image signal and a second image signal through multiple exposures, wherein the first image signal is an image signal generated according to a first preset exposure, and the second image signal is According to the image signal generated by the second preset exposure, the first preset exposure and the second preset exposure are two of the multiple exposures;

The fill light device 120 is used to perform near-infrared fill light in a strobe manner, specifically: the fill light device 120 performs near-infrared fill light during the exposure period of the first preset exposure, in the second The near-infrared fill light is not performed during the exposure period of the preset exposure;

The image processor 130 is configured to receive the first image signal and the second image signal output by the image sensor 110, generate a first target image according to the first image signal, and generate according to the second image signal Second target image;

The intelligent analysis device 140 is configured to obtain an image to be analyzed from the first target image and the second target image, and perform an intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

It should be noted that the image sensor 110 described in the embodiments of the present application may be exposed periodically, and may be exposed multiple times in each cycle. The first image signal and the second image signal are generated and output through multiple exposures described above, and may be the first image signal and the second image signal generated and output through multiple exposures in one cycle, but are not limited to multiple passes in one cycle The exposure generates and outputs the first image signal and the second image signal.

Wherein, the fill light device 120 performs near-infrared fill light during the exposure period of the first preset exposure, but does not perform near-infrared fill light during the exposure period of the second preset exposure, the first The preset exposure and the second preset exposure are different exposures.

Under this exposure and fill light control, when generating the first target image according to the first image signal generated by the first preset exposure, the first image signal can be interpolated, and the interpolated infrared-sensing image can be used as the first target The image, or the infrared-sensing image after image enhancement, is used as the first target image.

Under this exposure and fill light control, when generating the second target image based on the second image signal generated by the second preset exposure, the second image signal can be subjected to infrared removal processing to obtain a visible light image, and the visible light image is used as the second target The image, or the visible light image after image enhancement is used as the second target image; or, under this exposure and fill light control, when generating the second target image according to the second image signal generated by the second preset exposure, you can The second image signal of the frame is subjected to wide dynamic processing, and then the infrared processed image is processed to obtain a visible light image, and the visible light image is used as a second target image.

Among them, the structural schematic diagram of an image processing system shown in FIG. 1 is merely an example, and should not constitute a limitation on the embodiments of the present application. For example, in specific applications, the light supplementing device 120 may be combined with the image sensor 110 and image processing The device 130 or the intelligent analysis device 140 is electrically connected. Furthermore, the fill light device 120 can be controlled by the connected image sensor 110, image processor 130, or intelligent analysis device 140.

In addition, the image sensor 110, the fill light device 120, the image processor 130, and the intelligent analysis device 140 included in the image processing system can be integrated into an electronic device. At this time, the electronic device has fill light, image signal acquisition, and Image processing function. For example, the electronic device may be a camera, or other devices capable of acquiring images.

Of course, each component included in the image processing system may be deployed in at least two electronic devices. At this time, any one of the at least two electronic devices has functions of fill light, image signal acquisition, image processing, and intelligent analysis One or more of the functions. For example: the fill light device 120 is a separate device, and the image sensor 110, the image processor 130, and the intelligent analysis device 140 are all deployed in the camera; or, the fill light device 120 is a separate device, The image sensor 110 is deployed in the camera, and the image processor 130 and the intelligent analysis device 140 are deployed in a terminal or server associated with the camera.

In addition, it can be understood that the device where the image sensor 110 is located may further include an optical lens, so that light rays enter the image sensor 110 through the optical lens.

It should be noted that the fill light device 120 adopts a stroboscopic manner to perform near-infrared fill light on the target scene, that is, to perform non-continuous near-infrared light illumination on the target scene. The fill light device 120 is a device that can emit near-infrared light, such as a fill light; and the fill light of the fill light device 120 can be controlled manually, or can be controlled by a software program or a specific device The fill light of the light device 120 is reasonable.

In addition, the specific wavelength range of the near-infrared light used by the near-infrared fill light is not specifically limited in this application. As shown in the spectrum diagram of a near-infrared light source shown in FIG. 7, the near-infrared light source has a strong light intensity at about 850 nm. Therefore, in specific applications, in order to obtain the maximum response from the image sensor 110, this application The embodiment may use near-infrared light with a wavelength of 850 nm, but it is not limited thereto.

In addition, the fill light device 120 provides near-infrared light in a stroboscopic manner. Specifically, it refers to performing near-infrared fill light on an external scene by controlling the change of the light and dark of the near-infrared light. The bright process is considered to be near infrared fill light to the scene, and the process of the near infrared light in the fill light device 120 from the end to the start light is considered to be that the scene is not provided with near infrared light.

The image processing system provided by the embodiments of the present application is a single sensor perception system, that is, the image sensor 110 is a single.

Optionally, the image sensor 110 includes a plurality of photosensitive channels, the plurality of photosensitive channels includes an IR photosensitive channel, and further includes at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and a W photosensitive channel. The multiple photosensitive channels generate and output the first image signal and the second image signal through the multiple exposures;

Among them, R photosensitive channel is used for sensing light in the red and near infrared bands, G photosensitive channel is used for sensing light in the green and near infrared bands, and B photosensitive channel is used for sensing the blue and near infrared bands. Light, IR means infrared sensitive channel, used to sense light in near infrared band, W means all-pass sensitive channel, used to sense light in full band.

The image sensor 110 may be an RGBIR sensor, an RGBWIR sensor, an RWBIR sensor, an RWGIR sensor, or a BWGIR sensor; where R represents an R photosensitive channel, G represents a G photosensitive channel, B represents a B photosensitive channel, and IR represents an IR photosensitive channel, W means all-pass photosensitive channel.

Exemplarily, the image sensor 110 in the embodiment of the present application may be an RGBIR sensor, and the RGBIR sensor has a red-green-blue RGB photosensitive channel and an infrared IR photosensitive channel. Specifically, the RGB light-sensing channel can be sensitive to both the visible light band and the near-infrared band, but it is mainly used to light-sensitive the visible light band; and the IR light-sensing channel is a channel that is sensitive to the near-infrared band.

Wherein, when the image sensor 110 is an RGBIR sensor, the arrangement of the R photosensitive channel, G photosensitive channel, B photosensitive channel, and IR photosensitive channel can be seen in FIG. 4. The RGBIR image sensor senses the R photosensitive channel, G photosensitive channel, B photosensitive channel and IR photosensitive channel to obtain corresponding image signals. Among them, the sensitivity value corresponding to the R photosensitive channel includes the R channel value and the IR channel value; the sensitivity value corresponding to the G photosensitive channel includes the G channel value and the IR channel value, and the sensitivity value corresponding to the B photosensitive channel includes the B channel value and the IR channel value, The sensitivity value corresponding to the IR sensitivity channel includes the IR channel value. For the fill light device 120 providing near infrared fill light and not providing near infrared fill light, the R channel value and the IR light channel value sensed by the R photosensitive channel are different, and the G channel value and the IR channel perceived by the G light channel are different The value is different, the B channel value and the IR channel value perceived by the B photosensitive channel are different, and the IR channel value perceived by the IR photosensitive channel is different.

Therefore, when the fill light device 120 provides near infrared light fill light, the image signal captured by the RGBIR image sensor is the first image signal, and when the fill light device 120 does not provide near infrared light fill light, the image captured by the RGBIR image sensor The signal is the second image signal. Among them, the value of each channel of R photosensitive channel, G photosensitive channel, B photosensitive channel and IR photosensitive channel in the first image signal, and each of the R photosensitive channel, G photosensitive channel, B photosensitive channel and IR photosensitive channel in the second image signal The channel value is different. Correspondingly, for the RGBWIR sensor, RWBIR sensor, RWGIR sensor or BWGIR sensor, the channel values of each photosensitive channel in the first image signal are different from the channel values of the photosensitive channel in the second image signal.

In addition, for the case where the image sensor 110 is an RGBIR sensor, in order to ensure accurate color restoration after removing near-infrared light components and improve the quality of the scene image, the optical lens of the device where the image sensor 110 is located may be provided with a filter. The spectral region filtered by the filter may include [T1, T2]; wherein, 600nm≤T1≤800nm, 750nm≤T2≤1100nm, and T1<T2.

Referring to FIG. 6, it can be understood that the R, G, B, and IR photosensitive channels have large differences in response in the near-infrared band (650 nm to 1100 nm). In order to avoid the large difference in response of the above channels in certain spectral regions, the near-infrared light component is removed. For the problem of poor effect, a filter is provided on the optical lens to filter out the spectral region with a large difference in response. Specifically, the filter can be integrated on the above-mentioned optical lens by coating technology; in addition, the filter can be a band-stop filter or a bimodal filter with lower cost. It should be noted that When the filter is a bimodal filter, the spectral region filtered by the filter may further include a spectral region of [T3, +∞), 850nm≤T3≤1100nm, and T2<T3.

The light supplementing device 120 may adopt a strobe method to perform near-infrared light supplement on the target scene. Furthermore, the fill light device performs near infrared fill light during the exposure period of the first preset exposure, which may be: during the exposure period of the first preset exposure, the start of near infrared fill light The time is not earlier than the exposure start time of the first preset exposure, and the end time of performing near infrared fill light is not later than the exposure end time of the first preset exposure.

In order to facilitate understanding in the exposure time period of the first preset exposure, the start time of performing near infrared fill light is not earlier than the start time of exposure of the first preset exposure, and the end time of performing near infrared fill light is not late At the exposure end time of the first preset exposure, FIG. 5(a) and FIG. 5(b) exemplarily show a schematic diagram of the relationship between the exposure time and the fill time of the near infrared fill light. In FIG. 5(a), two exposures are used for the image sensor 110, that is, two exposures occur within one exposure period, and the two exposures are defined as an odd exposure and an even exposure, respectively. Perform near-infrared fill light on the target scene, that is, the even exposure is the first preset exposure. Specifically, the rising edge of near-infrared fill light is later than the start time of even exposure, and the falling edge of near-infrared fill light can be compared The exposure ends early. In FIG. 5(b), multiple exposures are used for the image sensor 110, that is, three exposures occur within one exposure period, and the three exposures are defined as A exposure, B exposure, and C exposure, respectively. The target scene performs near-infrared fill light, that is, the C exposure is the first preset exposure. Specifically, the rising edge of the near-infrared fill light is later than the start of the C exposure, and the falling edge of the near-infrared fill light can be shorter than the time when the C exposure ends early.

In addition, it can be understood that, since the near infrared fill light will enhance the brightness of the image, in order to ensure that the brightness of the first target image and the second target image remain within a suitable brightness range, in the embodiment of the present application, there is a supplement The exposure parameter corresponding to any exposure process of light may not be greater than the target maximum value, where the exposure parameter is the exposure duration and/or gain, and the target maximum value is the maximum value among the exposure parameters corresponding to the exposure without supplementary light .

In addition, by using multiple exposures of a single sensor and the near-infrared fill light of the fill light device, a second target image without near-infrared fill light and a first target image with near-infrared fill light can be captured. More specifically, in order for the image sensor 110 to capture the first image signal, the fill light device 120 provides near-infrared fill light at least during the exposure process in which the image sensor 110 captures the first image signal. In order to capture the second image signal without near infrared fill light, the fill light device 120 needs to not provide near infrared fill light during the exposure process in which the image sensor 110 captures the second image signal. Therefore, the number of near-infrared fill lights of the fill light device 120 per unit length of time is lower than the number of exposures of the image sensor 110 within a unit time length, and the interval between each two adjacent near-infrared fill lights is one or more times exposure. In this way, the fill light device 120 has near infrared fill light only during the partial exposure of the image sensor 110.

In addition, the specific timing of the fill light of the fill light device 120 in multiple exposures may be set according to actual scene requirements, that is, the first preset exposure may be set according to actual scene requirements. For multiple exposures of the image sensor 110, multiple exposures may include odd-numbered exposures and even-numbered exposures. Then, the configuration method of the first preset exposure may be as follows:

Exemplarily, in an implementation manner, the first preset exposure is one of odd exposures, and the second preset exposure is one of even exposures. At this time, the first image signal is a signal generated according to one of the odd exposures, and the second image signal is a signal generated according to one of the even exposures.

Exemplarily, in an implementation manner, the first preset exposure is one of even-numbered exposures, and the second preset exposure is one of odd-numbered exposures. At this time, the first image signal is a signal generated according to one of the even exposures, and the second image signal is a signal generated according to one of the odd exposures.

Exemplarily, in an implementation manner, the first preset exposure is one of the specified odd exposures, and the second preset exposure is other than the specified odd exposures One of them. At this time, the first image signal is a signal generated according to one of the specified odd exposures, and the second image signal is generated according to one of the other exposures other than the specified odd exposures signal.

Exemplarily, in an implementation manner, the first preset exposure is one of the specified even-numbered exposures, and the second preset exposure is other than the specified even-numbered exposures One of them. At this time, the first image signal is a signal generated according to one of the even-numbered exposures specified, and the second image signal is generated based on one of the other exposures other than the specified even-numbered exposure signal.

The timing of the fill light of the fill light device 120 in multiple exposures given above is merely an example, and should not constitute a limitation on the embodiments of the present application.

It should be noted that, after the image processor 130 parses and obtains the second target image and the first target image, in order to solve the prior art problem, the intelligent analysis device 140 may select the first target image and the second target image Obtain the image to be analyzed, and perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed. In a specific application, the intelligent analysis device 140 may acquire corresponding images to be analyzed according to scene requirements, and perform intelligent analysis on the acquired images to be analyzed.

Exemplarily, in an implementation manner, the intelligent analysis device 140 may obtain the first target image from the first target image and the second target image, and determine the first target image as a Describe the image to be analyzed. In this way, the intelligent analysis device can perform intelligent analysis based on the first target image by default.

Exemplarily, in another implementation manner, the intelligent analysis device 140 may obtain the second target image from the first target image and the second target image, and determine the second target image as The image to be analyzed. In this way, the intelligent analysis device can perform intelligent analysis based on the second target image by default.

Exemplarily, in yet another implementation manner, when the received selection signal is switched to the first selection signal, the intelligent analysis device 140 acquires the first target image and determines the first target image as the pending Analyzing the image; when the received selection signal is switched to the second selection signal, acquiring the second target image and determining the second target image as the image to be analyzed. In this way, the intelligent analysis device can switch from the first target image and the second target image to perform intelligent analysis.

It can be understood that, in this implementation, selecting the corresponding image according to the selection signal can improve the controllability of the image processing system, that is, switch the type of the acquired image according to different needs. In addition, the above specific implementation manner of selecting the corresponding image according to the selection signal is only an optional implementation manner. In addition, all the methods that can realize the selection signal are within the protection scope of the present application, and the present application does not limit this. Methods such as the mode selection or the default selection are reasonable.

In order to facilitate understanding of the perception process of the image processing system, the specific perception process of the image processing system will be described below in conjunction with FIGS. 3(a) and 3(b).

As shown in FIG. 3(a), the image processing system is embodied in the form of multiple units, and the multiple units jointly complete the image processing process. Of course, the division of the image processing system in FIG. 3(a) does not constitute a limitation on the present application, but is merely an exemplary description. Specifically, as shown in FIG. 3(a), the image processing system includes: a scene collection unit, a scene processing unit, a scene perception unit, and a scene fill light unit. The scene collection unit may include the above-mentioned optical lens, filter and image sensor 110. The scene fill light unit is the fill light device 120 described above. The function implemented by the scene processing unit is the function of the image processor 130 described above. The function is specifically: the scene processing unit obtains the first image signal and the second image signal output by the scene collection unit, and according to the first image signal A first target image is generated, and a second target image is generated based on the second image signal. Wherein, the scene perception unit is the above-mentioned intelligent analysis device 140, which is used to obtain the image to be analyzed from the first target image and the second target image, and intelligently analyze the image to be analyzed to obtain the to-be-analyzed The intelligent analysis result corresponding to the image.

In another way, as shown in FIG. 3(b), the image processing system includes: a scene collection unit, a scene processing unit, a selection unit, a scene perception unit, and a scene fill light unit. The scene collection unit may include the above-mentioned optical lens, filter and image sensor 110. The scene fill light unit is the fill light device 120 described above. The function implemented by the scene processing unit is the function of the image processor 130 described above. The function is specifically: the scene processing unit obtains the first image signal and the second image signal output by the scene collection unit, and according to the first image signal A first target image is generated, and a second target image is generated based on the second image signal. Wherein, the functions implemented by the selection unit and the scene perception unit are the functions implemented by the intelligent analysis device 140, specifically: when the received selection signal is switched to the first selection signal, the first target image is acquired and the The first target image is determined to be the image to be analyzed, and the image to be analyzed is intelligently analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed; when the received selection signal is switched to the second selection signal, the The second target image, determining the second target image as the image to be analyzed, and performing intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

It can be seen that this solution uses near infrared fill light to the target scene to regulate the light environment of the target scene, so that the image signal quality of the image sensor can be guaranteed, which can be used for output or intelligent analysis Image quality of the image. Therefore, this solution can improve the quality of the image to be analyzed for output or intelligent analysis.

Optionally, in an implementation manner, the multiple exposure of the image sensor 110 specifically includes: the image sensor 110 performs the multiple exposure according to a first exposure parameter, and the parameter type of the first exposure parameter includes exposure At least one of time and exposure gain;

The fill light device performs near-infrared fill light during the exposure time period of the first preset exposure, specifically: the fill light device performs the exposure time of the first preset exposure according to the first fill light parameter In the segment, near infrared fill light is performed, and the parameter type of the first fill light parameter includes at least one of fill light intensity and fill light concentration.

Optionally, in order to improve the degree of intelligence of image processing and image quality, the exposure parameters and/or fill light parameters may be adjusted based on image information corresponding to the image to be analyzed. Based on this processing idea, as shown in FIG. 2, the image processing system provided by the embodiment of the present application may further include: a control unit 150;

The control unit 150 is configured to obtain brightness information corresponding to the image to be analyzed, adjust the first fill light parameter to a second fill light parameter according to the brightness information corresponding to the image to be analyzed, and expose the first exposure The parameter is adjusted to the second exposure parameter; and the second fill light parameter is sent to the fill light device 120, and the second exposure parameter is sent to the image sensor 110 synchronously;

The fill light device 120 performs near infrared fill light during the exposure time period of the first preset exposure, specifically: the fill light device 120 receives the second fill light parameter from the control unit according to The second fill light parameter, performing near infrared fill light during the exposure time period of the first preset exposure;

The multiple exposure of the image sensor 110 is specifically: the image sensor 110 receives the second exposure parameter from the control unit, and performs the multiple exposure according to the second exposure parameter.

Among them, the image processing system shown in FIG. 2 is only an example, and should not constitute a limitation on the embodiment of the application. For example, in a specific application, the control unit 150 can be connected to the image in addition to the fill light device 120 The sensor 110, the image processor 130, or the intelligent analysis device 140 are connected so that the control unit 150 can interact with the image sensor 110, the image processor 130, or the intelligent analysis device 140 to complete image processing. In addition, it should be noted that the control unit 150 may be located in the same device as the fill light device 120, or may be located in a different device from the fill light device 120, which is reasonable. Moreover, in a specific application, the function performed by the control unit 150 may be performed by the image processor 130 or the intelligent analysis device 140.

Since the image brightness can reflect the exposure performance of the image sensor 110 and the fill light performance of the fill light device 120, the exposure parameters of the image sensor 110 and/or the fill light device can be adjusted based on the brightness information corresponding to the image to be analyzed 120 fill light parameters.

Exemplarily, in an implementation manner, the brightness information corresponding to the image to be analyzed may be obtained according to the intelligent analysis result corresponding to the image to be analyzed, which may specifically include:

When the intelligent analysis result corresponding to the image to be analyzed includes the position information of the target of interest included in the image to be analyzed, determining at least one target area in the image to be analyzed according to the position information;

The average brightness of the at least one target area is determined as brightness information corresponding to the image to be analyzed.

Wherein, at least one target area can be selected from the areas indicated by the location information, in this case, each target area is the area where the interest target is located.

Exemplarily, in one implementation, the adjusting the first exposure parameter to the second exposure parameter according to the brightness information corresponding to the image to be analyzed includes:

When the brightness information is higher than the first predetermined threshold, lower the first exposure parameter to obtain the second exposure parameter; when the brightness information is lower than the second predetermined threshold, increase the first exposure Parameters to obtain the second exposure parameter; wherein the first predetermined threshold is higher than the second predetermined threshold.

Exemplarily, in an implementation manner, the adjusting the first fill light parameter to the second fill light parameter according to the brightness information corresponding to the image to be analyzed may include:

When the brightness information is higher than a third predetermined threshold, lower the first fill light parameter to obtain the second fill light parameter; when the brightness information is lower than a fourth predetermined threshold, raise the first light fill parameter A fill light parameter to obtain the second fill light parameter; wherein the third predetermined threshold is higher than the fourth predetermined threshold.

It should be noted that the first predetermined threshold and the third predetermined threshold may be the same value or different values. Similarly, the second predetermined threshold and the fourth predetermined threshold may be the same value or different values. And, the specific values of the first predetermined threshold, the second predetermined threshold, the third predetermined threshold, and the fourth predetermined threshold may be set according to empirical values. In addition, the first fill light parameter and the second fill light parameter are only used to distinguish the fill light parameter before and after adjustment, and do not have any limited meaning. Similarly, the first exposure parameter and the second exposure parameter are only used to distinguish the before and after adjustment The exposure parameters do not have any limited meaning. In addition, the degree of increase or decrease of the fill light parameter and the exposure parameter can also be set based on empirical values.

In this implementation, the image processing system in this application further includes a control unit, which is used to adaptively control the fill light of the fill light device 120 and the exposure of the image sensor 110. As shown in FIG. 3(c), the image processing system is embodied in the form of multiple units, and the multiple units jointly complete the image processing process. Of course, the division of the image processing system in FIG. 3(c) does not constitute a limitation on the present application, but is merely an exemplary description. Specifically, as shown in FIG. 3(c), the electronic device includes: a scene collection unit, a scene processing unit, a scene perception unit, a scene fill light unit, and a control unit. The scene collection unit may include: the above-mentioned optical lens, filter and image sensor 110; the scene fill-in unit is the above-mentioned fill-light device 120; the control unit is the above-mentioned control unit 150; and the scene processing unit implements the above-mentioned image Functions implemented by the processor 130; the scene awareness unit implements the functions implemented by the intelligent analysis device 140 described above.

It should be noted that the control of the scene fill light unit and the scene collection unit in the system shown in FIG. 3(b) can also refer to FIG. 3(c), and the fill light control of the scene fill light unit can be performed by adding a control unit And the scene collection unit's collection control, the scene fill light unit and the scene collection unit can also adjust the fill light control of the scene fill light unit and the scene collection unit's collection control according to the intelligent analysis results fed back by the scene perception unit.

In some scenes, after obtaining the second target image and the first target image, the image processor 130 may further include the following steps: outputting the second target image for display, for example, the output second target image may be Displayed in a display device outside the system.

The image processor 130 may output only the second target image, and simultaneously output the second target image and the first target image. The specific image to be output is determined according to actual needs, and is not limited here.

The content related to generating a first target image based on the first image signal and generating a second target image based on the second image signal will be described below.

For the above single sensor perception system, there are many specific implementation manners of the image processor 130 generating the first target image according to the first image signal. Those skilled in the art can understand that since the signals of each channel of the sensor including the IR channel and at least two non-IR channels are staggered, when directly magnifying the image signal obtained by imaging the sensor, a mosaic phenomenon is found in the image, which is clear The degree is not good, so demosaicing is needed to generate a true-detail image. In order to obtain a clear and accurate first target image, the first image signal may be demosaiced, and then the demosaiced image signal is used to generate a first target image. Based on this, in one implementation, the image processor 130 generates a first target image according to the first image signal, including:

According to the channel values of a plurality of pixels included in the neighborhood of each pixel of the first image signal, interpolation processing is performed in an averaging manner, and the first target image is obtained according to the image after difference processing.

Wherein, according to actual needs, the difference-processed image may be determined as the first target image; or, the difference-processed image may be subjected to image enhancement processing, and the image after the image enhancement processing may be determined as The first target image. The specific method for determining the first target image is not limited in this application. In addition, image enhancement processing may include but is not limited to: histogram equalization, gamma correction, contrast lifting, etc., where histogram equalization converts the histogram of the original image to a probability density of 1 (ideal case) ) Image, Gamma correction uses a non-linear function (exponential function) to transform the gray value of the image, and contrast enhancement uses a linear function to transform the gray value of the image.

Wherein, the interpolation processing according to the channel values of the plurality of pixels included in the neighborhood of each pixel of the first image signal in an average manner includes:

Each channel value of each photosensitive channel of the first image signal is interpolated to obtain each channel value after interpolation processing of each photosensitive channel corresponding to each pixel in the first image signal; for each pixel The channel values after the interpolation processing of the corresponding photosensitive channels are averaged to obtain the image after the difference processing.

The interpolation algorithm used for interpolation may be a bilinear interpolation algorithm or a bicubic interpolation algorithm. The embodiments of the present application do not limit the interpolation algorithm. The first target image is obtained by averaging the channel values of the respective photosensitive channels corresponding to each pixel. In this case, the first target image is the demosaiced image. The first target image is an image including only a luminance signal. In the first target image, the luminance value of each pixel is: the average value of the corresponding channel values in the first image signal.

For the sake of clarity, a sensor including an IR channel and at least two non-IR channels is an RGBIR sensor, where the channel values of multiple pixels included in the neighborhood of each pixel according to the first image signal , Interpolating in an averaging manner, including:

Each IR photosensitive channel, R photosensitive channel, G photosensitive channel and B photosensitive channel of the first image signal are respectively interpolated to obtain the channel value after interpolation processing of each photosensitive channel corresponding to each pixel in the first image signal The average value of the channel values after interpolation processing of each photosensitive channel corresponding to each pixel is averaged to obtain the image after the difference processing.

Correspondingly, for the single sensor perception system described above, there are multiple specific implementations of the image processor 130 generating the second target image according to the second image signal. Exemplarily, in an implementation manner, the image processor 130 generating the second target image according to the second image signal may include:

Traverse the second image signal, adjust the channel value of each non-IR photosensitive channel traversed, respectively interpolate each channel value of each non-IR photosensitive channel after the channel value adjustment, and process according to the difference Image to obtain the second target image;

Wherein, the channel value adjustment for each non-IR photosensitive channel specifically includes: subtracting the IR parameter value corresponding to the corresponding pixel position of each channel value before the adjustment of the non-IR photosensitive channel, and the IR parameter value is the corresponding pixel A product of an IR value of a position and a preset correction value, and the IR value is an IR value sensed by the IR photosensitive channel at the position of the corresponding pixel.

Wherein, according to actual needs, the difference-processed image may be determined as the second target image; or, the difference-processed image may be subjected to image enhancement processing, and the image after the image enhancement processing may be determined as The second target image. The specific method for determining the second target image is not limited in this application.

It can be understood that by subtracting the IR parameter value corresponding to the corresponding pixel position of each traversed channel value of each non-IR channel, that is, removing the near infrared light component in the color signal, the near infrared light in the visible light signal can be avoided The components cross-talk with the three types of RGB signal components to improve the image effect under low illumination. It should be emphasized that the preset correction value can be set according to the actual situation. For example, the preset correction value can usually be set to 1, of course, according to the actual situation, the preset correction value can be set to 0 to Any integer or decimal in 1024, and those skilled in the art can understand that the value of the preset correction value is not limited to this.

For the sake of clarity, taking the sensor including the IR channel and at least two non-IR channels as RGBIR sensors as an example, the image processor 130 generates the second target image according to the second image signal, specifically:

Traverse the second image signal, subtract the IR parameter value corresponding to the corresponding pixel position of the traversed channel value of each R photosensitive channel, G photosensitive channel and B photosensitive channel, and adjust each R of the channel value The channel values of the photosensitive channel, the G photosensitive channel, and the B photosensitive channel are respectively interpolated, and the second target image is obtained according to the difference-processed image.

Exemplarily, in an implementation manner, the image processor 130 generating the second target image according to the second image signal may include:

Obtaining M frame second image signals including the current second image signal, performing wide dynamic synthesis processing on the M frame second image signals to obtain a wide dynamic image, and performing infrared removal processing on the wide dynamic image to obtain the A second target image; wherein the infrared removal processing includes:

Traverse the wide dynamic image, adjust the channel value of each non-IR photosensitive channel traversed, respectively interpolate the channel value of each non-IR photosensitive channel after the channel value adjustment, and process the image according to the difference To obtain the second target image.

The number of M frames is not limited, and M is less than the total number of exposures in one exposure period. High Dynamic (HDR) images have also become wide dynamic range images. Compared with low dynamic range images, there is no local overexposure, which can reflect more image details, so in this embodiment of the application A visible light image capable of obtaining more image details can be obtained, and at least two frames of second image signals can be subjected to wide dynamic synthesis processing to obtain a wide dynamic image signal.

Of course, a specific implementation manner of performing wide dynamic synthesis processing on the second image signal of each frame belongs to the prior art, and the embodiments of the present application will not be described in detail here. In addition, the process of performing infrared removal processing on the wide dynamic image signal to obtain a visible light image can refer to the aforementioned processing process for a frame of second image signal. Of course, when the second image signal of the fill light device 120 that does not perform near-infrared fill light is at least two frames, a frame of second image signal may also be selected, and a visible light image may be generated based on the selected frame of second image signal The specific generation process is the same as the generation process when the second image signal is one frame, which will not be repeated here.

The intelligent analysis in this application includes but is not limited to the types of objects included in the target scene, the area where the objects are located, etc. Correspondingly, the results of the intelligent analysis may include but not limited to: types of objects included in the target scene, Coordinate information of the area, location information of interest targets, etc.

It is understandable that different scenes have different image processing requirements. During the intelligent analysis of the image to be analyzed, the intelligent analysis device 140 can detect the target object and identify the target object based on the image to be analyzed. For example, according to the image to be analyzed, detect whether there is a target object in the target scene, and the location of the existing target object; for another example, identify the specific target object in the target scene according to the image to be analyzed, and identify the category of the target object Attribute information, etc. The target object may be a human face, a vehicle, a license plate, or other objects or objects.

Specifically, when performing intelligent analysis based on the image to be analyzed, the intelligent analysis device 140 may analyze the image to be analyzed based on a specific algorithm to perform image processing on the target scene, or, by using a neural network model, analyze the image to be analyzed to analyze the target scene It is reasonable to do image processing.

Optionally, in order to improve the accuracy of information perception, during the intelligent analysis of the image to be analyzed, the intelligent analysis device 140 may perform feature enhancement processing on the feature image before analyzing the feature image corresponding to the image to be analyzed.

Correspondingly, the intelligent analysis device 140 performs intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed, including:

Obtaining corresponding feature images from the image to be analyzed and performing feature enhancement processing to obtain enhanced feature images;

Based on the enhanced feature image, an intelligent analysis result corresponding to the image to be analyzed is obtained, and the intelligent analysis result includes an interest target contained in the image to be analyzed and/or location information of the interest target.

It should be noted that in the process of intelligent analysis, one or more frames of feature images can be generated, and then each frame of feature images is analyzed to obtain the results of the intelligent analysis. In order to improve the accuracy of information perception, before analyzing any frame of the feature image, the feature image can be subjected to feature enhancement processing.

It should be noted that there are multiple processing methods for feature enhancement processing. Exemplarily, in a specific implementation manner, the feature enhancement processing includes extreme value enhancement processing, where the extreme value enhancement processing is specifically: processing of localized extreme value filtering on the feature image. The so-called extreme value may be a maximum value or a minimum value.

Optionally, the processing of the extreme value enhancement processing includes: dividing the feature image into blocks to obtain multiple image blocks; for each image block, maximizing the pixels included in the image block, It is determined to be the processing result corresponding to the image block; each processing result is combined to obtain the image after the extreme value enhancement processing.

When the feature image is divided into blocks, there may be overlap between the image blocks. In addition, the number of image blocks is the resolution of the image after extreme value enhancement processing. It should be noted that the number of image blocks can be set according to the actual situation, which is not limited in this application. For ease of understanding, taking the number of image blocks as 100 as an example, the process of extreme value enhancement processing is introduced:

When the number of the image blocks is 100, for each image block in the 100 blocks, the maximum value in the pixels included in the image block is determined as the processing result corresponding to the image block, and 100 processing results are obtained; The 100 processing results are merged according to the positional relationship of the image blocks to obtain an image containing 100 pixels.

It should be emphasized that the specific implementation method of the extreme value enhancement processing is not limited to the above-mentioned method. For example: you can traverse each pixel position, for each pixel position, determine a maximum value for the pixel position, and use the maximum value to update the pixel value of the pixel position, where, for any pixel position The way of the large value may be: determining each pixel position adjacent to the pixel position, determining each adjacent pixel position and the maximum value of the pixels in the pixel position, and using the determined maximum value as the maximum value of the pixel position Great value.

In a second aspect, corresponding to the above image processing system, an embodiment of the present application further provides an image processing method.

It should be noted that the image processing method provided in the embodiments of the present application can be applied to an electronic device having the functions of an image processor, an intelligent analysis device, and a control unit. The functions performed by the electronic device are the same as the images in the above embodiments The functions performed by the processor and the intelligent analysis device are the same, and the specific implementation of the image processing method may refer to the foregoing embodiments.

As shown in FIG. 8, an image processing method provided by an embodiment of the present application may include:

S801: Obtain a first image signal and a second image signal output by the image sensor;

Wherein, the image sensor generates and outputs a first image signal and a second image signal through multiple exposures, wherein the first image signal is an image signal generated according to a first preset exposure, and the second image signal is According to the image signal generated by the second preset exposure, the first preset exposure and the second preset exposure are two of the multiple exposures; at the exposure time of the first preset exposure The fill light device in the section performs near infrared fill light, and the fill light device does not perform near infrared fill light in the exposure time period of the second preset exposure.

S802, generate a first target image according to the first image signal, and generate a second target image according to the second image signal;

S803, obtaining an image to be analyzed from the first target image and the second target image;

S804: Perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

Wherein, the image sensor includes a plurality of photosensitive channels, the plurality of photosensitive channels includes an IR photosensitive channel, and further includes at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and a W photosensitive channel. The photosensitive channel generates and outputs the first image signal and the second image signal through the multiple exposures;

Exemplarily, the image sensor is an RGBIR sensor, an RGBWIR sensor, an RWBIR sensor, an RWGIR sensor, or a BWGIR sensor;

Among them, R represents R photosensitive channel, G represents G photosensitive channel, B represents B photosensitive channel, IR represents IR photosensitive channel, and W represents all-pass photosensitive channel.

Optionally, the acquiring the image to be analyzed from the first target image and the second target image includes:

Acquiring the first target image from the first target image and the second target image, and determining the first target image as the image to be analyzed; or,

Obtain the second target image from the first target image and the second target image, and determine the second target image as the image to be analyzed.

When the received selection signal is switched to the first selection signal, acquire the first target image, and determine the first target image as the image to be analyzed;

When the received selection signal is switched to the second selection signal, the second target image is acquired, and the second target image is determined as the image to be analyzed.

Optionally, an image processing method provided by an embodiment of the present application further includes:

Sending a first control signal to the fill light device, wherein the first control signal is used to control the fill light device to perform near-infrared fill light during the exposure period of the first preset exposure; In the exposure time period of the second preset exposure, near infrared fill light is not performed.

Optionally, the first control signal is used to instruct the fill light device to perform the fill time of near infrared fill light, specifically, during the exposure period of the first preset exposure, perform near infrared fill light The start time of is not earlier than the exposure start time of the first preset exposure, and the end time of performing near infrared fill light is not later than the exposure end time of the first preset exposure.

Optionally, the first control signal is also used to indicate the number of fill lights of the fill light device, specifically, the number of near infrared fill lights of the fill light device per unit time length is lower than that of the image sensor The number of exposures per unit length of time, in which one or more exposures are spaced every two adjacent periods of near infrared fill light.

Optionally, the multiple exposures of the image sensor include odd exposures and even exposures; wherein,

The first preset exposure is one of odd exposures, and the second preset exposure is one of even exposures; or,

The first preset exposure is one of the even exposures, and the second preset exposure is one of the odd exposures; or,

The first preset exposure is one of the specified odd exposures, and the second preset exposure is one of the other exposures other than the specified odd exposures; or,

The first preset exposure is one of the specified even-numbered exposures, and the second preset exposure is one of the other exposures other than the specified even-numbered exposures.

Obtain the brightness information corresponding to the image to be analyzed, and adjust the first fill light parameter used by the fill light device to the second fill light parameter according to the brightness information corresponding to the image to be analyzed, and convert the image The first exposure parameter used by the sensor exposure is adjusted to the second exposure parameter;

Sending the second fill light parameter to the fill light device, and synchronously sending the second exposure parameter to the image sensor, so that the fill light device receives the second fill light parameter, based on the first Two fill light parameters, performing near-infrared fill light during the exposure period of the first preset exposure, and the image sensor receives the second exposure parameter, and performs the multiple exposure according to the second exposure parameter .

Optionally, the acquiring brightness information corresponding to the image to be analyzed includes:

Optionally, the adjusting the first exposure parameter used for exposure of the image sensor to the second exposure parameter according to the brightness information corresponding to the image to be analyzed includes:

When the brightness information is higher than the first predetermined threshold, the first exposure parameter used by the image sensor for exposure is adjusted down to obtain the second exposure parameter;

When the brightness information is lower than the second predetermined threshold, increase the first exposure parameter to obtain the second exposure parameter;

Wherein the first predetermined threshold is higher than the second predetermined threshold.

Optionally, the adjusting the first fill light parameter used by the fill light device to the second fill light parameter according to the brightness information corresponding to the image to be analyzed includes:

When the brightness information is higher than a third predetermined threshold, the first fill light parameter used by the fill light of the fill light device is adjusted down to obtain the second fill light parameter;

When the brightness information is lower than a fourth predetermined threshold, adjust the first fill light parameter to obtain the second fill light parameter;

Wherein the third predetermined threshold is higher than the fourth predetermined threshold.

Optionally, generating the first target image according to the first image signal includes:

Optionally, the obtaining the first target image according to the difference processed image includes:

Determining the image after the difference processing as the first target image; or,

The image after the difference processing is subjected to image enhancement processing, and the image after the image enhancement processing is determined as the first target image.

Optionally, the interpolating the channel values of the plurality of pixels included in the neighborhood of each pixel of the first image signal in an average manner includes:

Respectively interpolating the channel values of each photosensitive channel of the first image signal to obtain each channel value after interpolation processing of each photosensitive channel corresponding to each pixel in the first image signal;

An average value is obtained for each channel value after interpolation processing of each photosensitive channel corresponding to each pixel to obtain the image after the difference processing.

Optionally, the generating the second target image according to the second image signal includes:

Traverse the second image signal, adjust the channel value of each non-IR photosensitive channel traversed, respectively interpolate each channel value of each non-IR photosensitive channel after the channel value adjustment, and process according to the difference Image to obtain the second target image; wherein the channel value adjustment for each non-IR photosensitive channel specifically includes: subtracting the IR parameter corresponding to the corresponding pixel position of each channel value before the adjustment of the non-IR photosensitive channel Value, the IR parameter value is the product of the IR value of the corresponding pixel position and a preset correction value, and the IR value is the IR value sensed by the IR light-sensing channel at the corresponding pixel position.

Optionally, performing intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed includes:

It should be emphasized that the steps of fill light control and exposure control can be performed by the image processor or the intelligent analysis device, or by the controller in the device integrating the image processor, the intelligent analysis device and the controller , This is reasonable.

In addition, for the specific implementation and explanation of each step of the image processing method provided in the embodiments of the present application, reference may be made to the corresponding description content in the image processing system provided in the first aspect above, and details are not described herein again.

Corresponding to the above method embodiment, an embodiment of the present application further provides an image processing device. As shown in FIG. 9, an image processing device provided by an embodiment of the present application may include:

The image signal obtaining module 910 is configured to obtain the first image signal and the second image signal output by the image sensor, wherein the image sensor generates and outputs the first image signal and the second image signal through multiple exposures, the first The image signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are the Two of the multiple exposures; the fill light device performs near-infrared fill light during the exposure period of the first preset exposure, and the fill light device during the exposure period of the second preset exposure No near infrared fill light;

An image generation module 920, configured to generate a first target image based on the first image signal, and generate a second target image based on the second image signal;

An image selection module 930, configured to obtain an image to be analyzed from the first target image and the second target image;

The image analysis module 940 is configured to perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.

Optionally, the image selection module 930 is configured to: obtain the first target image from the first target image and the second target image, and determine the first target image as the image to be analyzed Or, obtain the second target image from the first target image and the second target image, and determine the second target image as the image to be analyzed.

Optionally, the image selection module 930 is configured to: when the received selection signal is switched to the first selection signal, acquire the first target image and determine the first target image as the image to be analyzed; When the received selection signal is switched to the second selection signal, the second target image is acquired, and the second target image is determined as the image to be analyzed.

Optionally, an image processing device provided by an embodiment of the present application further includes:

The signal sending module is used to send a first control signal to the fill light device, and the first control signal is used to control the fill light device to perform near infrared fill light during the exposure period of the first preset exposure , During the exposure period of the second preset exposure, near infrared fill light is not performed.

Optionally, the multiple exposures of the image sensor include odd exposures and even exposures; the first control signal is used to instruct the fill light device to perform near infrared fill light in the first preset exposure; among them,

The parameter adjustment module is used to obtain the brightness information corresponding to the image to be analyzed; according to the brightness information corresponding to the image to be analyzed, the first fill light parameter used by the fill light of the fill light device is adjusted to the second fill light Parameter, adjusting the first exposure parameter used by the image sensor exposure to the second exposure parameter; and sending the second fill light parameter to the fill light device, and synchronously sending the second exposure to the image sensor Parameters such that the fill light device receives the second fill light parameter, performs near-infrared fill light during the exposure period of the first preset exposure according to the second fill light parameter, and the The image sensor receives the second exposure parameter, and performs the multiple exposure according to the second exposure parameter.

Optionally, the parameter adjustment module acquiring brightness information corresponding to the image to be analyzed includes:

Optionally, the parameter adjustment module adjusts the first exposure parameter used by the image sensor to the second exposure parameter according to the brightness information corresponding to the image to be analyzed, including:

Optionally, the parameter adjustment module adjusts the first fill light parameter used by the fill light device to the second fill light parameter according to the brightness information corresponding to the image to be analyzed, including:

When the brightness information is higher than a third predetermined threshold, the first fill-light parameter used by fill-light of the fill-light device is adjusted down to obtain the second fill-light parameter;

When the brightness information is lower than a fourth predetermined threshold, the first fill-in light parameter is adjusted up to obtain the second fill-in light parameter;

Optionally, the image generation module 920 generates the first target image according to the first image signal, including:

Optionally, the image generation module 920 generating the second target image according to the second image signal includes:

Optionally, the image generation module 920 generates the second target image according to the second image signal, including:

Traverse the wide dynamic image, adjust the channel value of each non-IR photosensitive channel traversed, respectively interpolate each channel value of each non-IR photosensitive channel after the channel value adjustment, and process the image according to the difference To obtain the second target image.

Optionally, the image analysis module 940 performs intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed, including:

Corresponding to the above method, an embodiment of the present application further provides an electronic device. As shown in FIG. 10, the electronic device includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, where the processor 1001 communicates The interface 1002 and the memory 1003 communicate with each other through the communication bus 1004,

Memory 1003, used to store computer programs;

The processor 1001 is used to implement an image processing method provided in the embodiments of the present application when executing the program stored in the memory 1003.

The communication bus mentioned in the above electronic device may be a peripheral component interconnection standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard structure (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, and a control bus. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the above electronic device and other devices.

The memory may include random access memory (Random Access Memory, RAM), or non-volatile memory (Non-Volatile Memory, NVM), for example, at least one disk memory. Optionally, the memory may also be at least one storage device located away from the foregoing processor.

The aforementioned processor may be a general-purpose processor, including a central processor (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), dedicated integration Circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

In addition, based on an image processing method provided by an embodiment of the present application, an embodiment of the present application also provides a computer-readable storage medium in which a computer program is stored, and the computer program is processed The image processing method provided by the embodiments of the present application is implemented when the processor is executed.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is any such actual relationship or order. Moreover, the terms "include", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also those not explicitly listed Or other elements that are inherent to this process, method, article, or equipment. Without more restrictions, the element defined by the sentence "include one..." does not exclude that there are other identical elements in the process, method, article or equipment that includes the element.

The embodiments in this specification are described in a related manner. The same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method embodiment. The above are only the preferred embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in this application Within the scope of protection.

Claims

An image processing system, characterized in that it includes:

An image sensor for generating and outputting a first image signal and a second image signal through multiple exposures, wherein the first image signal is an image signal generated according to a first preset exposure, and the second image signal is based on An image signal generated by a second preset exposure, the first preset exposure and the second preset exposure are two of the multiple exposures;

The fill light device is used to perform near-infrared fill light in a strobe manner, specifically: the fill light device performs near-infrared fill light during the exposure period of the first preset exposure, and the second preset light No near-infrared fill light is applied during the exposure period of exposure;

An image processor for receiving the first image signal and the second image signal output by the image sensor, generating a first target image based on the first image signal, and generating a second target image based on the second image signal Target image

An intelligent analysis device is configured to obtain an image to be analyzed from the first target image and the second target image, and perform an intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.
The system according to claim 1, wherein the acquiring the image to be analyzed from the first target image and the second target image comprises:

Acquiring the first target image from the first target image and the second target image, and determining the first target image as an image to be analyzed; or,

Obtain the second target image from the first target image and the second target image, and determine the second target image as an image to be analyzed.
The system according to claim 1, wherein the acquiring the image to be analyzed from the first target image and the second target image comprises:

When the received selection signal is switched to the first selection signal, acquire the first target image and determine the first target image as the image to be analyzed;

When the received selection signal is switched to the second selection signal, the second target image is acquired, and the second target image is determined as the image to be analyzed.
The system according to claim 1, wherein the image processor is further configured to output the second target image.
The system according to claim 1, wherein the image sensor includes a plurality of photosensitive channels, the plurality of photosensitive channels includes an IR photosensitive channel, and further includes an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and a W photosensitive channel At least two of the channels, the multiple photosensitive channels generate and output the first image signal and the second image signal through the multiple exposures;

Among them, R photosensitive channel is used for sensing light in the red and near infrared bands, G photosensitive channel is used for sensing light in the green and near infrared bands, and B photosensitive channel is used for sensing the blue and near infrared bands. Light, IR means infrared sensitive channel, used to sense light in near infrared band, W means all-pass sensitive channel, used to sense light in full band.
The system according to claim 5, wherein the image sensor is an RGBIR sensor, an RGBWIR sensor, an RWBIR sensor, an RWGIR sensor, or a BWGIR sensor;

Among them, R represents R photosensitive channel, G represents G photosensitive channel, B represents B photosensitive channel, IR represents IR photosensitive channel, and W represents all-pass photosensitive channel.
The system according to any one of claims 1 to 6, wherein the fill light device performs near infrared fill light during the exposure time period of the first preset exposure, specifically:

In the exposure time period of the first preset exposure, the start time of performing near infrared fill light is not earlier than the start time of exposure of the first preset exposure, and the end time of performing near infrared fill light is not later than the The exposure end time of the first preset exposure.
The system according to claim 7, wherein the number of near-infrared fill lights of the fill light device per unit time length is lower than the number of exposures of the image sensor within a unit time length, wherein each adjacent two In the time period of sub-near infrared fill light, one or more exposures are performed at intervals.
The system according to claim 7, wherein the multiple exposures include odd exposures and even exposures;

The first preset exposure is one of odd exposures, and the second preset exposure is one of even exposures; or,

The first preset exposure is one of even exposures, and the second preset exposure is one of odd exposures; or,

The first preset exposure is one of the specified odd exposures, and the second preset exposure is one of the other exposures other than the specified odd exposures; or,

The first preset exposure is one of the specified even-numbered exposures, and the second preset exposure is one of the other exposures other than the specified even-numbered exposures.
The system of claim 1, wherein:

The multiple exposure of the image sensor is specifically: the image sensor performs the multiple exposure according to a first exposure parameter, wherein the parameter type of the first exposure parameter includes at least one of exposure time and exposure gain;

The fill light device performs near-infrared fill light during the exposure time period of the first preset exposure, specifically: the fill light device performs the exposure time of the first preset exposure according to the first fill light parameter In the segment, near infrared fill light is performed, wherein the parameter type of the first fill light parameter includes at least one of fill light intensity and fill light concentration.
The system according to claim 10, further comprising:

The control unit is configured to obtain brightness information corresponding to the image to be analyzed, adjust the first fill light parameter to a second fill light parameter according to the brightness information corresponding to the image to be analyzed, and adjust the first exposure parameter Adjust to the second exposure parameter; and send the second fill light parameter to the fill light device, and synchronously send the second exposure parameter to the image sensor;

The fill light device performs near-infrared fill light during the exposure period of the first preset exposure, specifically: the fill light device receives the second fill light parameter from the control unit according to the A second fill light parameter, performing near-infrared fill light during the exposure period of the first preset exposure;

The multiple exposure of the image sensor is specifically: the image sensor receives the second exposure parameter from the control unit, and performs the multiple exposure according to the second exposure parameter.
The system according to claim 11, wherein the acquiring brightness information corresponding to the image to be analyzed includes:

When the intelligent analysis result corresponding to the image to be analyzed includes position information of an interest target included in the image to be analyzed, determining at least one target area in the image to be analyzed according to the position information;

The average brightness of the at least one target area is determined as brightness information corresponding to the image to be analyzed.
The system according to claim 11, wherein the adjusting the first exposure parameter to the second exposure parameter according to the brightness information corresponding to the image to be analyzed includes:

When the brightness information is higher than the first predetermined threshold, lower the first exposure parameter to obtain a second exposure parameter;

When the brightness information is lower than the second predetermined threshold, increase the first exposure parameter to obtain the second exposure parameter;

Wherein the first predetermined threshold is higher than the second predetermined threshold.
The system according to claim 11, wherein the adjusting the first fill light parameter to the second fill light parameter according to the brightness information corresponding to the image to be analyzed includes:

When the brightness information is higher than a third predetermined threshold, the first fill light parameter is adjusted down to obtain a second fill light parameter;

When the brightness information is lower than the fourth predetermined threshold, adjust the first fill light parameter to obtain a second fill light parameter;

Wherein the third predetermined threshold is higher than the fourth predetermined threshold.
The system according to claim 1, wherein generating the first target image according to the first image signal includes:

According to the channel values of a plurality of pixels included in the neighborhood of each pixel of the first image signal, interpolation processing is performed in an averaging manner, and the first target image is obtained according to the difference-processed image.
The system according to claim 15, wherein the first target image obtained by the image processed according to the difference includes:

Determining the image after the difference processing as the first target image; or,

The image after the difference processing is subjected to image enhancement processing, and the image after the image enhancement processing is determined as the first target image.
The system according to claim 15, wherein the channel values of the plurality of pixels included in the neighborhood of each pixel of the first image signal are interpolated in an average manner, including:

Respectively interpolating the channel values of each photosensitive channel of the first image signal to obtain each channel value after interpolation processing of each photosensitive channel corresponding to each pixel in the first image signal;

An average value is obtained for each channel value after interpolation processing of each photosensitive channel corresponding to each pixel to obtain the image after the difference processing.
The system according to claim 1, wherein the generating the second target image according to the second image signal comprises:

Traverse the second image signal, adjust the channel value of each non-IR photosensitive channel traversed, respectively interpolate each channel value of each non-IR photosensitive channel after the channel value adjustment, and process according to the difference Image to get the second target image;

Wherein, the channel value adjustment for each non-IR photosensitive channel specifically includes: subtracting the IR parameter value corresponding to the corresponding pixel position of each channel value before the adjustment of the non-IR photosensitive channel, and the IR parameter value is the corresponding pixel A product of an IR value of a position and a preset correction value, and the IR value is an IR value sensed by the IR photosensitive channel at the position of the corresponding pixel.
The system according to claim 1, wherein the generating the second target image according to the second image signal comprises:

Obtaining M frame second image signals including the current second image signal, performing wide dynamic synthesis processing on the M frame second image signals to obtain a wide dynamic image, and performing infrared removal processing on the wide dynamic image to obtain the Second target image;

Wherein, the infrared removal processing includes:

Traverse the wide dynamic image, adjust the channel value of each non-IR photosensitive channel traversed, respectively interpolate the channel value of each non-IR photosensitive channel after the channel value adjustment, and process the image according to the difference To get the second target image.
The system according to claim 1, wherein performing intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed includes:

Obtaining corresponding feature images from the image to be analyzed and performing feature enhancement processing to obtain enhanced feature images;

Based on the enhanced feature image, an intelligent analysis result corresponding to the image to be analyzed is obtained, and the intelligent analysis result includes an interest target contained in the image to be analyzed and/or location information of the interest target.
The system according to claim 20, wherein the feature enhancement process includes an extreme value enhancement process,

Wherein, the extremum enhancement processing is specifically: a process of performing localized extremum filtering on the feature image.
The system according to claim 21, wherein the process of the extreme value enhancement processing includes:

Block the feature image to obtain multiple image blocks; for each image block, determine the maximum value of the pixels included in the image block as the processing result corresponding to the image block; perform each processing result Combine to get the image after extreme value enhancement processing.
An image processing method, characterized in that it includes:

Obtaining a first image signal and a second image signal output by an image sensor, wherein the image sensor generates and outputs a first image signal and a second image signal through multiple exposures, the first image signal is based on a first preset An image signal generated by exposure, the second image signal is an image signal generated according to a second preset exposure, and the first preset exposure and the second preset exposure are two of the multiple exposures Exposure; during the exposure period of the first preset exposure, the fill light device performs near infrared fill light, and during the exposure period of the second preset exposure, the fill light device does not perform near infrared fill light;

Generating a first target image based on the first image signal, and generating a second target image based on the second image signal;

Obtaining an image to be analyzed from the first target image and the second target image;

Perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.
The method according to claim 23, wherein the acquiring the image to be analyzed from the first target image and the second target image comprises:

Acquiring the first target image from the first target image and the second target image, and determining the first target image as an image to be analyzed; or,

Obtain the second target image from the first target image and the second target image, and determine the second target image as an image to be analyzed.
The method according to claim 23, wherein the acquiring the image to be analyzed from the first target image and the second target image comprises:

When the received selection signal is switched to the first selection signal, acquire the first target image and determine the first target image as the image to be analyzed;

When the received selection signal is switched to the second selection signal, the second target image is acquired, and the second target image is determined as the image to be analyzed.
The method according to any one of claims 23 to 25, further comprising:

Sending a first control signal to the fill light device, wherein the first control signal is used to control the fill light device to perform near-infrared fill light during the exposure period of the first preset exposure; In the exposure time period of the second preset exposure, near infrared fill light is not performed.
The method according to claim 26, wherein the first control signal is used to instruct the fill light device to perform a fill time of near infrared fill light, specifically, the exposure at the first preset exposure In the time period, the start time of performing near infrared fill light is not earlier than the exposure start time of the first preset exposure, and the end time of performing near infrared fill light is not later than the exposure end time of the first preset exposure.
The method according to claim 27, wherein the first control signal is further used to indicate the number of fill times of the fill light device, specifically, the near infrared fill of the fill light device within a unit time length The number of times of light is lower than the number of times of exposure of the image sensor within a unit time length, wherein, during each time period of two adjacent near infrared fill lights, one or more exposures are spaced apart.
The method according to claim 26, wherein the multiple exposures of the image sensor include odd exposures and even exposures; wherein,

The first preset exposure is one of odd exposures, and the second preset exposure is one of even exposures; or,

The first preset exposure is one of the even exposures, and the second preset exposure is one of the odd exposures; or,

The first preset exposure is one of the specified odd exposures, and the second preset exposure is one of the other exposures other than the specified odd exposures; or,

The first preset exposure is one of the specified even-numbered exposures, and the second preset exposure is one of the other exposures other than the specified even-numbered exposures.
The method according to any one of claims 23 to 25, further comprising:

Obtain the brightness information corresponding to the image to be analyzed, and adjust the first fill light parameter used by the fill light device to the second fill light parameter according to the brightness information corresponding to the image to be analyzed, and convert the image The first exposure parameter used by the sensor exposure is adjusted to the second exposure parameter;

Sending the second fill light parameter to the fill light device, and synchronously sending the second exposure parameter to the image sensor, so that the fill light device receives the second fill light parameter, based on the first Two fill light parameters, performing near-infrared fill light during the exposure period of the first preset exposure, and the image sensor receives the second exposure parameter, and performs the multiple exposure according to the second exposure parameter .
The method according to claim 30, wherein the acquiring brightness information corresponding to the image to be analyzed includes:

When the intelligent analysis result corresponding to the image to be analyzed includes the position information of the target of interest included in the image to be analyzed, determining at least one target area in the image to be analyzed according to the position information;

The average brightness of the at least one target area is determined as brightness information corresponding to the image to be analyzed.
The method according to claim 23, wherein performing intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed includes:

Obtaining corresponding feature images from the image to be analyzed and performing feature enhancement processing to obtain enhanced feature images;

Based on the enhanced feature image, an intelligent analysis result corresponding to the image to be analyzed is obtained, and the intelligent analysis result includes an interest target contained in the image to be analyzed and/or location information of the interest target.
An image processing device, characterized in that it includes:

An image signal obtaining module for obtaining a first image signal and a second image signal output by an image sensor, wherein the image sensor generates and outputs a first image signal and a second image signal through multiple exposures, the first image The signal is an image signal generated according to a first preset exposure, the second image signal is an image signal generated according to a second preset exposure, the first preset exposure and the second preset exposure are the multiple Two of the two exposures; during the exposure period of the first preset exposure, the fill light device performs near-infrared fill light, and during the exposure period of the second preset exposure, the fill light device does not Perform near infrared fill light;

An image generating module, configured to generate a first target image based on the first image signal, and generate a second target image based on the second image signal;

An image selection module, configured to obtain an image to be analyzed from the first target image and the second target image;

The image analysis module is configured to perform intelligent analysis on the image to be analyzed to obtain an intelligent analysis result corresponding to the image to be analyzed.
An electronic device characterized by comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

Memory, used to store computer programs;

The processor, when used to execute the program stored on the memory, implements the method steps of any one of claims 23-32.
A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps of any one of claims 23-32 are realized.