WO2020051898A1 - 双光图像自动曝光方法、装置、双光图像相机及机器存储介质 - Google Patents

双光图像自动曝光方法、装置、双光图像相机及机器存储介质 Download PDF

Info

Publication number
WO2020051898A1
WO2020051898A1 PCT/CN2018/105802 CN2018105802W WO2020051898A1 WO 2020051898 A1 WO2020051898 A1 WO 2020051898A1 CN 2018105802 W CN2018105802 W CN 2018105802W WO 2020051898 A1 WO2020051898 A1 WO 2020051898A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
exposure
infrared
value
visible light
Prior art date
Application number
PCT/CN2018/105802
Other languages
English (en)
French (fr)
Inventor
陈玉麟
孙岳
Original Assignee
浙江宇视科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江宇视科技有限公司 filed Critical 浙江宇视科技有限公司
Priority to HUE18933406A priority Critical patent/HUE066105T2/hu
Priority to PCT/CN2018/105802 priority patent/WO2020051898A1/zh
Priority to PL18933406.3T priority patent/PL3852350T3/pl
Priority to US17/271,407 priority patent/US11689785B2/en
Priority to CN201880096822.0A priority patent/CN112602316B/zh
Priority to EP18933406.3A priority patent/EP3852350B1/en
Priority to ES18933406T priority patent/ES2971615T3/es
Publication of WO2020051898A1 publication Critical patent/WO2020051898A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/72—Combination of two or more compensation controls
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means

Definitions

  • the present application relates to the technical field of cameras, and in particular, to a dual-light image automatic exposure method, device, dual-light image camera, and machine storage medium.
  • a dual-light image camera also known as a bispectral camera
  • Traditional dual-spectral cameras generally use dual sensors, which use infrared sensors and visible light sensors to simultaneously capture infrared and visible light images in the same scene, and then use image fusion technology to effectively fuse infrared and visible light images to obtain high-quality color fusion. image.
  • this type of solution generally uses a dual lens or a single lens with a more complicated hardware structure to perform logical beam splitting to separate infrared and visible light, and to image the infrared sensor and the visible light sensor separately, and then image the fusion of the two.
  • the hardware cost of traditional bispectral cameras is usually higher.
  • the purpose of this application includes, for example, providing a dual-light image automatic exposure method, device, dual-light image camera, and machine storage medium, which provides an effective exposure control method for a single-sensor dual-light camera. Helps get better exposure.
  • an embodiment of the present application provides a dual-light image automatic exposure method, which is characterized in that the method applies a dual-light image camera, and the dual-light image camera includes an image sensor and a processor connected in sequence, wherein: The number of the image sensors is one, and the dual-light image camera is further configured with an infrared cut-off filter; the method is executed by the processor, and the method includes: obtaining an original image collected by the image sensor; Logically split the original image to obtain an infrared image and a visible light image; determine whether the infrared cut-off filter is currently used; if the infrared cut-off filter is used, use a single light exposure algorithm to perform exposure processing on the visible light image, To obtain a visible light image that meets the target exposure effect; if the infrared cut-off filter is not used, the infrared image and the visible light image are subjected to exposure processing by using a dual light exposure algorithm to obtain a visible light image and An infrared image conforming to
  • the step of determining whether to use the infrared cutoff filter includes: determining whether the current ambient brightness is greater than a preset brightness; if yes, determining to use the infrared cutoff filter; if not, determining The infrared cut-off filter is not used.
  • the step of performing exposure processing on the infrared image and the visible light image by using a dual light exposure algorithm includes: obtaining a first brightness value of the visible light image and a first brightness value of the infrared image. Two brightness values; obtaining a first exposure target value of the visible light image and a second exposure target value of the infrared image; and calculating a first exposure value of the visible light image based on the first brightness value and the first exposure target value.
  • the image and the infrared image are subjected to exposure processing.
  • the step of calculating a first exposure parameter of the visible light image according to the first brightness value and the first exposure target value includes: according to the first brightness value and the Describing the difference between the first exposure target values, looking up a preset first difference comparison table, and determining a first exposure parameter of the visible light image; according to the second brightness value and the second exposure target value,
  • the step of calculating a second exposure parameter of the infrared image includes: searching a preset second difference comparison table according to a difference between the second brightness value and the second exposure target value to determine the second difference value comparison table.
  • the second exposure parameter of the infrared image wherein the first difference value comparison table is the same as or different from the second difference value comparison table.
  • the first exposure parameter includes a first shutter value, a first sensor gain, and a first ISP gain
  • the second exposure parameter includes a second shutter value, a second sensor gain, and a first Two ISP gains.
  • the dual-light image camera is further configured with a white light lamp and an infrared lamp; the first exposure parameter further includes the intensity of the white light lamp; and the second exposure parameter further includes the intensity of the infrared lamp.
  • the step of performing exposure processing on the visible light image and the infrared image according to the first exposure parameter and the second exposure parameter includes: according to the first shutter value And the second shutter value to determine an effective shutter value shared by the visible light image and the infrared image; and determine that the visible light image and the infrared image are shared according to the first sensor gain and the second sensor gain According to the effective shutter value, the effective sensor gain, the first ISP gain, and the white light intensity, performing exposure processing on the visible light image; according to the effective shutter value, the effective Senor gain, the second ISP gain, and the infrared lamp intensity, perform exposure processing on the infrared image.
  • the step of determining an effective shutter value shared by the visible light image and the infrared image according to the first shutter value and the second shutter value includes: selecting the first shutter value A minimum value of the shutter value and the second shutter value, and determining the selected minimum value as an effective shutter value common to the visible light image and the infrared image; or selecting the first shutter value and the A maximum value of the second shutter value, and determining the selected maximum value as an effective shutter value shared by the visible light image and the infrared image; or calculating a value of the first shutter value and the second shutter value An average value, determining the average value as an effective shutter value shared by the visible light image and the infrared image; or calculating a weighted shutter of the first shutter value and the second shutter value using a preset weighting algorithm The weighted shutter value is determined as an effective shutter value shared by the visible light image and the infrared image.
  • the step of determining an effective sensor gain common to the visible image and the infrared image according to the first sensor gain and the second sensor gain includes: selecting the first sensor gain a minimum value of the sensor gain and the second sensor gain, and determining the selected minimum value as an effective sensor gain common to the visible light image and the infrared image; or selecting the first sensor gain and the A maximum value of the second sensor gain, and determining the selected maximum value as an effective sensor gain shared by the visible light image and the infrared image; or calculating the first sensor gain and the second sensor gain An average value, determining the average value as an effective sensor gain common to the visible image and the infrared image; or calculating a weighted gain of the first sensor gain and the second sensor gain by using a preset weighting algorithm , Determining the weighted gain as an effective sensor gain shared by the visible light image and the infrared image.
  • the step of performing an exposure process on the visible light image according to the effective shutter value, the effective sensor gain, the first ISP gain, and the white light intensity includes: : Preset a first control order of the effective shutter value, the effective sensor gain, the first ISP gain, and the white light intensity, and sequentially perform exposure processing on the visible light image according to the first control order Until the visible light image reaches the first exposure effect and stop; performing exposure processing on the infrared image according to the effective shutter value, the effective sensor gain, the second ISP gain, and the infrared light intensity
  • the step includes: presetting a second control order of the effective shutter value, the effective sensor gain, the second ISP gain, and the intensity of the infrared lamp, and sequentially ranking the infrared rays according to the second control order. The image is subjected to exposure processing until it stops when the infrared image reaches the second exposure effect.
  • the first exposure effect is that a difference between a screen brightness value of the visible light image after exposure processing and a first exposure target value of the visible light image is smaller than a preset first A threshold value;
  • the second exposure effect is that a difference between a screen brightness value of the infrared image after exposure processing and a second exposure target value of the infrared image is less than a preset second threshold value.
  • an embodiment of the present application further provides a dual-light image automatic exposure device, which is characterized in that the device uses a dual-light image camera, and the dual-light image camera includes an image sensor and a processor connected in sequence, wherein, The number of the image sensors is one, and the dual-light image camera is further configured with an infrared cut-off filter; the device is configured in the processor, and the device includes an original image acquisition module configured to acquire the Original image collected by the image sensor; a logical light splitting module configured to perform logical light splitting processing on the original image to obtain an infrared image and a visible light image; a filter determination module configured to determine whether the infrared cutoff filter is currently used; single light An exposure control module configured to perform exposure processing on the visible light image using a single light exposure algorithm if the infrared cutoff filter is used to obtain a visible light image that matches the target exposure effect; a dual light exposure control module configured to not use The infrared cut-off filter uses a dual light exposure algorithm to Vis
  • the filter determination module is configured to: determine whether the current ambient brightness is greater than a preset brightness; if yes, determine to use the infrared cutoff filter; if not, determine not to use the infrared Cut filter.
  • the dual light exposure control module is configured to: obtain a first brightness value of the visible light image and a second brightness value of the infrared image; obtain a first exposure of the visible light image A target value and a second exposure target value of the infrared image; calculating a first exposure parameter of the visible light image according to the first brightness value and the first exposure target value; The second exposure target value calculates a second exposure parameter of the infrared image; and performs exposure processing on the visible light image and the infrared image according to the first exposure parameter and the second exposure parameter.
  • an embodiment of the present application provides a dual-light image camera, including: an image sensor and a processor connected in sequence; wherein the number of the image sensors is one, and the dual-light image camera An infrared cut-off filter is also configured; the processor is provided with the dual-light image automatic exposure device according to any one of the second aspects.
  • a lens is further provided before the image sensor; wherein the lens is an infrared confocal lens.
  • the light transmittance of the lens is higher than a preset light transmittance.
  • the processor is further connected with a fill light; the fill light includes an infrared fill light and / or a white light fill light.
  • the image sensor is an RGB-IR sensor.
  • an embodiment of the present application provides a machine storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and executed by a processor, all the The machine-executable instructions cause the processor to perform the method of any one of the first aspects.
  • the embodiments of the present application provide a method and device for automatically exposing a dual-light image, a dual-light image camera, and a machine storage medium, which can perform logical spectroscopic processing on an original image collected by a single sensor of the dual-light image camera to obtain an infrared image and a visible light image.
  • a single light exposure algorithm is used to perform exposure processing on the visible light image
  • a dual-light exposure algorithm is used to perform exposure processing on the infrared image and the visible light image.
  • FIG. 1 is a schematic structural diagram of a dual-light image camera according to an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of another dual-light image camera according to an embodiment of the present application.
  • FIG. 3 shows a flowchart of a two-light image automatic exposure method according to an embodiment of the present application
  • FIG. 4 shows a flowchart of a two-light exposure control method according to an embodiment of the present application
  • FIG. 5 shows a schematic diagram of a two-light exposure control strategy according to an embodiment of the present application
  • FIG. 6 shows a structural block diagram of an automatic exposure apparatus according to an embodiment of the present application.
  • a method and device for automatically exposing a dual-light image, a dual-light image camera, and a machine storage medium provided in the embodiments of the application. The embodiments of the application are described in detail below.
  • a dual-light image camera also referred to as a dual-light image camera system
  • a dual-light image camera system proposed in the embodiments of the present application is first introduced below.
  • the dual-light image camera provided in the embodiment of the present application includes at least an image sensor and a processor connected in sequence; wherein the number of the image sensors is one. Therefore, the dual-light image camera provided in the embodiment of the present application may also be referred to as a dual-light image camera based on a single sensor. In addition, the dual-light image camera is also equipped with an infrared cut-off filter.
  • the processor in the dual-light image camera can execute the dual-light image automatic exposure method provided in the embodiment of the present application, or the dual-light image automatic exposure device provided in the embodiment of the present application can be provided thereon.
  • FIG. 1 a schematic structural diagram of a dual-light image camera shown in FIG. 1, which illustrates a lens 10, an image sensor 20, a processor 30, and a light supplementer 40 connected in this order.
  • the requirements of the dual-light image camera provided by the embodiment of the present application are relatively simple.
  • the lens should be an infrared confocal lens.
  • the light transmittance of the lens may be higher than a preset light transmittance.
  • the lens has good light transmittance to both visible and near-infrared light.
  • the infrared confocal lens can ensure that both visible light images and infrared images on the same focal plane can be clearly imaged.
  • the image sensor can be a single conventional RGB-IR sensor.
  • the RGB-IR sensor has the ability to integrate visible and infrared images. It also has a high signal-to-noise ratio and good permeability for visible images. Good color reproduction and other advantages.
  • the infrared image and the visible light image in different scenes can have better imaging effects.
  • the processor 30 can be implemented by at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA), or a central processing unit (CPU) , A graphics processor (GPU), or other types of processing devices with data processing capabilities and / or instruction execution capabilities, or a combination of one or more, and can control other components of the dual-light image camera (such as image sensors, Light, etc.) to perform the desired function.
  • DSP digital signal processor
  • FPGA field programmable gate array
  • PLA programmable logic array
  • CPU central processing unit
  • GPU graphics processor
  • processing devices with data processing capabilities and / or instruction execution capabilities or a combination of one or more, and can control other components of the dual-light image camera (such as image sensors, Light, etc.) to perform the desired function.
  • FIG. 2 a schematic structural diagram of another dual-light image camera shown in FIG. 2, which is based on FIG. 1, it also illustrates that the processor 30 includes a statistics unit 30 a and an automatic exposure control unit 30 b.
  • the statistical unit 30a illustrates a logical spectroscopic sub-unit, a first ISP gain sub-unit, and a second ISP gain sub-unit, and also illustrates RGB statistical data and IR statistical data processed by logical spectroscopic and ISP gain processing.
  • the logical light splitting subunit mainly converts the original image output by the RGB-IR sensor into an infrared grayscale image (hereinafter referred to as an infrared image) and a visible light color image (hereinafter referred to as a visible light image).
  • the first ISP gain sub-unit is used to amplify the digital signal of the visible light image, which mainly improves the overall brightness of the visible light image
  • the second ISP gain sub-unit is used to amplify the digital signal of the infrared image, which mainly improves the overall brightness of the infrared image.
  • the dual-light image camera provided in the embodiment of the present application supports separate processing of the infrared image and the visible light image, respectively.
  • the statistics unit 30a may be implemented by programming using logic devices such as FPGA, DSP, and CPU.
  • the automatic exposure control unit 30b may determine the exposure parameters of the visible light image and the infrared image based on the RGB statistical data and the IR statistical data. Specifically, according to the actual situation (such as whether the infrared cut-off filter is used), a dual-light exposure control strategy or a single-light exposure is adopted. Control Strategy. Specifically, the automatic exposure control unit can separately control some exposure parameters of the visible light image and the infrared image, that is, to achieve dual control of the visible light image and the infrared image. For example, the ISP gain can be implemented on FPGA, DSP and other logic devices.
  • the intensity of the fill light can also be controlled separately by the infrared fill light and the white light fill light, and because the dual-light image camera proposed in the embodiment of the present application uses a single sensor, the shutter parameters and sensor gain in the exposure parameters are only in the Effective on the sensor, so these parameters need to be controlled in a single way for the visible light image and the infrared image.
  • the automatic exposure control unit adjusts single-channel exposure parameters (such as shutter parameters, sensor gain) and dual-channel exposure parameters (such as fill light intensity, ISP gain), thereby affecting the image acquisition data of the RGB-IR sensor, that is, by continuous feedback Control and adjust, and finally make the infrared image and visible light image reach the best image effect.
  • the automatic exposure control unit 30b may be implemented by a logic period programming such as a CPU.
  • the light supplement may include an infrared light supplement and / or a white light supplement light.
  • the light supplement may include an infrared light supplement.
  • the full-spectrum fill light of the light lamp and the white light fill light may of course be only an infrared fill light, which is not limited here. Since the processor provided in the embodiment of the present application can logically split the original image to obtain a visible light image and an infrared image, it can be ensured that the infrared supplementary light does not affect the visible light, and the white supplementary light does not affect the infrared image.
  • infrared light can be used to enhance the brightness and imaging effect of infrared images in low-light environments
  • white light can be used to enhance the brightness and imaging effects of visible images, thereby ensuring that high-quality infrared images can still be obtained in low-light environments.
  • Visible light image which improves the final fusion image effect.
  • an embodiment of the present application further provides a dual-light image automatic exposure method
  • This method uses a dual-light image camera.
  • the dual-light image camera includes an image sensor and a processor connected in sequence.
  • the number of image sensors is one and the dual-light image camera is also configured with an infrared cut-off filter. Execution, referring to a flowchart of a two-light image automatic exposure method shown in FIG. 3, the method includes:
  • step S302 an original image acquired by the image sensor is acquired.
  • the image sensor may be an RGB-IB sensor.
  • Step S304 Perform logical spectroscopic processing on the original image to obtain an infrared image and a visible light image.
  • the statistical data of the infrared image and the statistical data of the visible light image may be obtained separately, and the brightness of the infrared image and the brightness of the visible light image may be calculated according to the corresponding statistical data. For example, the image is divided into blocks (divided into M * N blocks), and then the brightness data of each block is counted. The brightness data of all M * N blocks is the statistical data of the image.
  • step S306 it is determined whether an infrared cut-off filter is currently used. If it is used, go to step S308; if not, go to step S310.
  • the current gain value can be obtained according to a conventional gain calculation method adopted by most cameras. Generally, the current gain value is related to the image brightness value. In specific implementation, whether the infrared cut-off filter is used or not can be implemented by a filter switch.
  • a single light exposure algorithm is used to perform exposure processing on the visible light image to obtain a visible light image that meets the target exposure effect.
  • the single-light exposure algorithm can also be called a single-light exposure control strategy, that is, mainly performing exposure control for one type of light image (such as a visible light image). It can be understood that if an infrared cut-off filter is used, since the infrared cut-off filter can filter out the infrared light in the ambient light, adjusting the exposure parameters at this time has no effect on the infrared image, so in this case, the effect of adjusting the visible light image is the Lord.
  • a dual-light exposure algorithm is used to perform exposure processing on the infrared image and the visible light image to obtain a visible light image conforming to the first exposure effect and an infrared image conforming to the second exposure effect.
  • the first exposure effect obtained by processing the double-light exposure algorithm for the visible light image and the target exposure effect obtained by the single-light exposure algorithm may be the same or different, which is not limited herein.
  • the target exposure effect, the first exposure effect, and the second exposure effect can be flexibly set according to requirements.
  • the two-light exposure algorithm can also be referred to as a two-light exposure control strategy, that is, exposure processing is performed for two types of light images (such as an infrared image and a visible light image).
  • two types of light images such as an infrared image and a visible light image.
  • the infrared cut-off filter is not used, the ambient light entering the RGB-IR sensor contains both infrared and visible light, so it will affect both infrared and visible light images.
  • a single light exposure algorithm is used, The image exposure effect of the infrared image and the visible light image cannot be taken into consideration, so this embodiment proposes a dual light exposure algorithm to try to make the infrared image and the visible light image achieve a better image effect, and then the infrared image that achieves a better effect can be achieved. Image fusion with visible light image and display.
  • the above dual-light image automatic exposure method can perform logical spectroscopic processing on the original image collected by the image sensor of the dual-light image camera to obtain an infrared image and a visible light image, and adopts a single light when using an infrared cutoff filter.
  • the exposure algorithm performs exposure processing on the visible light image.
  • the dual light exposure algorithm is used to perform exposure processing on the infrared image and the visible light image. This method enables a single-sensor-based dual-light image camera to perform effective exposure processing on an image, and selectively selects different exposure algorithms according to the use of the infrared filter, thereby better improving the image exposure effect.
  • the current screen brightness value of the image may be counted using a central weighting or global metering method. According to the screen brightness value and a preset exposure target value, To determine whether to adjust the exposure. If the difference is greater than a preset threshold, adjust the exposure; otherwise, do not adjust the exposure.
  • a conventional automatic exposure Auto Exposure, AE
  • a dual light exposure algorithm when used to perform exposure processing on an infrared image and a visible light image, referring to a flowchart of a dual light exposure control method shown in FIG. 4, the following steps may be implemented:
  • Step S402 Acquire a first brightness value of a visible light image and a second brightness value of an infrared image.
  • An implementation manner may be, for example, first obtaining statistical data of an image, and then calculating a brightness value based on the statistical data.
  • Step S404 Obtain a first exposure target value of the visible light image and a second exposure target value of the infrared image.
  • the embodiment of the present application provides a method for obtaining an exposure target value. It is assumed that different gain steps such as 0dB, 6dB, 12dB, 18dB, etc. are set in advance, and according to different gain steps, preset values for different gain steps are preset.
  • the exposure target value is 50, 49, 47, 45, etc., and so on, and then the current gain value of the image is obtained, which gain range is determined by the current gain value, and then the current gain value is linearly interpolated to calculate the corresponding exposure.
  • Step S406 Calculate a first exposure parameter of the visible light image according to the first brightness value and the first exposure target value. For example, according to a difference between the first brightness value and the first exposure target value, a preset first difference comparison table is searched to determine a first exposure parameter of the visible light image.
  • Step S408 Calculate a second exposure parameter of the infrared image according to the second brightness value and the second exposure target value. For example, according to the difference between the second brightness value and the second exposure target value, a preset second difference comparison table is searched to determine a second exposure parameter of the infrared image.
  • the first difference comparison table in step S406 is the same as or different from the second difference comparison table in step S408.
  • Step S410 Perform exposure processing on the visible light image and the infrared image according to the first exposure parameter and the second exposure parameter.
  • the first exposure parameter may include a first shutter value, a first sensor gain, and a first ISP gain; the second exposure parameter may include a second shutter value, a second sensor gain, and a second ISP gain. If the dual-light image camera is further configured with a white light and an infrared light; the first exposure parameter may further include the intensity of the white light; the second exposure parameter may further include the intensity of the infrared light. In addition, the exposure parameter may include an aperture and the like.
  • the dual-light image camera can achieve two-way separate control of infrared and visible light images, and calculate the first ISP gain, second ISP gain, white light intensity, and infrared light intensity, respectively.
  • first ISP gain the first ISP gain
  • second ISP gain the second ISP gain
  • white light intensity the first ISP gain
  • infrared light intensity the second ISP gain
  • infrared light intensity the first ISP gain
  • infrared light intensity e.g., the dual-light image camera has only one sensor, it can only share a set of parameters to calculate the effective shutter value and effective sensor gain together.
  • an effective shutter value common to the visible light image and the infrared image may be determined according to the first shutter value and the second shutter value; Determining an effective sensor gain common to the visible light image and the infrared image according to the first sensor gain and the second sensor gain; thereby performing exposure processing on the visible light image according to the effective shutter value, effective sensor gain, first ISP gain, and white light intensity; and According to the effective shutter value, effective sensor gain, second ISP gain, and infrared lamp intensity, the infrared image is subjected to exposure processing.
  • the calculation method of various exposure parameters is similar.
  • the shutter is used as an example to explain the specific calculation process. For example, different brightness differences (that is, the difference between the screen brightness value of the image and the exposure target value) are set in advance. Different shutter sizes to form a brightness difference comparison table; then calculate the difference between the image brightness value and the exposure target value; finally, based on the currently calculated difference value and a preset brightness difference comparison table, you can determine The shutter size corresponding to the calculated difference.
  • different brightness differences that is, the difference between the screen brightness value of the image and the exposure target value
  • Different shutter sizes to form a brightness difference comparison table; then calculate the difference between the image brightness value and the exposure target value; finally, based on the currently calculated difference value and a preset brightness difference comparison table, you can determine The shutter size corresponding to the calculated difference.
  • a preset weighting algorithm is used to calculate the weighting gain of the first sensor gain and the second sensor gain, and the weighting gain is determined as an effective sensor gain shared by the visible light image and the infrared image.
  • the effective shutter value shared by the visible light image and the infrared image can be flexibly selected from any of the above methods 1 to 4, and the effective sensor gain can also be flexibly selected from any of the above methods 1 to 4, or Any other calculation method (not limited to the above four methods) can be used, which is not limited here.
  • the effective shutter value selection method may be the same as or different from the effective sensor gain selection method.
  • the control priority of different exposure parameters has different effects on the imaging effect of the image, therefore, when performing exposure processing on visible light images according to the effective shutter value, effective sensor gain, first ISP gain, and white light intensity, it can be set in advance A first control sequence that determines the effective shutter value, effective sensor gain, first ISP gain, and white light intensity, and sequentially performs exposure processing on the visible light image in accordance with the first control sequence, and stops until the visible light image reaches the first exposure effect.
  • the first exposure effect may be that a difference between a screen brightness value of the visible light image after exposure processing and a first exposure target value of the visible light image is less than a preset first threshold value.
  • the first control order of the visible light image (also referred to as the first priority order) from front to back may be: effective shutter value, effective sensor gain, first ISP gain, and white light intensity.
  • a second setting of the effective shutter value, effective sensor gain, second ISP gain, and infrared light intensity may be set in advance.
  • the second exposure effect may be that a difference between a screen brightness value of the infrared image after exposure processing and a second exposure target value of the infrared image is smaller than a preset second threshold.
  • the second control order (also referred to as the second priority order) of the infrared images from front to back can be: effective shutter value, effective sensor gain, infrared light intensity, and second ISP gain.
  • the dual-light exposure algorithm When the dual-light exposure algorithm is used to perform exposure processing on the infrared image and the visible light image to obtain a visible light image that conforms to the first exposure effect and an infrared image that conforms to the second exposure effect, for easier understanding, you can also refer to A schematic diagram of a two-light exposure control strategy.
  • an exposure control strategy compatible with infrared light and visible light is further illustrated.
  • the two-light exposure algorithm can perform exposure processing on the infrared image and the visible light image based on this strategy. All of them meet the exposure requirements, that is, achieve exposure stability (that is, the absolute value difference between the screen brightness value of the image and the preset exposure target value is less than the preset threshold).
  • the exposure parameter size is adjusted from small to large, and in high-light environments, the exposure parameter size is adjusted from large to small. Due to the low-light environment and the high-light environment's exposure parameter adjustment strategy and The calculation methods for different exposure parameter sizes are similar. Therefore, the embodiment of the present application uses a low-light environment as an example to explain the adjustment process of each exposure parameter. According to the exposure stability status of the infrared image and the visible light image, it can be divided into the following four cases:
  • Case 1 The exposure of the infrared image and the visible light image is unstable, respectively, according to the screen brightness value of the infrared image (that is, the aforementioned second brightness value) and the exposure target value of the infrared image (ie, the aforementioned second exposure target value)
  • the difference between the first exposure target value calculates an exposure parameter of the visible light image (ie, the aforementioned first exposure parameter).
  • the first control order of visible light images from front to back can be: effective shutter value, effective sensor gain, first ISP gain, white light intensity; and the second control order of infrared images from front to back can be: effective shutter value , Effective sensor gain, infrared light intensity, and second ISP gain as examples, refer to a specific exposure adjustment example described below:
  • Second sensor gain value the gain values of the infrared image and the visible light image
  • the infrared image calculates the intensity of the infrared lamp according to the difference between the second brightness value and the second exposure target value.
  • the intensity of the infrared lamp can be adjusted in a linear manner.
  • various exposure parameters are adjusted in turn according to preset control sequences corresponding to the infrared image and the visible light image, respectively, until the exposure of at least one of the infrared image and the visible light image has been adjusted and stabilized.
  • Case 2 If the exposure of the visible light image is stable and the exposure of the infrared image is unstable, continue to calculate and adjust the exposure parameters of the infrared image based on the screen brightness value of the infrared image and the exposure target value of the infrared image until the infrared image exposure is stable;
  • Case 3 If the exposure of the infrared image is stable and the exposure of the visible light image is unstable, continue to calculate and adjust the exposure parameters of the visible light image based on the screen brightness value of the visible light image and the exposure target value of the visible light image until the visible light image exposure is stable;
  • Case 4 The exposure of the infrared image and the visible light image has been adjusted steadily, or all the exposure parameters of the infrared image and all the exposure parameters of the visible light image have been adjusted to the maximum, then the exposure adjustment ends.
  • the above dual-light image automatic exposure method can perform targeted exposure control on a single-sensor-based dual-light image camera, and can adaptively select according to the use status of the infrared cut-off filter.
  • Different automatic exposure control strategies (such as single-light exposure algorithm or dual-light exposure algorithm) make the final image of the camera have higher signal-to-noise ratio, good permeability, and good color reproduction effects.
  • the dual-light image camera provided in this embodiment may further include a light supplement.
  • the light supplement may include an infrared light and a white light.
  • the two-light control can be performed on the light supplement through a dual-light exposure algorithm.
  • the infrared image supplements the light
  • the white light supplements the visible image, thereby effectively improving the image effect of the camera in low-light environments.
  • this embodiment further provides a dual-light image automatic exposure device.
  • the device employs a dual-light image camera.
  • the dual-light image camera includes an image sensor and a processor connected in sequence. The number of sensors is one, and the dual-light image camera is also equipped with an infrared cut-off filter. The device is set on the processor side. Referring to the structural block diagram of an automatic exposure device shown in FIG. 6, the device includes:
  • An original image acquisition module 602 configured to acquire an original image acquired by an image sensor
  • the logical light splitting module 604 is configured to perform logical light splitting processing on the original image to obtain an infrared image and a visible light image;
  • the filter determination module 606 is configured to determine whether an infrared cut-off filter is currently used
  • the single-light exposure control module 608 is configured to use a single-light exposure algorithm to perform an exposure process on the visible light image if an infrared cut-off filter is used to obtain a visible light image that conforms to the target exposure effect;
  • the dual-light exposure control module 610 is configured to perform an exposure process on the infrared image and the visible light image by using a dual-light exposure algorithm without using an infrared cutoff filter, so as to obtain a visible light image conforming to the first exposure effect and an infrared light conforming to the second exposure effect.
  • the above dual-light image automatic exposure device can perform logical spectroscopic processing on the original image collected by the image sensor of the dual-light image camera to obtain an infrared image and a visible light image, and adopts a single light when using an infrared cut-off filter
  • the exposure algorithm performs exposure processing on the visible light image.
  • the dual light exposure algorithm is used to perform exposure processing on the infrared image and the visible light image. This method enables a single-sensor-based dual-light image camera to perform effective exposure processing on an image, and selectively selects different exposure algorithms according to the use of the infrared filter, thereby better improving the image exposure effect.
  • the filter determination module is configured to: determine whether the current ambient brightness is greater than a preset brightness; if yes, determine to use an infrared cut-off filter; if not, determine not to use an infrared cut-off filter.
  • the dual-light exposure control module is configured to: obtain a first brightness value of the visible light image and a second brightness value of the infrared image; obtain a first exposure target value of the visible light image and a second exposure target value of the infrared image Calculate the first exposure parameter of the visible light image according to the first brightness value and the first exposure target value; calculate the second exposure parameter of the infrared image according to the second brightness value and the second exposure target value; according to the first exposure parameter and the first Two exposure parameters: exposure processing for visible and infrared images.
  • the dual-light exposure control module may be further configured to: according to the difference between the first brightness value and the first exposure target value, look up a preset first difference comparison table to determine the first exposure of the visible light image Parameters; based on the difference between the second brightness value and the second exposure target value, searching a preset second difference comparison table to determine a second exposure parameter of the infrared image; wherein the first difference comparison table and the second exposure value
  • the difference table is the same or different.
  • the first exposure parameter may include a first shutter value, a first sensor gain, and a first ISP gain; the second exposure parameter includes a second shutter value, a second sensor gain, and a second ISP gain.
  • the dual-light image camera is further configured with a white light and an infrared light; the first exposure parameter also includes the intensity of the white light; the second exposure parameter also includes the intensity of the infrared light.
  • the above-mentioned dual-light exposure control module may be further configured to determine an effective shutter value shared by the visible light image and the infrared image according to the first shutter value and the second shutter value; according to the first sensor gain and the second sensor gain To determine the effective sensor gain shared by visible and infrared images; perform exposure processing on visible light images based on effective shutter value, effective sensor gain, first ISP gain, and white light intensity; according to effective shutter value, effective sensor gain, and second ISP Gain and infrared lamp intensity, exposure processing for infrared images.
  • the above dual-light exposure control module may be further configured to set a first control order of an effective shutter value, an effective sensor gain, a first ISP gain, and a white light intensity in advance, and sequentially expose the visible light image according to the first control order. Stop until the visible light image reaches the first exposure effect; preset the second control order of the effective shutter value, effective sensor gain, second ISP gain, and infrared light intensity, and sequentially expose the infrared image according to the second control order until Stop when the infrared image reaches the second exposure effect.
  • This embodiment also provides a machine storage medium.
  • the machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions cause the processor to execute the dual-light image. Automatic exposure method.
  • the dual-light image automatic exposure method, device, dual-light image camera, and computer program product of a machine storage medium include a computer-readable storage medium storing a program code.
  • the program code includes instructions that can be used for The method described in the foregoing method embodiment is executed. For specific implementation, refer to the method embodiment, and details are not described herein again.
  • the terms “installation”, “connected”, and “connected” should be understood in a broad sense unless otherwise specified and limited. For example, they may be fixed connections or detachable connections. , Or integrally connected; it can be mechanical or electrical; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements.
  • installation should be understood in a broad sense unless otherwise specified and limited. For example, they may be fixed connections or detachable connections. , Or integrally connected; it can be mechanical or electrical; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements.
  • the specific meanings of the above terms in this application can be understood in specific situations.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.
  • the foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .
  • a dual-light image camera based on a single sensor can perform effective exposure processing on an image to obtain an image with better exposure effect.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Camera Bodies And Camera Details Or Accessories (AREA)
  • Studio Devices (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

本申请提供了一种双光图像自动曝光方法、装置、双光图像相机及机器存储介质，涉及相机技术领域，该方法应用双光图像相机，双光图像相机包括依次连接的图像传感器和处理器，其中，图像传感器的数量为一个，且双光图像相机还配置有红外截止滤镜；该方法由处理器执行，该方法包括：获取图像传感器采集的原始图像；对原始图像进行逻辑分光处理，得到红外图像和可见光图像；判断当前是否使用红外截止滤镜；如果使用红外截止滤镜，采用单光曝光算法对可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像；如果不使用红外截止滤镜，采用双光曝光算法对红外图像和可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像；其中，第一曝光效果和目标曝光效果相同或不同。本申请可以为单传感器双光相机提供一种有效的曝光控制方式。

Description

双光图像自动曝光方法、装置、双光图像相机及机器存储介质 技术领域
本申请涉及相机技术领域，尤其是涉及一种双光图像自动曝光方法、装置、双光图像相机及机器存储介质。
背景技术
传统彩色摄像机仅能够采集可见光波段的信息，并利用可见光波段的信息进行成像。在诸如低照度、雾霾天等恶劣场景下，随着可见光强度的降低或者场景信息被阻挡，传统彩色摄像机成像得到的诸如信噪比、可见性等彩色图像效果显著降质。而在这些场景下，通过适当的曝光控制可以得到清晰、高信噪比的红外图像；但是红外图像仅有灰度信息而无色彩信息。因此，在诸如低照度、雾霾天等恶劣场景下，单一地利用可见光波段或者红外波段进行成像，均无法得到满意的成像效果。
基于此，双光图像相机(又称双光谱相机)应运而生。传统的双光谱相机一般采用双传感器，利用红外传感器和可见光传感器同时采集同一场景下的红外图像和可见光图像，然后利用图像融合技术将红外图像和可见光图像进行有效融合，以得到高质量的彩色融合图像。然而，该类方案一般采用双镜头或硬件工艺构造比较复杂的单镜头进行逻辑分光，以将红外光和可见光进行分离处理，并在红外传感器和可见光传感器上分别成像，然后对二者融合的图像采用常规曝光控制并得到符合要求的融合图像。传统的双光谱相机的硬件成本通常较高。对于此，目前初步提出了单传感器双光相机，然而目前的单传感器双光相机只是沿用与传统的双光谱相机相同的常规曝光控制方式，曝光处理后的图像效果不佳。目前也尚未提出一种针对单传感器双光相机的有效曝光控制方式。
申请内容
有鉴于此，本申请的目的包括，例如，提供一种双光图像自动曝光方法、装置、双光图像相机及机器存储介质，为单传感器双光相机提供了一种有效的曝光控制方式，有助于得到曝光效果更好的图像。
为了实现至少一个本申请的目的，本申请的一个实施例中采用的技术方案如下：
第一方面，本申请实施例提供了一种双光图像自动曝光方法，其特征在于，所述方法应用双光图像相机，所述双光图像相机包括依次连接的图像传感器和处理器，其中，所述图像传感器的数量为一个，且所述双光图像相机还配置有红外截止滤镜；所述方法由所述处理器执行，所述方法包括：获取所述图像传感器采集的原始图像；对所述原始图像进行逻辑分光处理，得到红外图像和可见光图像；判断当前是否使用所述红外截止滤镜；如果使用所述红外截止滤镜，采用单光曝光算法对所述可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像；如果不使用所述红外截止滤镜，采用双光曝光算法对所述红外图像和所述可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像；其中，所述第一曝光效果和所述目标曝光效果相同或不同。
在本申请可选的实施例中，判断是否使用所述红外截止滤镜的步骤，包括：判断当前环境亮度是否大于预设亮度；如果是，确定使用所述红外截止滤镜；如果否，确定不使用所述红外截止滤镜。
在本申请可选的实施例中，采用双光曝光算法对所述红外图像和所述可见光图像进行曝光处理的步骤，包括：获取所述可见光图像的第一亮度值和所述红外图像的第二亮度值；获取所述可见光图像的第一曝光目标值和所述红外图像的第二曝光目标值；根据所述第一亮度值和所述第一曝光目标值，计算所述可见光图像的第一曝光参数；根据所述第二亮度值和所述第二曝光目标值，计算所述红外图像的第二曝光参数；根据所述第一曝光参数和所述第二曝光参数，对所述可见光图像和所述红外图像进行曝光处理。
在本申请可选的实施例中，根据所述第一亮度值和所述第一曝光目标值，计算所述可见光图像的第一曝光参数的步骤，包括：根据所述第一亮度值与所述第一曝光目标值之间的差值，查找预先设置的第一差值对照表，确定所述可见光图像的第一曝光参数；根据所述第二亮度值和所述第二曝光目标值，计算所述红外图像的第二曝光参数的步骤，包括：根据所述第二亮度值与所述第二曝光目标值之间的差值，查找预先设置的第二差值对照表，确定所述红外图像的第二曝光参数；其中，所述第一差值对照表与所述第二差值对照表相同或不同。
在本申请可选的实施例中，所述第一曝光参数包括第一快门值、第一sensor增益和第一ISP增益；所述第二曝光参数包括第二快门值、第二sensor增益和第二ISP增益。
在本申请可选的实施例中，所述双光图像相机还配置有白光灯和红外灯；所述第一曝光参数还包括白光灯强度；所述第二曝光参数还包括红外灯强度。
在本申请可选的实施例中，根据所述第一曝光参数和所述第二曝光参数，对所述可见光图像和所述红外图像进行曝光处理的步骤，包括：根据所述第一快门值和所述第二快门 值，确定所述可见光图像和所述红外图像共用的有效快门值；根据所述第一senor增益和所述第二sensor增益，确定所述可见光图像和所述红外图像共用的有效senor增益；根据所述有效快门值、所述有效senor增益、所述第一ISP增益和所述白光灯强度，对所述可见光图像进行曝光处理；根据所述有效快门值、所述有效senor增益、所述第二ISP增益和所述红外灯强度，对所述红外图像进行曝光处理。
在本申请可选的实施例中，根据所述第一快门值和所述第二快门值，确定所述可见光图像和所述红外图像共用的有效快门值的步骤，包括：选取所述第一快门值和所述第二快门值中的最小值，将选取的所述最小值确定为所述可见光图像和所述红外图像共用的有效快门值；或，选取所述第一快门值和所述第二快门值中的最大值，将选取的所述最大值确定为所述可见光图像和所述红外图像共用的有效快门值；或，计算所述第一快门值和所述第二快门值的平均值，将所述平均值确定为所述可见光图像和所述红外图像共用的有效快门值；或，采用预设的加权算法计算所述第一快门值和所述第二快门值的加权快门值，将所述加权快门值确定为所述可见光图像和所述红外图像共用的有效快门值。
在本申请可选的实施例中，根据所述第一sensor增益和所述第二sensor增益，确定所述可见光图像和所述红外图像共用的有效sensor增益的步骤，包括：选取所述第一sensor增益和所述第二sensor增益中的最小值，将选取的所述最小值确定为所述可见光图像和所述红外图像共用的有效sensor增益；或，选取所述第一sensor增益和所述第二sensor增益中的最大值，将选取的所述最大值确定为所述可见光图像和所述红外图像共用的有效sensor增益；或，计算所述第一sensor增益和所述第二sensor增益的平均值，将所述平均值确定为所述可见光图像和所述红外图像共用的有效sensor增益；或，采用预设的加权算法计算所述第一sensor增益和所述第二sensor增益的加权增益，将所述加权增益确定为所述可见光图像和所述红外图像共用的有效sensor增益。
在本申请可选的实施例中，所述根据所述有效快门值、所述有效senor增益、所述第一ISP增益和所述白光灯强度，对所述可见光图像进行曝光处理的步骤，包括：预先设定所述有效快门值、所述有效senor增益、所述第一ISP增益和所述白光灯强度的第一控制排序，按照所述第一控制排序依次对所述可见光图像进行曝光处理，直至所述可见光图像达到所述第一曝光效果时停止；根据所述有效快门值、所述有效senor增益、所述第二ISP增益和所述红外灯强度，对所述红外图像进行曝光处理的步骤，包括：预先设定所述有效快门值、所述有效senor增益、所述第二ISP增益和所述红外灯强度的第二控制排序，按照所述第二控制排序依次对所述红外图像进行曝光处理，直至所述红外图像达到所述第二曝光效果时停止。
在本申请可选的实施例中，所述第一曝光效果为：所述可见光图像经曝光处理后的画面亮度值与所述可见光图像的第一曝光目标值之间的差值小于预设第一阈值；所述第二曝光效果为：所述红外图像经曝光处理后的画面亮度值与所述红外图像的第二曝光目标值之间的差值小于预设第二阈值。
第二方面，本申请实施例还提供一种双光图像自动曝光装置，其特征在于，所述装置应用双光图像相机，所述双光图像相机包括依次连接的图像传感器和处理器，其中，所述图像传感器的数量为一个，且所述双光图像相机还配置有红外截止滤镜；所述装置配置在所述处理器内，所述装置包括：原始图像获取模块，配置成获取所述图像传感器采集的原始图像；逻辑分光模块，配置成对所述原始图像进行逻辑分光处理，得到红外图像和可见光图像；滤镜判断模块，配置成判断当前是否使用所述红外截止滤镜；单光曝光控制模块，配置成如果使用所述红外截止滤镜，采用单光曝光算法对所述可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像；双光曝光控制模块，配置成如果不使用所述红外截止滤镜，采用双光曝光算法对所述红外图像和所述可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像；其中，所述第一曝光效果和所述目标曝光效果相同或不同。
在本申请可选的实施例中，所述滤镜判断模块配置成：判断当前环境亮度是否大于预设亮度；如果是，确定使用所述红外截止滤镜；如果否，确定不使用所述红外截止滤镜。
在本申请可选的实施例中，所述双光曝光控制模块配置成：获取所述可见光图像的第一亮度值和所述红外图像的第二亮度值；获取所述可见光图像的第一曝光目标值和所述红外图像的第二曝光目标值；根据所述第一亮度值和所述第一曝光目标值，计算所述可见光图像的第一曝光参数；根据所述第二亮度值和所述第二曝光目标值，计算所述红外图像的第二曝光参数；根据所述第一曝光参数和所述第二曝光参数，对所述可见光图像和所述红外图像进行曝光处理。
第三方面，本申请实施例提供了一种双光图像相机，其特征在于，包括：依次连接的图像传感器和处理器；其中，所述图像传感器的数量为一个，且所述双光图像相机还配置有红外截止滤镜；所述处理器上设置有第二方面任一项所述的双光图像自动曝光装置。
在本申请可选的实施例中，所述图像传感器之前还设置有镜头；其中，所述镜头为红外共焦镜头。
在本申请可选的实施例中，所述镜头的透光率高于预设透光率。
在本申请可选的实施例中，所述处理器还连接有补光器；所述补光器包括红外补光灯和/或白光补光灯。
在本申请可选的实施例中，所述图像传感器为RGB-IR传感器。
第四方面，本申请实施例提供了一种机器存储介质，其特征在于，所述机器可读存储介质存储有机器可执行指令，所述机器可执行指令在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行第一方面任一项所述的方法。
本申请实施例提供了一种双光图像自动曝光方法、装置、双光图像相机及机器存储介质，能够对双光图像相机的单传感器采集的原始图像进行逻辑分光处理，得到红外图像和可见光图像，并在使用红外截止滤镜时采用单光曝光算法对可见光图像进行曝光处理，在不使用红外截止滤镜时采用双光曝光算法对红外图像和可见光图像进行曝光处理。这种方式使基于单传感器的双光图像相机能够对图像进行有效的曝光处理，而且根据红外滤镜的使用情况有针对性地选择不同的曝光算法，从而较好地提升图像曝光效果。
本申请实施例的其他特征和优点将在随后的说明书中阐述，或者，部分特征和优点可以从说明书推知或毫无疑义地确定，或者通过实施本申请的上述技术即可得知。
为使本申请的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本申请具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例所提供的一种双光图像相机的结构示意图；
图2示出了本申请实施例所提供的另一种双光图像相机的结构示意图；
图3示出了本申请实施例所提供的一种双光图像自动曝光方法流程图；
图4示出了本申请实施例所提供的一种双光曝光控制方法流程图；
图5示出了本申请实施例所提供的一种双光曝光控制策略示意图；
图6示出了本申请实施例所提供的一种自动曝光装置的结构框图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全 部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
考虑到基于双传感器的双光图像相机所需成本较高，而成本较低的基于单传感器的双光图像相机目前仅是沿用传统的曝光控制方式，曝光效果不佳，为改善此问题，本申请实施例提供的一种双光图像自动曝光方法、装置、双光图像相机及机器存储介质，以下对本申请实施例进行详细介绍。
为了更好地理解本申请实施例的技术方案，以下首先对本申请实施例提出的双光图像相机(又可称为双光图像相机系统)进行介绍。
本申请实施例提供的双光图像相机，至少包括依次连接的图像传感器和处理器；其中，图像传感器的数量为一个。因此本申请实施例提供的双光图像相机也可以称为基于单传感器的双光图像相机。此外，双光图像相机还配置有红外截止滤镜。双光图像相机中的处理器即可执行本申请实施例提出的双光图像自动曝光方法，亦或，其上可设置有本申请实施例提供的双光图像自动曝光装置。
为便于理解，可参见图1所示的一种双光图像相机的结构示意图，示意出依次连接的镜头10、图像传感器20、处理器30和补光器40。
关于镜头10：本申请实施例提供的双光图像相机对镜头的要求较简单，镜头应当为红外共焦镜头，进一步镜头的透光率还可以高于预设透光率。具体而言，镜头对可见光和近红外光均具有较好的透光率。而红外共焦镜头可以保证同一个焦面上可见光图像与红外图像均能够清晰成像。
关于图像传感器20：图像传感器可采用单个的常规RGB-IR传感器即可，RGB-IR传感器具有集成可见光图像和红外图像的能力，对于可见光图像还具有较高的信噪比、通透性好、色彩还原效果好等优势。通过采用RGB-IR传感器，结合本申请实施例提供的曝光控制方法，有助于使不同场景下的红外图像和可见光图像均具有较好的成像效果。
关于处理器30：可以采用数字信号处理器(DSP)、现场可编程门阵列(FPGA)、可编程逻辑阵列(PLA)中的至少一种硬件形式来实现，也可以是中央处理器(CPU)、图形处理器(GPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理器件中的一种或几种的组合，并且可以控制双光图像相机的其它组件(诸如图像传感器、补光器等)以执行期望的功能。
在一种实施方式中，参见图2所示的另一种双光图像相机的结构示意图，该图在图1的基础上，还示意出处理器30包括统计单元30a和自动曝光控制单元30b。
统计单元30a中示意出了逻辑分光子单元、第一ISP增益子单元、第二ISP增益子单元、还示意出了经逻辑分光和ISP增益处理后的RGB统计数据和IR统计数据。其中，逻辑分光子单元主要是将RGB-IR传感器输出的原始图像转换为红外灰度图像(以下简称红外图像)和可见光彩色图像(以下简称可见光图像)。第一ISP增益子单元用于对可见光图像进行数字信号放大，主要提升可见光图像的整体亮度，第二ISP增益子单元用于对红外图像进行数字信号放大，主要提升红外图像的整体亮度。从图2中可见，本申请实施例提供的双光图像相机支持分别对红外图像和可见光图像进行单独处理。在一种实施方式中，统计单元30a可采用FPGA、DSP、CPU等逻辑器件编程实现。
自动曝光控制单元30b可基于RGB统计数据和IR统计数据对确定可见光图像和红外图像的曝光参数，具体可根据实际情况(诸如红外截止滤镜是否使用)而采用双光曝光控制策略或单光曝光控制策略。具体而言，自动曝光控制单元可以对可见光图像和红外图像的一些曝光参数分别控制，也即实现可见光图像和红外图像的双路控制，诸如，ISP增益可在FPGA、DSP等逻辑器件上实现双路控制，补光强度也可以通过红外补光灯和白光补光灯分开控制，而由于本申请实施例提出的双光图像相机采用单传感器，而曝光参数中的快门参数和sensor增益均只在传感器上生效，因此这些参数需要对可见光图像和红外图像进行单路共同控制。自动曝光控制单元通过对单路曝光参数(诸如快门参数、sensor增益)和双路曝光参数(诸如补光强度、ISP增益)进行调整，从而影响RGB-IR传感器的图像采集数据，即通过不断反馈控制调节，最终使红外图像和可见光图像达到最佳的图像效果。在一种实施方式中，自动曝光控制单元30b可采用诸如CPU等逻辑期间编程实现。
关于补光器40：补光器可以包括红外补光灯和/或白光补光灯，在实际应用中，本申请实施例对补光器无特殊限制要求，补光器可以是包含有红外补光灯和白光补光灯的全光谱补光灯，当然也可以仅是红外补光灯，在此不进行限制。由于本申请实施例提供的处理器能够对原始图像进行逻辑分光，得到可见光图像和红外图像，因此可保证红外补光对可见光不影响，白光补光对红外图像不影响。基于此，低照环境下可通过红外补光来提升红外图像亮度及成像效果，白光补光来提升可见光图像的亮度及成像效果，从而保证低照环境下仍可得到质量较好的红外图像和可见光图像，进而提升最终融合后的图像效果。
在前述基于单传感器的双光图像相机的基础上，为了能够使RGB-IR传感器采集的红外图像和可见光图像达到更好的图像效果，本申请实施例进一步提供了一种双光图像自动曝光方法，该方法应用双光图像相机，双光图像相机包括依次连接的图像传感器和处理器，其中，图像传感器的数量为一个，且双光图像相机还配置有红外截止滤镜；该方法由处理器执行，参见图3所示的一种双光图像自动曝光方法流程图，该方法包括：
步骤S302，获取图像传感器采集的原始图像。其中，图像传感器可以为RGB-IB传感器。
步骤S304，对原始图像进行逻辑分光处理，得到红外图像和可见光图像。具体实现时，还可以分别获得红外图像的统计数据和可见光图像的统计数据，并根据对应的统计数据分别计算红外图像亮度和可见光图像亮度。诸如，将图像进行分块处理(分为M*N个区块)，然后统计每个区块的亮度数据，所有的M*N块的亮度数据即为该图像的统计数据。
步骤S306，判断当前是否使用红外截止滤镜。如果使用，执行步骤S308；如果不使用，执行步骤S310。
具体实施时，可以判断当前环境亮度是否大于预设亮度；如果是，确定使用红外截止滤镜；如果否，确定不使用红外截止滤镜。也即，在环境较亮时，使用红外截止滤镜，在环境较暗时，不使用红外截止滤镜。此外，可以根据当前增益值判断当前红外截止滤镜的状态，若当前增益值小于Gain 下限值，表明使用红外截止滤镜，若当前增益值大于Gain 下限值，表明不使用红外截止滤镜。其中，当前增益值可按照大多相机采用的常规增益计算方式得到，通常而言，当前增益值是与图像亮度值相关的。具体实施时，红外截止滤镜的使用与否可以采用滤镜开关实现。
步骤S308，采用单光曝光算法对可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像。其中，单光曝光算法又可称为单光曝光控制策略，也即主要针对一种类型的光图像(诸如可见光图像)进行曝光控制。可以理解的是，如果使用了红外截止滤镜，由于红外截止滤镜能够将环境光中的红外光过滤掉，此时调节曝光参数对红外图像不影响，所以该情况下以调节可见光图像效果为主。
步骤S310，采用双光曝光算法对红外图像和可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像。可以理解的是，可见光图像经双光曝光算法进行处理得到的第一曝光效果和经单光曝光算法得到的目标曝光效果可以相同，也可以不同，在此不进行限制。在实际应用中，可以根据需求而灵活设置目标曝光效果、第一曝光效果和第二曝光效果。
其中，双光曝光算法又可称为双光曝光控制策略，也即针对两种类型的光图像(诸如红外图像和可见光图像)进行曝光处理。可以理解的是，如果不使用红外截止滤镜，进入至RGB-IR传感器的环境光中同时包含有红外光和可见光，因此会对红外图像和可见光图像均造成影响，若采用单光曝光算法则无法兼顾红外图像和可见光图像的图像曝光效果，因此本实施例提出一种双光曝光算法，以尽量使红外图像和可见光图像均达到较好的图像效果，进而可对达到较好效果的红外图像和可见光图像进行图像融合并显示。
本申请实施例提供的上述双光图像自动曝光方法，能够对双光图像相机的图像传感器采集的原始图像进行逻辑分光处理，得到红外图像和可见光图像，并在使用红外截止滤镜时采用单光曝光算法对可见光图像进行曝光处理，在不使用红外截止滤镜时采用双光曝光算法对红外图像和可见光图像进行曝光处理。这种方式使基于单传感器的双光图像相机能够对图像进行有效的曝光处理，而且根据红外滤镜的使用情况有针对性地选择不同的曝光算法，从而较好地提升图像曝光效果。
在一些实施方式中，在采用单光曝光算法对可见光图像进行曝光处理时，可以利用中央权重或全局测光方法统计该图像的当前画面亮度值，根据画面亮度值和预设的曝光目标值之间的差值来判断是否调节曝光，若差值大于预设阈值，则调节曝光；否则不调节曝光。具体也可以参见常规的自动曝光(Auto Exposure，AE)算法实现，在此不再赘述。
在一些实施方式中，在采用双光曝光算法对红外图像和可见光图像进行曝光处理时，参见图4所示的一种双光曝光控制方法流程图，可以采用如下步骤实现：
步骤S402，获取可见光图像的第一亮度值和红外图像的第二亮度值。一种实现方式可以是，诸如，首先获取图像的统计数据，然后基于统计数据计算亮度值。
步骤S404，获取可见光图像的第一曝光目标值和红外图像的第二曝光目标值。本申请实施例给出一种曝光目标值的获取方式，假设预先设定不同的增益档位0dB、6dB、12dB、18dB等依次类推，根据不同的增益档位，预设不同增益档位下的曝光目标值依次为50、49、47、45等依次类推，然后获取图像的当前增益值，判断当前增益值处于哪一个增益档位，然后对当前增益值进行线性插值处理，计算得到对应的曝光目标值，如：当前增益值为15dB，则根据线性插值计算为(18–15)/(18-12)*(47-45)+45＝46，那么15dB对应的预设曝光目标值即为46。
步骤S406，根据第一亮度值和第一曝光目标值，计算可见光图像的第一曝光参数。诸如，根据第一亮度值与第一曝光目标值之间的差值，查找预先设置的第一差值对照表，确定可见光图像的第一曝光参数。
步骤S408，根据第二亮度值和第二曝光目标值，计算红外图像的第二曝光参数。诸如，根据第二亮度值与第二曝光目标值之间的差值，查找预先设置的第二差值对照表，确定红外图像的第二曝光参数。
在具体实现时，步骤S406中的第一差值对照表与步骤S408中的第二差值对照表相同或不同。
步骤S410，根据第一曝光参数和第二曝光参数，对可见光图像和红外图像进行曝光处理。
在一些实施方式中，第一曝光参数可以包括第一快门值、第一sensor增益和第一ISP增益；第二曝光参数可以包括第二快门值、第二sensor增益和第二ISP增益。如果双光图像相机还配置有白光灯和红外灯；则第一曝光参数还可以包括白光灯强度；第二曝光参数还可以包括红外灯强度。此外，曝光参数还可以包括光圈等。
对于ISP增益和补光灯而言，双光图像相机可以实现对红外图像和可见光图像的双路分开控制，分别计算第一ISP增益、第二ISP增益、白光灯强度和红外灯强度；而对于快门值和sensor增益而言，由于双光图像相机仅具有一个传感器，仅能共用一套参数，共同计算有效快门值和有效sensor增益。
基于此，在根据第一曝光参数和第二曝光参数，对可见光图像和红外图像进行曝光处理时，可以根据第一快门值和第二快门值，确定可见光图像和红外图像共用的有效快门值；根据第一senor增益和第二sensor增益，确定可见光图像和红外图像共用的有效senor增益；从而根据有效快门值、有效senor增益、第一ISP增益和白光灯强度，对可见光图像进行曝光处理；以及根据有效快门值、有效senor增益、第二ISP增益和红外灯强度，对红外图像进行曝光处理。各种曝光参数的计算方法类似，以快门为例说明具体的计算过程，诸如，预先设定不同的亮度差(也即，图像的画面亮度值和曝光目标值二者之间的差值)对应的不同快门大小，形成亮度差值对照表；然后计算图像亮度值和曝光目标值二者之间的差值；最后根据当前计算得到的差值以及预设的亮度差值对照表，即可确定计算得到的差值所对应的快门大小。
在确定可见光图像和红外图像共用的有效快门值，以及确定可见光图像和红外图像共用的有效sensor增益时，可以采用如下方式：
方式1：最小值选取法
选取第一快门值和第二快门值中的最小值，将选取的最小值确定为可见光图像和红外图像共用的有效快门值；
选取第一sensor增益和第二sensor增益中的最小值，将选取的最小值确定为可见光图像和红外图像共用的有效sensor增益；
方式2：最大值选取法
选取第一快门值和第二快门值中的最大值，将选取的最大值确定为可见光图像和红外图像共用的有效快门值；
选取第一sensor增益和第二sensor增益中的最大值，将选取的最大值确定为可见光图像和红外图像共用的有效sensor增益；
方式3：平均值选取法
计算第一快门值和第二快门值的平均值，将平均值确定为可见光图像和红外图像共用的有效快门值；
计算第一sensor增益和第二sensor增益的平均值，将平均值确定为可见光图像和红外图像共用的有效sensor增益；
方式4：加权值选取法
采用预设的加权算法计算第一快门值和第二快门值的加权快门值，将加权快门值确定为可见光图像和红外图像共用的有效快门值；
采用预设的加权算法计算第一sensor增益和第二sensor增益的加权增益，将加权增益确定为可见光图像和红外图像共用的有效sensor增益。
在实际应用中，可见光图像和红外图像共用的有效快门值可以灵活选用上述方式1至方式4中的任意一种，有效sensor增益也可以灵活选用上述方式1至方式4中的任意一种，或者选用其它计算方式(不局限于上述四种方式)均可，在此不进行限定。在具体选取时，有效快门值选取的方式可以与有效sensor增益选取的方式相同或不同。
考虑到不同曝光参数的控制优先级对图像的成像效果的影响不同，因此，在根据有效快门值、有效senor增益、第一ISP增益和白光灯强度，对可见光图像进行曝光处理时，可以预先设定有效快门值、有效senor增益、第一ISP增益和白光灯强度的第一控制排序，按照第一控制排序依次对可见光图像进行曝光处理，直至可见光图像达到第一曝光效果时停止。诸如，第一曝光效果可以为，可见光图像经曝光处理后的画面亮度值与可见光图像的第一曝光目标值之间的差值小于预设第一阈值。诸如，可见光图像的第一控制排序(又可称为第一优先级顺序)由前至后可以为：有效快门值、有效senor增益、第一ISP增益、白光灯强度。
在根据有效快门值、有效senor增益、第二ISP增益和红外灯强度，对红外图像进行曝光处理时，可以预先设定有效快门值、有效senor增益、第二ISP增益和红外灯强度的第二控制排序，按照第二控制排序依次对红外图像进行曝光处理，直至红外图像达到第二曝光效果时停止。诸如，第二曝光效果可以为，红外图像经曝光处理后的画面亮度值与红外图像的第二曝光目标值之间的差值小于预设第二阈值。诸如，红外图像的第二控制排序(又可称为第二优先级顺序)由前至后可以为：有效快门值、有效senor增益、红外灯强度、第二ISP增益。
在采用双光曝光算法对红外图像和可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像时，为便于理解，还可以参见图5所示的一种双光曝光控制策略示意图，在该图中还进一步示意出了一种红外光与可见光兼容性的 曝光控制策略，双光曝光算法可基于该策略对红外图像和可见光图像进行曝光处理，并使其均符合曝光要求，也即达到曝光稳定(也即，图像的画面亮度值与预设的曝光目标值之间的绝对值差值小于预设阈值)。通常而言，在低照环境下，曝光参数大小是从小往大调节，在高亮环境下，曝光参数大小是从大往小调节，由于低照环境和高亮环境下的曝光参数调节策略及不同曝光参数大小的计算方法类似，因此本申请实施例以低照环境为例解释说明各曝光参数的调节过程，依据红外图像和可见光图像的曝光稳定状态可分为以下4种情况：
情况1：红外图像与可见光图像曝光均不稳定，则分别根据红外图像的画面亮度值(也即，前述第二亮度值)与红外图像的曝光目标值(也即，前述第二曝光目标值)之间的差值计算红外图像的曝光参数(也即，前述第二曝光参数)，可见光图像的画面亮度值(也即，前述第一亮度值)与可见光图像的曝光目标值(也即，前述第一曝光目标值)之间的差值计算可见光图像的曝光参数(也即，前述第一曝光参数)。以可见光图像的第一控制排序由前至后可以为：有效快门值、有效senor增益、第一ISP增益、白光灯强度；以及红外图像的第二控制排序由前至后可以为：有效快门值、有效senor增益、红外灯强度、第二ISP增益为例，参照如下所述的一种具体的曝光调节示例：
首先，分别计算红外图像和可见光图像需要调整的快门值(也即，第一快门值和第二快门值)，然后基于第一快门值和第二快门值选取有效快门值，若此时的红外图像和可见光图像的曝光均不稳定，则继续调节快门值至最大，若此时红外图像和可见光图像的曝光还是不稳定，再开始调节红外图像和可见光图像的增益值(第一sensor增益值和第二sensor增益值)，然后基于第一sensor增益值和第二sensor增益值选取有效sensor增益，若调至有效sensor增益的红外图像和可见光图像的曝光仍不稳定，则继续调节增益值至最大，若此时红外图像和可见光图像的曝光还是不稳定，则红外图像根据第二亮度值和第二曝光目标值之间的差值计算红外灯强度，具体可采用线性方式来调节红外灯强度，也即，图像亮度值和曝光目标值之间的差值越大，红外灯调节的强度值越大；当红外灯强度调节至最大时，红外图像的曝光仍旧不稳定，则继续调节红外图像的第二ISP增益；可见光图像则根据第一亮度值和第一曝光目标值之间的差值先计算第一ISP增益，如果调至第一ISP增益时仍旧曝光不稳定，则继续往高调整可见光图像的ISP增益值至最大，如还不稳定则继续调节白光灯强度大小，具体可参照红外灯调节方式。只要红外图像和可见光图像曝光均不稳定，则根据红外图像和可见光图像各自对应的预设控制排序依次调节各种曝光参数，直到至少红外图像和可见光图像其中之一的曝光已调节稳定为止。
情况2：若可见光图像曝光已稳定，红外图像曝光不稳定，则继续根据红外图像的画面亮度值与红外图像的曝光目标值计算并调节红外图像的曝光参数，直至红外图像曝光稳定；
情况3：若红外图像曝光已稳定，可见光图像曝光不稳定，则继续根据可见光图像的画面亮度值与可见光图像的曝光目标值计算并调节可见光图像的曝光参数，直至可见光图像曝光稳定；
情况4：红外图像和可见光图像曝光均已调节稳定或者红外图像的所有曝光参数和可见光图像的所有曝光参数均已调至最大，则曝光调节结束。
对于红外图像和可见光图像而言，无论二者当前的曝光稳定状态属于上述哪一种情况，最终均可以通过调整曝光将其逐步收敛到情况4结束。
综上所述，本实施例提供的上述双光图像自动曝光方法，可对基于单传感器的双光图像相机进行有针对性地曝光控制，能够根据红外截止滤镜的使用状态而自适应地选择不同的自动曝光控制策略(诸如，单光曝光算法或双光曝光算法)，使相机最终得到的图像具有较高的信噪比、通透性好、色彩还原效果好等优势。而且，本实施例提供的上述双光图像自动曝光方法在双光曝光算法中，可以实现对红外图像和可见光图像的曝光参数进行双路控制(诸如，ISP增益和补光灯)和共同控制(诸如，快门和sensor增益)，从而实现红外图像和可见光图像在不同场景下均能达到较好的图像效果。此外，本实施例提供的双光图像相机还可以带补光器，该补光器可以包含有红外灯和白光灯，通过双光曝光算法即可对补光器进行双路控制，红外灯对红外图像进行补光，白光灯对可见光图像进行补光，从而有效提升相机在低照环境下的图像效果。
对应于前述双光图像自动曝光方法，本实施例进一步提供了一种双光图像自动曝光装置，该装置应用双光图像相机，双光图像相机包括依次连接的图像传感器和处理器，其中，图像传感器的数量为一个，且双光图像相机还配置有红外截止滤镜。该装置设置在处理器侧，参见图6所示的一种自动曝光装置的结构框图，该装置包括：
原始图像获取模块602，配置成获取图像传感器采集的原始图像；
逻辑分光模块604，配置成对原始图像进行逻辑分光处理，得到红外图像和可见光图像；
滤镜判断模块606，配置成判断当前是否使用红外截止滤镜；
单光曝光控制模块608，配置成如果使用红外截止滤镜，采用单光曝光算法对可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像；
双光曝光控制模块610，配置成如果不使用红外截止滤镜，采用双光曝光算法对红外图像和可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像；其中，第一曝光效果和目标曝光效果相同或不同。
本申请实施例提供的上述双光图像自动曝光装置，能够对双光图像相机的图像传感器采集的原始图像进行逻辑分光处理，得到红外图像和可见光图像，并在使用红外截止滤镜时采用单光曝光算法对可见光图像进行曝光处理，在不使用红外截止滤镜时采用双光曝光算法对红外图像和可见光图像进行曝光处理。这种方式使基于单传感器的双光图像相机能够对图像进行有效的曝光处理，而且根据红外滤镜的使用情况有针对性地选择不同的曝光算法，从而较好地提升图像曝光效果。
在一种实施方式中，滤镜判断模块配置成：判断当前环境亮度是否大于预设亮度；如果是，确定使用红外截止滤镜；如果否，确定不使用红外截止滤镜。
在一种实施方式中，双光曝光控制模块配置成：获取可见光图像的第一亮度值和红外图像的第二亮度值；获取可见光图像的第一曝光目标值和红外图像的第二曝光目标值；根据第一亮度值和第一曝光目标值，计算可见光图像的第一曝光参数；根据第二亮度值和第二曝光目标值，计算红外图像的第二曝光参数；根据第一曝光参数和第二曝光参数，对可见光图像和红外图像进行曝光处理。在具体实施时，双光曝光控制模块可进一步配置成：根据第一亮度值与第一曝光目标值之间的差值，查找预先设置的第一差值对照表，确定可见光图像的第一曝光参数；根据第二亮度值与第二曝光目标值之间的差值，查找预先设置的第二差值对照表，确定红外图像的第二曝光参数；其中，第一差值对照表与第二差值对照表相同或不同。
诸如，第一曝光参数可以包括第一快门值、第一sensor增益和第一ISP增益；第二曝光参数包括第二快门值、第二sensor增益和第二ISP增益。如果双光图像相机还配置有白光灯和红外灯；第一曝光参数还包括白光灯强度；第二曝光参数还包括红外灯强度。
在具体实施时，上述双光曝光控制模块还可进一步配置成：根据第一快门值和第二快门值，确定可见光图像和红外图像共用的有效快门值；根据第一senor增益和第二sensor增益，确定可见光图像和红外图像共用的有效senor增益；根据有效快门值、有效senor增益、第一ISP增益和白光灯强度，对可见光图像进行曝光处理；根据有效快门值、有效senor增益、第二ISP增益和红外灯强度，对红外图像进行曝光处理。其中，有效快门值和有效senor增益的确定方式可参照前述方法实施例，在此不再赘述。
上述双光曝光控制模块还可进一步配置成：预先设定有效快门值、有效senor增益、第一ISP增益和白光灯强度的第一控制排序，按照第一控制排序依次对可见光图像进行曝光 处理，直至可见光图像达到第一曝光效果时停止；预先设定有效快门值、有效senor增益、第二ISP增益和红外灯强度的第二控制排序，按照第二控制排序依次对红外图像进行曝光处理，直至红外图像达到第二曝光效果时停止。
本实施例所提供的装置，其实现原理及产生的技术效果和前述实施例相同，为简要描述，装置实施例部分未提及之处，可参考前述方法实施例中相应内容。
本实施例还提供了一种机器存储介质，该机器可读存储介质存储有机器可执行指令，机器可执行指令在被处理器调用和执行时，机器可执行指令促使处理器执行上述双光图像自动曝光方法。
本申请实施例所提供的双光图像自动曝光方法、装置、双光图像相机及机器存储介质的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行前面方法实施例中所述的方法，具体实现可参见方法实施例，在此不再赘述。
另外，在本申请实施例的描述中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本申请中的具体含义。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本申请的具体实施方式，用以说明本申请的技术方案，而非对其限制，本申请的保护范围并不局限于此，尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本申请实施例技术方案的精神和范围，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。
工业实用性：
通过应用本申请的技术方案，基于单传感器的双光图像相机能够对图像进行有效的曝光处理，得到曝光效果更好的图像。

Claims (20)

  1. 一种双光图像自动曝光方法，其特征在于，所述方法应用双光图像相机，所述双光图像相机包括依次连接的图像传感器和处理器，其中，所述图像传感器的数量为一个，且所述双光图像相机还配置有红外截止滤镜；
    所述方法由所述处理器执行，所述方法包括：
    获取所述图像传感器采集的原始图像；
    对所述原始图像进行逻辑分光处理，得到红外图像和可见光图像；
    判断当前是否使用所述红外截止滤镜；
    如果使用所述红外截止滤镜，采用单光曝光算法对所述可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像；
    如果不使用所述红外截止滤镜，采用双光曝光算法对所述红外图像和所述可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像；其中，所述第一曝光效果和所述目标曝光效果相同或不同。
  2. 根据权利要求1所述的方法，其特征在于，判断是否使用所述红外截止滤镜的步骤，包括：
    判断当前环境亮度是否大于预设亮度；
    如果是，确定使用所述红外截止滤镜；如果否，确定不使用所述红外截止滤镜。
  3. 根据权利要求1所述的方法，其特征在于，采用双光曝光算法对所述红外图像和所述可见光图像进行曝光处理的步骤，包括：
    获取所述可见光图像的第一亮度值和所述红外图像的第二亮度值；
    获取所述可见光图像的第一曝光目标值和所述红外图像的第二曝光目标值；
    根据所述第一亮度值和所述第一曝光目标值，计算所述可见光图像的第一曝光参数；
    根据所述第二亮度值和所述第二曝光目标值，计算所述红外图像的第二曝光参数；
    根据所述第一曝光参数和所述第二曝光参数，对所述可见光图像和所述红外图像进行曝光处理。
  4. 根据权利要求3所述的方法，其特征在于，根据所述第一亮度值和所述第一曝光目标值，计算所述可见光图像的第一曝光参数的步骤，包括：
    根据所述第一亮度值与所述第一曝光目标值之间的差值，查找预先设置的第一差值对照表，确定所述可见光图像的第一曝光参数；
    根据所述第二亮度值和所述第二曝光目标值，计算所述红外图像的第二曝光参数的步骤，包括：
    根据所述第二亮度值与所述第二曝光目标值之间的差值，查找预先设置的第二差值对照表，确定所述红外图像的第二曝光参数；
    其中，所述第一差值对照表与所述第二差值对照表相同或不同。
  5. 根据权利要求3所述的方法，其特征在于，所述第一曝光参数包括第一快门值、第一sensor增益和第一ISP增益；所述第二曝光参数包括第二快门值、第二sensor增益和第二ISP增益。
  6. 根据权利要求5所述的方法，其特征在于，所述双光图像相机还配置有白光灯和红外灯；所述第一曝光参数还包括白光灯强度；所述第二曝光参数还包括红外灯强度。
  7. 根据权利要求6所述的方法，其特征在于，根据所述第一曝光参数和所述第二曝光参数，对所述可见光图像和所述红外图像进行曝光处理的步骤，包括：
    根据所述第一快门值和所述第二快门值，确定所述可见光图像和所述红外图像共用的有效快门值；
    根据所述第一senor增益和所述第二sensor增益，确定所述可见光图像和所述红外图像共用的有效senor增益；
    根据所述有效快门值、所述有效senor增益、所述第一ISP增益和所述白光灯强度，对所述可见光图像进行曝光处理；
    根据所述有效快门值、所述有效senor增益、所述第二ISP增益和所述红外灯强度，对所述红外图像进行曝光处理。
  8. 根据权利要求7所述的方法，其特征在于，根据所述第一快门值和所述第二快门值，确定所述可见光图像和所述红外图像共用的有效快门值的步骤，包括：
    选取所述第一快门值和所述第二快门值中的最小值，将选取的所述最小值确定为所述可见光图像和所述红外图像共用的有效快门值；
    或，
    选取所述第一快门值和所述第二快门值中的最大值，将选取的所述最大值确定为所述可见光图像和所述红外图像共用的有效快门值；
    或，
    计算所述第一快门值和所述第二快门值的平均值，将所述平均值确定为所述可见光图像和所述红外图像共用的有效快门值；
    或，
    采用预设的加权算法计算所述第一快门值和所述第二快门值的加权快门值，将所述加权快门值确定为所述可见光图像和所述红外图像共用的有效快门值。
  9. 根据权利要求7所述的方法，其特征在于，根据所述第一sensor增益和所述第二sensor增益，确定所述可见光图像和所述红外图像共用的有效sensor增益的步骤，包括：
    选取所述第一sensor增益和所述第二sensor增益中的最小值，将选取的所述最小值确定为所述可见光图像和所述红外图像共用的有效sensor增益；
    或，
    选取所述第一sensor增益和所述第二sensor增益中的最大值，将选取的所述最大值确定为所述可见光图像和所述红外图像共用的有效sensor增益；
    或，
    计算所述第一sensor增益和所述第二sensor增益的平均值，将所述平均值确定为所述可见光图像和所述红外图像共用的有效sensor增益；
    或，
    采用预设的加权算法计算所述第一sensor增益和所述第二sensor增益的加权增益，将所述加权增益确定为所述可见光图像和所述红外图像共用的有效sensor增益。
  10. 根据权利要求7所述的方法，其特征在于，所述根据所述有效快门值、所述有效senor增益、所述第一ISP增益和所述白光灯强度，对所述可见光图像进行曝光处理的步骤，包括：
    预先设定所述有效快门值、所述有效senor增益、所述第一ISP增益和所述白光灯强度的第一控制排序，按照所述第一控制排序依次对所述可见光图像进行曝光处理，直至所述可见光图像达到所述第一曝光效果时停止；
    根据所述有效快门值、所述有效senor增益、所述第二ISP增益和所述红外灯强度，对所述红外图像进行曝光处理的步骤，包括：
    预先设定所述有效快门值、所述有效senor增益、所述第二ISP增益和所述红外灯强度的第二控制排序，按照所述第二控制排序依次对所述红外图像进行曝光处理，直至所述红外图像达到所述第二曝光效果时停止。
  11. 根据权利要求10所述的方法，其特征在于，所述第一曝光效果为：所述可见光图像经曝光处理后的画面亮度值与所述可见光图像的第一曝光目标值之间的差值小于预设第一阈值；
    所述第二曝光效果为：所述红外图像经曝光处理后的画面亮度值与所述红外图像的第二曝光目标值之间的差值小于预设第二阈值。
  12. 一种双光图像自动曝光装置，其特征在于，所述装置应用双光图像相机，所述双光图像相机包括依次连接的图像传感器和处理器，其中，所述图像传感器的数量为一个，且所述双光图像相机还配置有红外截止滤镜；
    所述装置配置在所述处理器内，所述装置包括：
    原始图像获取模块，配置成获取所述图像传感器采集的原始图像；
    逻辑分光模块，配置成对所述原始图像进行逻辑分光处理，得到红外图像和可见光图像；
    滤镜判断模块，配置成判断当前是否使用所述红外截止滤镜；
    单光曝光控制模块，配置成如果使用所述红外截止滤镜，采用单光曝光算法对所述可见光图像进行曝光处理，以得到符合目标曝光效果的可见光图像；
    双光曝光控制模块，配置成如果不使用所述红外截止滤镜，采用双光曝光算法对所述红外图像和所述可见光图像进行曝光处理，以得到符合第一曝光效果的可见光图像和符合第二曝光效果的红外图像；其中，所述第一曝光效果和所述目标曝光效果相同或不同。
  13. 根据权利要求12所述的装置，其特征在于，所述滤镜判断模块配置成：
    判断当前环境亮度是否大于预设亮度；
    如果是，确定使用所述红外截止滤镜；如果否，确定不使用所述红外截止滤镜。
  14. 根据权利要求12所述的装置，其特征在于，所述双光曝光控制模块配置成：
    获取所述可见光图像的第一亮度值和所述红外图像的第二亮度值；
    获取所述可见光图像的第一曝光目标值和所述红外图像的第二曝光目标值；
    根据所述第一亮度值和所述第一曝光目标值，计算所述可见光图像的第一曝光参数；
    根据所述第二亮度值和所述第二曝光目标值，计算所述红外图像的第二曝光参数；
    根据所述第一曝光参数和所述第二曝光参数，对所述可见光图像和所述红外图像进行曝光处理。
  15. 一种双光图像相机，其特征在于，包括：依次连接的图像传感器和处理器；其中，所述图像传感器的数量为一个，且所述双光图像相机还配置有红外截止滤镜；
    所述处理器上设置有权利要求12至14任一项所述的双光图像自动曝光装置。
  16. 根据权利要求15所述的相机，其特征在于，所述图像传感器之前还设置有镜头；其中，所述镜头为红外共焦镜头。
  17. 根据权利要求16所述的相机，其特征在于，所述镜头的透光率高于预设透光率。
  18. 根据权利要求15所述的相机，其特征在于，所述处理器还连接有补光器；
    所述补光器包括红外补光灯和/或白光补光灯。
  19. 根据权利要求15所述的相机，其特征在于，所述图像传感器为RGB-IR传感器。
  20. 一种机器存储介质，其特征在于，所述机器可读存储介质存储有机器可执行指令，所述机器可执行指令在被处理器调用和执行时，所述机器可执行指令促使所述处理器执行权利要求1至11任一项所述的方法。
PCT/CN2018/105802 2018-09-14 2018-09-14 双光图像自动曝光方法、装置、双光图像相机及机器存储介质 WO2020051898A1 (zh)

Priority Applications (7)

Application Number Priority Date Filing Date Title
HUE18933406A HUE066105T2 (hu) 2018-09-14 2018-09-14 Automatikus exponálási eljárás és berendezés kettõs megvilágítású képekhez, kettõs megvilágítású kamera és számítógépes adathordozó
PCT/CN2018/105802 WO2020051898A1 (zh) 2018-09-14 2018-09-14 双光图像自动曝光方法、装置、双光图像相机及机器存储介质
PL18933406.3T PL3852350T3 (pl) 2018-09-14 2018-09-14 Sposób oraz urządzenie do automatycznej ekspozycji dla obrazu w podwójnym spektrum, oraz kamera do obrazów w podwójnym spektrum oraz maszynowy nośnik pamięci
US17/271,407 US11689785B2 (en) 2018-09-14 2018-09-14 Dual-spectrum image automatic exposure method and apparatus, and dual-spectrum image camera and machine storage medium
CN201880096822.0A CN112602316B (zh) 2018-09-14 2018-09-14 双光图像自动曝光方法、装置、双光图像相机及机器存储介质
EP18933406.3A EP3852350B1 (en) 2018-09-14 2018-09-14 Automatic exposure method and apparatus for dual-light image, and dual-light image camera and machine storage medium
ES18933406T ES2971615T3 (es) 2018-09-14 2018-09-14 Método y aparato de exposición automática para imagen de doble espectro y cámara de imagen de doble espectro y medio de almacenamiento de máquina

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/105802 WO2020051898A1 (zh) 2018-09-14 2018-09-14 双光图像自动曝光方法、装置、双光图像相机及机器存储介质

Publications (1)

Publication Number Publication Date
WO2020051898A1 true WO2020051898A1 (zh) 2020-03-19

Family

ID=69778129

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/105802 WO2020051898A1 (zh) 2018-09-14 2018-09-14 双光图像自动曝光方法、装置、双光图像相机及机器存储介质

Country Status (7)

Country Link
US (1) US11689785B2 (zh)
EP (1) EP3852350B1 (zh)
CN (1) CN112602316B (zh)
ES (1) ES2971615T3 (zh)
HU (1) HUE066105T2 (zh)
PL (1) PL3852350T3 (zh)
WO (1) WO2020051898A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112532890A (zh) * 2020-11-02 2021-03-19 浙江大华技术股份有限公司 曝光控制方法、摄像设备及计算机可读存储介质
CN113286094A (zh) * 2021-05-27 2021-08-20 重庆紫光华山智安科技有限公司 图像自动曝光方法、装置、设备及介质
WO2021223245A1 (en) * 2020-05-08 2021-11-11 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for controlling exposure level, and computer usable medium storing software for implementing method
CN114093015A (zh) * 2022-01-21 2022-02-25 杭州魔点科技有限公司 针对人脸识别的智能补光方法、电子设备及存储介质
CN114143421A (zh) * 2020-09-04 2022-03-04 聚晶半导体股份有限公司 双传感器摄像系统及其校准方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3091114B1 (fr) * 2018-12-19 2021-06-11 Valeo Comfort & Driving Assistance dispositif de capture d’images et système de surveillance d’un conducteur associé
CN114549382B (zh) * 2022-02-21 2023-08-11 北京爱芯科技有限公司 一种红外图像与可见光图像融合的方法及系统
CN116962886A (zh) * 2022-04-07 2023-10-27 安霸国际有限合伙企业 用于rgb-ir传感器的智能自动曝光控制
CN115802169B (zh) * 2022-11-29 2024-04-30 福州鑫图光电有限公司 一种基于亮度直方图的自动曝光方法及终端

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105554485A (zh) * 2015-12-18 2016-05-04 广东欧珀移动通信有限公司 成像方法、成像装置及电子装置
CN107197170A (zh) * 2017-07-14 2017-09-22 维沃移动通信有限公司 一种曝光控制方法及移动终端
US20170330053A1 (en) * 2016-05-11 2017-11-16 Center For Integrated Smart Sensors Foundation Color night vision system and operation method thereof
CN107438170A (zh) * 2016-05-25 2017-12-05 杭州海康威视数字技术股份有限公司 一种图像透雾方法及实现图像透雾的图像采集设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120154596A1 (en) 2009-08-25 2012-06-21 Andrew Augustine Wajs Reducing noise in a color image
US8379123B2 (en) 2010-12-13 2013-02-19 Research In Motion Limited System and method of capturing low-light images on a mobile device
US9143704B2 (en) 2012-01-20 2015-09-22 Htc Corporation Image capturing device and method thereof
JP6264234B2 (ja) * 2014-09-02 2018-01-24 株式会社Ｊｖｃケンウッド 撮像装置、撮像装置の制御方法及び制御プログラム
JP6185213B2 (ja) * 2015-03-31 2017-08-23 富士フイルム株式会社 撮像装置及び撮像装置の画像処理方法及びプログラム
CN106060364A (zh) * 2016-07-28 2016-10-26 浙江宇视科技有限公司 光学透雾彩色图像采集方法及摄像机
CN107370917B (zh) * 2017-06-30 2020-01-10 Oppo广东移动通信有限公司 控制方法、电子装置和计算机可读存储介质
CN108377340B (zh) * 2018-05-10 2024-07-05 浙江芯劢微电子股份有限公司 一种基于rgb-ir传感器日夜模式自动切换方法及装置
JP7057221B2 (ja) * 2018-05-24 2022-04-19 キヤノン株式会社 撮像装置、その制御方法、および制御プログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105554485A (zh) * 2015-12-18 2016-05-04 广东欧珀移动通信有限公司 成像方法、成像装置及电子装置
US20170330053A1 (en) * 2016-05-11 2017-11-16 Center For Integrated Smart Sensors Foundation Color night vision system and operation method thereof
CN107438170A (zh) * 2016-05-25 2017-12-05 杭州海康威视数字技术股份有限公司 一种图像透雾方法及实现图像透雾的图像采集设备
CN107197170A (zh) * 2017-07-14 2017-09-22 维沃移动通信有限公司 一种曝光控制方法及移动终端

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3852350A4 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021223245A1 (en) * 2020-05-08 2021-11-11 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for controlling exposure level, and computer usable medium storing software for implementing method
CN114143421A (zh) * 2020-09-04 2022-03-04 聚晶半导体股份有限公司 双传感器摄像系统及其校准方法
CN114143421B (zh) * 2020-09-04 2024-04-05 聚晶半导体股份有限公司 双传感器摄像系统及其校准方法
CN112532890A (zh) * 2020-11-02 2021-03-19 浙江大华技术股份有限公司 曝光控制方法、摄像设备及计算机可读存储介质
CN113286094A (zh) * 2021-05-27 2021-08-20 重庆紫光华山智安科技有限公司 图像自动曝光方法、装置、设备及介质
CN114093015A (zh) * 2022-01-21 2022-02-25 杭州魔点科技有限公司 针对人脸识别的智能补光方法、电子设备及存储介质

Also Published As

Publication number Publication date
EP3852350B1 (en) 2024-01-31
CN112602316A (zh) 2021-04-02
US11689785B2 (en) 2023-06-27
US20210314501A1 (en) 2021-10-07
ES2971615T3 (es) 2024-06-06
CN112602316B (zh) 2022-06-24
PL3852350T3 (pl) 2024-06-10
EP3852350A1 (en) 2021-07-21
EP3852350A4 (en) 2022-03-23
HUE066105T2 (hu) 2024-07-28

Similar Documents

Publication Publication Date Title
WO2020051898A1 (zh) 双光图像自动曝光方法、装置、双光图像相机及机器存储介质
CN110536051B (zh) 摄像设备及其控制方法和存储介质
WO2021077963A1 (zh) 图像融合的方法、装置、电子设备及可读存储介质
US8106965B2 (en) Image capturing device which corrects a target luminance, based on which an exposure condition is determined
US20160227100A1 (en) Dual camera systems and methods for rapid 3a convergence and high dynamic range exposure metering
CN109474770B (zh) 一种成像装置及成像方法
CN102300049A (zh) 图像信号处理装置
CN113286094B (zh) 图像自动曝光方法、装置、设备及介质
CN114189634B (zh) 图像采集方法、电子设备及计算机存储介质
WO2019019904A1 (zh) 白平衡处理方法、装置和终端
CN110868548B (zh) 一种图像处理方法及电子设备
US8964106B2 (en) Exposure control device and electronic camera having the same
US11687316B2 (en) Audio based image capture settings
US10977777B2 (en) Image processing apparatus, method for controlling the same, and recording medium
WO2020133331A1 (en) Systems and methods for exposure control
JP7247609B2 (ja) 撮像装置、撮像方法およびプログラム
US11568526B2 (en) Dual sensor imaging system and imaging method thereof
CN114143418B (zh) 双传感器摄像系统及其摄像方法
JP2011223378A (ja) 電子カメラ
US11616907B2 (en) Image capturing apparatus, image capturing system, control method therefor, and storage medium
US11270421B2 (en) Image processing apparatus and image processing method that reduce image noise, and storage medium
CN110072050B (zh) 曝光参数的自适应调整方法、装置及一种拍摄设备
US20190052803A1 (en) Image processing system, imaging apparatus, image processing apparatus, control method, and storage medium
CN118264914A (zh) 一种摄像装置
JP2017225036A (ja) 撮像装置及び画像処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18933406

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018933406

Country of ref document: EP

Effective date: 20210414