WO2023193712A1 - 一种多光谱处理方法和多光谱探测装置 - Google Patents

一种多光谱处理方法和多光谱探测装置 Download PDF

Info

Publication number
WO2023193712A1
WO2023193712A1 PCT/CN2023/086214 CN2023086214W WO2023193712A1 WO 2023193712 A1 WO2023193712 A1 WO 2023193712A1 CN 2023086214 W CN2023086214 W CN 2023086214W WO 2023193712 A1 WO2023193712 A1 WO 2023193712A1
Authority
WO
WIPO (PCT)
Prior art keywords
switching device
control circuit
spectrum
pixel unit
row
Prior art date
Application number
PCT/CN2023/086214
Other languages
English (en)
French (fr)
Inventor
胡佳
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023193712A1 publication Critical patent/WO2023193712A1/zh

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G01J3/027—Control of working procedures of a spectrometer; Failure detection; Bandwidth calculation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2803—Investigating the spectrum using photoelectric array detector
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G01J2003/2826—Multispectral imaging, e.g. filter imaging

Definitions

  • Embodiments of the present application relate to the field of chip technology, and in particular, to a multispectral processing method and a multispectral detection device.
  • Multispectral technology is gradually being applied to ambient light detection and material identification, and provides richer information for accurate color restoration.
  • ADC Analog-to-Digital Converter
  • IPD Imaging Photon Detector
  • the Si surface of each IPD is coated with interference filtering, and the number and thickness of the film system are designed and manufactured according to the designed bandpass spectrum to form the final multispectral detector.
  • the total area of the chip is fixed.
  • the area occupied by each pixel includes the IPD structure, and each pixel needs to occupy a larger area. Therefore, the number of pixels is limited and the resolution of the image is low, making it impossible to collect high-resolution signals under the acceptable target surface of chips with architectures such as mobile phones.
  • the high gain of the ADC serves the dark light environment.
  • the ADC that collects current and the above-mentioned photoelectric conversion process have no means to suppress noise (such as correlated double sampling, etc.), and the noise caused by dark current and other factors
  • the level is poor, the dark light performance is poor, and it is unable to achieve effective brightness in ultra-night environments, detection of spectral information, and high-quality imaging.
  • Embodiments of the present application provide a multispectral processing method and a multispectral detection device, which can support high-quality multispectral imaging with small pixels and low noise in multispectral processing, achieve high-resolution signal collection, and improve low-light performance.
  • a multispectral detection device includes a logic control circuit, a plurality of spectrum arrays, a row and column control circuit, a plurality of current-collecting analog-to-digital converters ADC (integral type ADC2) and a plurality of collection voltages.
  • ADC analog-to-digital converter
  • each spectrum array includes a plurality of single-channel spectral units, and each of the plurality of single-channel spectral units includes a plurality of pixel units; for each of the plurality of pixel units Pixel unit, each pixel unit is coupled with a column control circuit, one of a plurality of current-collecting ADCs for collecting current, and one of a plurality of voltage-collecting ADCs for collecting voltage; each pixel unit includes a photodetector PD , a voltage detection circuit and a current detection circuit, the voltage detection circuit includes a first switching device, the current detection circuit includes a second switching device; a logic control circuit for controlling the first switching device and the second switching device in each pixel unit. Imaging or ambient light detection is performed in different working modes; the logic control circuit is also used to control the row and column control circuits to adopt different signal merging methods for multiple pixel units in multiple spectrum arrays. Perform imaging enhancement or ambient light detection enhancement.
  • this application through a newly designed circuit structure of a pixel unit, the working mode of the pixel unit can be switched as needed, specifically through the conduction and switching of the first switching device and the second switching device. Off determines the operating mode of each pixel unit for imaging or ambient light detection.
  • this application combines pixel units in multiple frequency arrays through row and column control circuits to perform ambient light detection enhancement or imaging enhancement in a variety of combination methods. In this way, through multiple combinations of new pixel units in multiple frequency arrays, high-quality imaging and highly dynamic ambient light detection at high resolution can be achieved, that is, small pixels and low noise can be supported in multispectral processing. High-quality multispectral imaging enables high-resolution signal acquisition and improved low-light performance.
  • the logic control circuit is used to control the first switching device in each pixel unit to turn on and the second switching device to turn off, so as to convert the current of the PD in each pixel unit into photoelectric conversion.
  • the signal is transmitted to the ADC that collects the voltage through the first switching device for imaging; or, a logic control circuit is used to control the first switching device in each pixel unit to turn off and the second switching device to turn on, so that each pixel unit
  • the current signal after photoelectric conversion by the PD in the device is transmitted to the ADC that collects current through the second switching device for ambient light detection.
  • the photoelectrically converted current signal when performing multispectral imaging, can be combined and input into the ADC that collects voltage through the conduction of the first switching device in the voltage detection circuit; when performing multispectral ambient light detection When the current detection circuit is turned on, the photoelectrically converted current signal can be combined and input into the ADC that collects current to achieve high-quality imaging of multiple spectra and highly dynamic ambient light detection. .
  • the current detection circuit in each pixel unit also includes a current mirror
  • the voltage detection circuit in each pixel unit also includes a reset tube, a source follower tube, and a row selector tube; for each pixel unit,
  • the output terminal of the PD is coupled to the first terminal of the first switching device and the first terminal of the second switching device.
  • the second terminal of the first switching device is coupled to the first terminal of the reset tube and the first terminal of the source follower tube.
  • the source The second end of the follower tube is coupled with the first end of the row selection tube, the second end of the row selection tube is coupled with the first input end of the row and column control circuit, the first output end of the row and column control circuit is coupled with the ADC that collects the voltage, and the row and column control circuit is coupled with the ADC that collects the voltage.
  • the second output end of the control circuit is coupled to the ADC that collects current; the second end of the second switching device is coupled to the first end of the current mirror; the first output end of the current mirror is coupled to the second input end of the column control circuit.
  • multiple purposes such as multispectral imaging or ambient light detection can be achieved by switching the current detection circuit and the voltage detection circuit.
  • the logic control circuit is used to: when it is determined to detect ambient light intensity, control the first switching device in each pixel unit to turn off and the second switching device to turn on; using a single spectrum array as unit, the control row and column control circuit combines the signals of all the single-channel spectral units in each spectrum array as a group, and the signal after combining the signals in one group is used to indicate the ambient light intensity of a spectrum array area. Therefore, through the ambient light intensity detection method provided by this application, for each pixel unit, part of the circuit (including the capacitor C) coupled to the first switching device in the pixel unit is not working, and no signal clamping occurs. ) brings about the saturation problem and is suitable for high dynamic range light detection. Moreover, this application can also be used to detect ambient light of different intensities through different current addition methods.
  • the logic control circuit is used to: when it is determined to detect ambient light intensity, control the first switching device in each pixel unit to turn off and the second switching device to turn on; control the row and column control circuit to Multiple spectrum arrays are divided into regions, and the single-channel spectral units in all spectrum arrays in each region are combined as a group for signal combination.
  • the signal after the signal combination is performed in one group is used to indicate the ambient light intensity of a spectrum array area. In this way, by further merging and dividing multiple spectrum arrays, noise can be reduced or light intensity detection can be increased. Measurement sensitivity.
  • the logic control circuit is used to: when it is determined to detect the type of ambient light, control the first switching device in each pixel unit to turn off and the second switching device to turn on; control the row and column control circuit to Multiple spectrum arrays are divided into regions, and the same single-channel spectral units in all spectrum arrays in each area are combined as a group for signal integration to obtain the combined signal of each single-channel spectral unit in each area.
  • the combined signal of a group is used to indicate an ambient light type. Therefore, the ambient light type detection method in ultra-high dynamic scenes provided by this application can be applied to different ambient light brightness scenarios by partitioning multiple spectrum arrays and using a flexible signal merging method to achieve ambient light type detection.
  • This multi-region, highly dynamic, multi-channel spectral information can be used to assist other cameras in regional environment type detection and regional AWB parameter adaptation, assisting in more accurate color restoration of single or mixed color temperature scenes.
  • the logic control circuit is used to: when it is determined to perform multispectral imaging, control the first switching device in each pixel unit to turn on and the second switching device to turn off; control the row and column control circuit to switch multiple Multiple pixel units of each single-channel spectral unit of each spectrum array in the spectrum array are combined as a group to obtain a combined signal of each single-channel spectrum unit of each spectrum array.
  • Each unit of each spectrum array is The combined signals of the channel spectral units are used to perform multispectral imaging.
  • the multispectral imaging method provided by this application can be used for hyperspectral imaging.
  • high-resolution and high-dynamic multispectral information color restoration and image effects of higher color gamut can be achieved.
  • this application can achieve high-resolution hyperspectral imaging, take into account resolution and color accuracy, achieve low noise, and is suitable for high-quality image restoration and night scenes. Improved effectiveness.
  • the logic control circuit is used to: when it is determined to perform multispectral imaging, control the first switching device in each pixel unit to turn on and the second switching device to turn off; control the row and column control circuit to switch multiple Multiple single-channel spectral units of each spectrum array in the spectrum array are grouped, and each group includes multiple adjacent single-channel spectrum units; the row and column control circuit is controlled to combine the signals of each group of each spectrum array to obtain each The combined signal of each group of each spectral array is used for multispectral imaging.
  • This method of merging different spectral channels adjacent to each other, although sacrificing a certain color gamut space, can take into account the improvement of color and dynamic range during multi-spectral imaging.
  • signal merging includes: charge merging, or analog domain merging, or digital domain merging. But it is not limited to these three merging methods.
  • the logic control circuit is also used to: when determining that the ambient light intensity of the previous frame image is greater than or equal to the preset threshold, when acquiring the signal of each pixel unit of the subsequent frame image, reduce each Exposure and/or gain of the pixel unit; when it is determined that the ambient light intensity of the previous frame image is less than the preset threshold, when acquiring the signal of each pixel unit of the next frame image, increase the exposure and/or gain of each pixel unit gain.
  • the readout method of each pixel unit at the bottom layer can dynamically adapt to low gain or high gain based on the brightness detection information of the previous frame, further meeting the needs of the dynamic range of light intensity detection.
  • a multispectral processing method is provided, which is applied to a multispectral detection device.
  • the multispectral detection device includes a logic control circuit, multiple spectrum arrays, row and column control circuits, multiple analog-to-digital converters ADCs for collecting current, and multiple An ADC that collects voltage; each spectrum array includes a plurality of single-channel spectral units, and each of the plurality of single-channel spectral units includes a plurality of pixel units; for each pixel unit of the plurality of pixel units, Each pixel unit is coupled to a row-column control circuit, one of a plurality of current-collecting ADCs for collecting current, and one of a plurality of voltage-collecting ADCs for collecting voltage; each pixel unit includes a photodetector PD, a voltage detection circuit and a current detection circuit, the voltage detection circuit includes a first switching device, and the current detection circuit includes a second switching device.
  • the method includes: a multispectral detection device control logic control circuit for the first switching device and the second switching device in each pixel unit to perform imaging or ambient light detection in different working modes; the multispectral detection device control logic control circuit
  • the row and column control circuit adopts different signal merging methods for multiple pixel units in multiple spectrum arrays to perform imaging enhancement or ambient light detection enhancement.
  • the multispectral detection device control logic control circuit is used for the first switching device and the second switching device in each pixel unit to perform imaging or ambient light detection in different working modes, including: multispectral detection
  • the device control logic control circuit is used to turn on the first switching device in each pixel unit and turn off the second switching device, so as to transmit the current signal after photoelectric conversion by the PD in each pixel unit through the first switching device.
  • ADC that collects the voltage for imaging; or, the multispectral detection device controls the logic control circuit to turn off the first switching device in each pixel unit and turn on the second switching device so that the PD in each pixel unit is photoelectrically
  • the converted current signal is transmitted to the ADC that collects current through the second switching device for ambient light detection.
  • the current detection circuit in each pixel unit also includes a current mirror
  • the voltage detection circuit in each pixel unit also includes a reset tube, a source follower tube, and a row selector tube; for each pixel unit,
  • the output terminal of the PD is coupled to the first terminal of the first switching device and the first terminal of the second switching device.
  • the second terminal of the first switching device is coupled to the first terminal of the reset tube and the first terminal of the source follower tube.
  • the source The second end of the follower tube is coupled with the first end of the row selection tube, the second end of the row selection tube is coupled with the first input end of the row and column control circuit, the first output end of the row and column control circuit is coupled with the ADC that collects the voltage, and the row and column control circuit is coupled with the ADC that collects the voltage.
  • the second output end of the control circuit is coupled to the ADC that collects current; the second end of the second switching device is coupled to the first end of the current mirror; the first output end of the current mirror is coupled to the second input end of the column control circuit.
  • multiple pairs of row and column control circuits use different signal merging methods for multiple pixel units in multiple spectrum arrays to enhance imaging or ambient light detection, including: when determining to detect ambient light intensity, Control the first switching device in each pixel unit to turn off and the second switching device to turn on; taking a single spectrum array as a unit, control the row and column control circuit to combine all single-channel spectral units in each spectrum array as a group for signal merging , the signal combined by a group is used to indicate the ambient light intensity of a spectrum array area.
  • multiple pairs of row and column control circuits use different signal merging methods for multiple pixel units in multiple spectrum arrays to enhance imaging or ambient light detection, including: when determining to detect ambient light intensity, Control the first switching device in each pixel unit to turn off and the second switching device to turn on; control the row and column control circuits to divide multiple spectrum arrays into regions, and treat the single-channel spectrum units in all spectrum arrays in each region as one
  • the signals are combined into groups, and the signal after combining the signals into one group is used to indicate the ambient light intensity of a spectrum array area.
  • multiple pairs of row and column control circuits use different signal merging methods for multiple pixel units in multiple spectrum arrays to enhance imaging or ambient light detection, including: when determining to detect the type of ambient light, Control the first switching device in each pixel unit to turn off and the second switching device to turn on; control the row and column control circuits to divide multiple spectrum arrays into regions, and use the same single-channel spectrum unit in all spectrum arrays in each region as One group performs signal merging to obtain the combined signal of each single-channel spectral unit in each area. The signal after one group in each area combines signals is used to indicate an ambient light type.
  • multiple pairs of row and column control circuits use different signal merging methods for multiple pixel units in multiple spectral arrays to enhance imaging or ambient light detection, including: when determining to perform multispectral imaging, control The first switching device in each pixel unit is turned on, and the second switching device is turned off; the row and column control circuit is controlled to signal multiple pixel units of each single-channel spectrum unit of each spectrum array in the multiple spectrum arrays as a group. Combined, the combined signal of each single-channel spectral unit of each spectrum array is obtained, and the combined signal of each single-channel spectral unit of each spectrum array is used for multispectral imaging.
  • multiple pairs of row and column control circuits use different signal merging methods for multiple pixel units in multiple spectral arrays to enhance imaging or ambient light detection, including: when determining to perform multispectral imaging, control The first switching device in each pixel unit is turned on, and the second switching device is turned off; the row and column control circuit is controlled to group multiple single-channel spectral units of each spectrum array in multiple spectrum arrays, and each grouping includes adjacent Multiple single-channel spectrum units; control the row and column control circuit to combine the signals of each group of each spectrum array to obtain the combined signal of each group of each spectrum array, and the combined signal of each group of each spectrum array is used for Perform multispectral imaging.
  • signal merging includes: charge merging, or analog domain merging, or digital domain merging.
  • the method also includes: when determining that the ambient light intensity of the previous frame image is greater than or equal to a preset threshold, when acquiring the signal of each pixel unit of the subsequent frame image, reducing the exposure and/or gain of each pixel unit; determining When the ambient light intensity of the previous frame image is less than the preset threshold, when acquiring the signal of each pixel unit of the subsequent frame image, the exposure and/or gain of each pixel unit is increased.
  • embodiments of the present application provide a computer-readable storage medium that includes computer instructions.
  • the computer instructions When the computer instructions are run on an electronic device, the electronic device causes the electronic device to execute the above second aspect and any possible implementation of the second aspect. multispectral processing methods.
  • embodiments of the present application provide a computer program product.
  • the computer program product When the computer program product is run on a computer or processor, it causes the computer or processor to execute the above second aspect and any possible implementation of the second aspect. multispectral processing methods.
  • a chip including the multispectral detection device as in the first aspect and any possible implementation of the first aspect.
  • any multi-spectral detection device, core, computer-readable storage medium or computer program product provided above can be applied to the corresponding method provided above, therefore, the beneficial effects it can achieve can be Please refer to the beneficial effects of the corresponding method, which will not be described again here.
  • Figure 1 is a schematic diagram of a system architecture for capturing images provided by an embodiment of the present application
  • Figure 2 is a schematic diagram of the partial circuit structure of a multispectral detector in an image sensor provided by an embodiment of the present application;
  • Figure 3 is a schematic diagram of a partial circuit structure of a multispectral detector provided by an embodiment of the present application.
  • Figure 4 is a schematic structural diagram of a multispectral detection device provided by an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of a multispectral detection device provided by an embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a multispectral detection device provided by an embodiment of the present application.
  • Figure 7 is a schematic structural diagram of each pixel unit in a multispectral detection device provided by an embodiment of the present application.
  • Figure 8 is a schematic flow chart of an ambient light intensity detection method provided by an embodiment of the present application.
  • Figure 9 is a schematic diagram of merging single-channel spectral units in each spectrum array into an all-pass channel according to an embodiment of the present application.
  • Figure 10 is a schematic flow chart of an ambient light type detection method provided by an embodiment of the present application.
  • Figure 11 is a schematic diagram of merging single-channel spectral units in each spectrum array provided by an embodiment of the present application.
  • Figure 12 is a schematic diagram of the same single-channel spectral unit merger provided by the embodiment of the present application.
  • Figure 13 is a schematic flow chart of a multispectral imaging method provided by an embodiment of the present application.
  • Figure 14 is a schematic diagram of merging pixel units in a spectrum array provided by an embodiment of the present application.
  • Figure 15 is a schematic diagram of signal merging of different single-channel spectral units in a spectrum array provided by an embodiment of the present application.
  • Figure 16 is a schematic diagram of the architecture of a multispectral detection device provided by an embodiment of the present application.
  • first and second are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as “first” and “second” may explicitly or implicitly include one or more of these features. In the description of this embodiment, unless otherwise specified, “plurality” means two or more.
  • the optical image generated by the lens device (Lens) of the scene can be projected onto the surface of the image sensor, and then converted into an electrical signal, which is amplified by the pre-amplifier circuit and automatically controlled by Chroma.
  • Control, ACC) and analog to digital converter (analog to digital, A/D) the digital image signal converted is transmitted to the digital signal processing chip for processing, and then transmitted through the input/output (Input/Output, I/O) interface It is processed in the Central Processing Unit (CPU), and finally the image is output to the display screen.
  • CPU Central Processing Unit
  • White balance refers to the camera's ability to restore white objects under different light source lighting conditions.
  • the physical image restoration captured by the camera should have the same characteristics as the human eye observing the measured object under the same lighting conditions. The colors match. Therefore, during the white balance process, ambient light detection is required to provide accurate color restoration of the image.
  • multispectral technology is gradually being applied to ambient light detection.
  • it can also be used in processes such as material identification to provide richer information for accurate color restoration.
  • multispectral technology has gradually become a mainstream solution.
  • ambient light detection can be achieved based on IPD and ADC that collects current, and each photosensitive channel achieves specific spectrum screening with the help of interference filtering.
  • IPD image sensor
  • ADC analog to digital converter
  • each photosensitive channel achieves specific spectrum screening with the help of interference filtering.
  • FIG 2 it is a part of the circuit of the image sensor.
  • This part of the circuit can be understood as the core circuit of the multispectral detector.
  • it can also include the core circuit.
  • the PD in the figure is used to respond to the received optical signal of the transmitted band, that is, to hit the filtered spectrum to the IPD.
  • the IPD can respond to the optical signal, conduct photoelectric conversion of the optical signal, and obtain the current signal and transmit it to this part of the circuit.
  • each pixel position can correspond to an IPD.
  • the circuits in the circuit shown in Figure 2 except the IPD can be considered as ADC circuits that collect current.
  • the multispectral detector includes multiple IPDs and multiple ADCs that collect current. circuit, each IPD is coupled with an ADC circuit that collects current, and one ADC circuit that collects current can be coupled with multiple IPDs.
  • the silicon (Si) surface of each IPD can be coated with interference filtering, and the number and thickness of the film system are designed and manufactured according to the designed bandpass spectrum to form the final multispectral detector.
  • the high-gain noise level of the ADC is poor, so it is impossible to achieve high-resolution signal collection under an acceptable target surface for mobile phones and other architectures.
  • the low-light performance is poor, and it is impossible to achieve effective brightness in a super-night environment. and detection of spectral information.
  • multispectral detectors can achieve high-resolution, low-noise photoelectric conversion and signal quantification based on PIN photodiodes, 4T readout structures and ramp-type ADCs.
  • the dynamic range of the multispectral detector shown in Figure 3 is limited by the FD capacitance. Even if it relies on a multi-capacitance switching design, it cannot achieve high dynamic capabilities that are not saturated under sunlight in a limited area.
  • the dynamic range can be understood as the maximum tonal range from the blackest to the whitest that the camera can capture, indicating the camera's ability to record grayscale levels.
  • this application proposes a multispectral detection device.
  • the circuit structure of a single pixel and the circuit structure around the pixel are different from the existing ones.
  • the current signal flow direction after photoelectric conversion can be realized through switch control, supporting small pixels and low noise.
  • this application can provide a higher resolution multispectral device, and it is a multifunctional multispectral device that can combine imaging, light source type detection and light source intensity detection. Specifically, this can be achieved through the new pixel structure provided by this application and the new combined readout method.
  • the new pixel combination readout method proposed in this application can realize a variety of functions, such as high-resolution multi-spectral high-dynamic imaging, which can achieve better color effects, safety and health detection during imaging; high-dynamic Multi-area ambient light intensity detection can be used to assist in the effect upgrade of high-dynamic range images (High-Dynamic Range, HDR); high-dynamic multi-area ambient light type detection can be used to assist in partition color restoration accuracy and effect upgrade.
  • existing multispectral devices can be upgraded. As an independent multi-functional camera, it can bring benefits to system-level high dynamics and accurate colors at a controllable cost.
  • the multispectral detection device can be used in high-resolution, multi-functional hyperspectral detectors, and can be specifically used in a variety of usage scenarios.
  • multispectral cameras used in mobile phone applications can achieve high-resolution multispectral imaging and improve color accuracy;
  • multi-partition ambient light type detection that can be flexibly configured according to the scene can assist in achieving fine-partitioned automatic white balance (Automatic white balance). , AWB), to achieve accurate color restoration of mixed color temperature scenes;
  • multi-zone ambient light intensity detection that can be flexibly configured according to the scene, assists in achieving more accurate automatic exposure (Auto exposure, AE) control, and assists in achieving more accurate detection of highly dynamic scenes High Dynamic Range ratio (HDR) configuration.
  • Multispectral-based skin quality detection, face anti-counterfeiting detection, etc. can also be implemented on mobile phones.
  • multispectral cameras can also be used in mobile phones, watches, and accessories to implement health detection such as material composition detection, blood sugar, blood oxygen, and blood pressure.
  • it can also be used in whole-house intelligence and vehicle-mounted fields to achieve living body detection and assist in providing information required for safety scenarios.
  • the multispectral detection device provided by this application can be applied to photosensitive sensing devices such as image sensors.
  • the multispectral detection device provided by this application is introduced below.
  • FIG. 4 shows a multispectral detection device 40 provided by the present application.
  • the multispectral detection device includes a logic control circuit, multiple spectrum arrays, row and column control circuits (SWAP SEL), multiple ADCs for collecting current and multiple ADC that collects voltage.
  • SWAP SEL row and column control circuits
  • the ADC that collects current in this application may be, for example, an integrating ADC or other current-collecting type ADCs; the ADC that collects voltage in this application may be, for example, a step-type ADC or other voltage-collecting type ADCs.
  • Each spectrum array includes a plurality of single-channel spectral units, and each of the plurality of single-channel spectral units includes a plurality of pixel units.
  • each spectrum array can be understood as a minimum unit for ambient light type detection and a minimum unit for ambient light intensity detection.
  • the array composed of C 11 , C 12 , C 13 ,..., C mn can be understood as a spectrum array, in which a single C 11 or C 12 ,..., or C mn , etc. is a single-channel spectral unit , or called a spectrum channel, m and n are integers equal to or equal to 1.
  • Each C 11 or C 12 , ..., or C mn includes multiple pixel units of the same spectrum channel: (1, 1), (1, 2), ..., (h, k), that is, (h, k) represents A minimum pixel unit indicating the location information of the pixel unit.
  • ADC group 1 represents multiple ADCs that collect voltages
  • ADC group 2 represents multiple digital-to-analog converters ADCs that collect current.
  • each pixel unit is connected with a row and column control circuit, one of the plurality of current-collecting ADCs, one of the current-collecting ADCs (ADC2), and a plurality of voltage-collecting ADCs.
  • One of the voltage-collecting ADCs (ADC1) is coupled.
  • Each pixel unit includes a photodetector PD, a voltage detection circuit and a current detection circuit.
  • the voltage detection circuit includes a first switching device TX1, and the current detection circuit includes a second switching device TX2;
  • a logic control circuit for controlling the first switching device TX1 and the second switching device TX2 in each pixel unit to perform imaging or ambient light detection in different working modes;
  • the logic control circuit is also used to control the row and column control circuit SWAP SEL using different signal merging methods for multiple pixel units in multiple spectrum arrays to perform imaging enhancement or ambient light detection enhancement.
  • the working mode of the pixel unit can be switched as needed, specifically through the conduction of the first switching device TX1 and the second switching device TX2. On and off determine the operating mode of each pixel unit for imaging or ambient light detection.
  • this application combines pixel units in multiple frequency arrays through row and column control circuits to perform ambient light detection enhancement or imaging enhancement in a variety of combination methods.
  • high-quality imaging at high resolution and highly dynamic environments can be achieved through multiple combinations of new pixel units in multiple frequency arrays.
  • Light detection supports high-quality multispectral imaging with small pixels and low noise in multispectral processing, achieving high-resolution signal acquisition and improving low-light performance.
  • the logic control circuit is used to control the first switching device TX1 in each pixel unit to turn on and the second switching device TX2 to turn off, so as to convert the current of the PD in each pixel unit into photoelectric conversion.
  • the signal is transmitted to the ADC (ADC1) that collects the voltage through the first switching device TX1 for imaging; or, a logic control circuit is used to control the first switching device TX1 in each pixel unit to turn off and the second switching device TX2 to turn on, In order to transmit the current signal after photoelectric conversion by the PD in each pixel unit to the ADC (ADC2) that collects current through the second switching device TX2 for ambient light detection.
  • this application can improve the high-resolution signal collection under the target surface that can be accepted by mobile phones and other structures through the arrangement of multiple spectrum arrays.
  • the circuit structure of a single pixel unit provided by this application is different from the existing circuit structure of a single pixel.
  • the current signal after photoelectric conversion by the PD in a single pixel unit provided by this application can be transmitted to ADC1 through the first switching device TX1, or to ADC2 through the second switching device TX2. It is not a single pixel unit as in the prior art.
  • the circuit structure of the current signal can only be transmitted through one type of ADC.
  • the single-channel spectral units in multiple spectrum arrays can be combined according to the preset grouping type. That is to say, combined with the new circuit structure of a single pixel unit provided by this application, a single-channel spectrum based on single-channel spectrum can be obtained.
  • the multiple grouping types of the unit can realize the detection of ambient light type or intensity under multiple grouping types. It can also realize ultra-high dynamic range ambient light detection, solving the existing problem of being unable to perform high-resolution imaging and sunlight. and other ultra-high dynamic range ambient light detection problems.
  • the current detection circuit in each pixel unit also includes a current mirror M, and each pixel unit
  • the voltage detection circuit also includes a reset tube RST, a source follower tube SF, a capacitor C and a row selector tube SEL1.
  • the output terminal a of the PD is coupled to the first terminal b of the first switching device TX1 and the first terminal c of the second switching device TX2, and the second terminal d of the first switching device TX1 is coupled to the first terminal d of the reset tube RST.
  • the first terminal e is coupled to the first terminal f of the source follower tube SF
  • the second terminal g of the source follower tube SF is coupled to the first terminal h of the row selector tube SEL1
  • the second terminal i of the row selector tube SEL1 is connected to the row and column control circuit.
  • the first input terminal j of the (SWAP SEL) is coupled, the first output terminal k of the row and column control circuit (SWAP SEL) is coupled to the ADC (ADC1) that collects the voltage, and the second output terminal l of the row and column control circuit (SWAP SEL) is coupled to the acquisition
  • ADC1 ADC1
  • SWAP SEL ADC1
  • ADC2 ADC2
  • the current ADC (ADC2) is coupled;
  • the second terminal m of the second switching device TX2 is coupled with the first terminal n of the current mirror M, and the first output terminal o of the current mirror M is coupled with the second input of the column control circuit (SWAP SEL) terminal p coupling.
  • the capacitor C is coupled between the input terminal of the PD and the second terminal d of the first switching device TX1.
  • the two row and column control circuits SWAP SEL shown in Figure 7 are actually the same one, that is, the same SWAP SEL is coupled with at least one ADC1 and with at least one ADC2.
  • the bitLine in Figure 7 refers to the column signal line connected to the second terminal g of the source follower SF, and the iLine refers to the column signal line connected to the o terminal of the current mirror M.
  • each pixel unit in addition to PD, the first switching device TX1 and the second switching device TX2, it also includes a current mirror M, a reset tube RST, a source follower tube SF, and a capacitor C. and row EAC SEL1.
  • the number of row and column control circuits (SWAP SEL) can also be at least one.
  • Each pixel unit is coupled to one of at least one row and row control circuit (SWAP SEL).
  • Each row and column control circuit (SWAP SEL) in the (SWAP SEL) is coupled to one of the plurality of ADC1 and coupled to one of the plurality of ADC2.
  • the logic control circuit can control the current signal after photoelectric conversion by the PD in the pixel unit to flow through the first switching device TX1 or the second switching device TX2.
  • the current signal flows through the first switching device TX1
  • the current signal can be transmitted to the source follower SF through TX1
  • the source follower SF can output the received current signal to the row selector SEL1 without loss.
  • the row selector SEL1 is used in the spectrum array to determine that when the pixel unit of a certain row is selected according to the instructions of the logic control circuit, the current signal of the pixel unit of the row can be output to the row and column control circuit (SWAP SEL).
  • the row and column control circuit (SWAP SEL) is used in the spectrum array to combine the signals of the single-channel spectrum unit according to the preset combination type according to the instructions of the logic control circuit.
  • the current signal flows through the second switching device TX2
  • the current signal can be mirrored and transmitted to the column control circuit (SWAP SEL) through the current mirror M, so that the column control circuit (SWAP SEL) can perform the single-channel spectrum unit according to the instructions of the logic control circuit.
  • Preset combination types for signal merging are possible.
  • the multispectral detection device Based on the multispectral detection device provided by this application, the following is a scenario in which the multispectral detection device is applied to ambient light intensity detection scenarios in ultra-high dynamic scenes, ambient light type detection scenarios in ultra-high dynamic scenarios, and high-resolution and high-dynamic multispectral imaging scenarios. Introduced separately.
  • the multispectral detection device provided by this application is not limited to these three scenarios, and can also be used in other scenarios, which is not limited by this application.
  • Embodiments of the present application provide an ambient light intensity detection method for ultra-high dynamic scenes, as shown in Figure 8.
  • the method includes:
  • the multispectral detection device determines to perform ambient light intensity detection, it combines the single-channel spectral units in each of the multiple spectrum arrays into an all-pass channel, obtains the combined signals of the multiple all-pass channels, and reads them out.
  • the logic control circuit in the multispectral detection device when the logic control circuit in the multispectral detection device receives the ambient light intensity detection indication, since the digital-to-analog converter ADC that collects current has a higher dynamic range detection capability, the logic control circuit can control multiple spectrums.
  • the first switching device TX1 of each pixel unit is turned off, and the second switching device TX2 is turned on, and instructs the row and column control circuit SWAP SEL.
  • the row and column control circuit is controlled to switch all the switching devices in each spectrum array.
  • the single-channel spectral unit combines signals as a group, and the signal after combining signals in a group is used to indicate the ambient light intensity of a frequency array area.
  • the schematic diagram is shown in Figure 9. It can be understood that the single-channel spectral units C 11 , C 12 , C in a spectrum array are 13 ,...,C mn are combined into an all-pass channel to obtain the all-pass channel (1,1) in Figure 9, and the single-channel spectral units C 11 , C 12 , C 13 ,..., C in another spectrum array are mn is combined into an all-pass channel to obtain the all-pass channel (1, 2) in Figure 9.
  • Available all-pass channels include (1,1), (1,2), ..., (x, y).
  • the merging method is not limited to charge merging, analog domain merging, and digital domain merging.
  • the charge merger can be understood as the charge merger of multiple single-channel spectral units C 11 , C 12 , C 13 , ..., C mn , specifically the pixels in the single-channel spectral unit C 11 Charges of units (1, 1), (1 , 2), ..., (h, k), pixel units (1, 1), (1, 2), ..., (h, k), the charges of the pixel units (1, 1), (1, 2), ..., (h, k) in the single-channel spectral unit C 13 , ..., the pixel units ( in the single-channel spectral unit C mn The combination of charges of 1,1), (1,2),..., (h,k).
  • the combination of the charges of the two pixel units can be understood as turning on the second switching device TX2 coupled to the two pixel units at the same time, so that the photoelectrically converted current signals of the two pixel units are transmitted to the same current path. transmitted to the row and column control circuit SWAP SEL.
  • the current signal of the all-pass channel can be obtained after the charges of each single-channel spectrum unit in the same spectrum array are combined.
  • the simulation domain merging can be understood as the simulation domain merging of multiple single-channel spectral units C 11 , C 12 , C 13 , ..., C mn , specifically the pixels in the single-channel spectral unit C 11 Analog domain signals of units (1,1), (1,2),...,(h,k), pixel units (1,1), (1,2),...,( in single channel spectral unit C 12 Analog domain signals of h, k), analog domain signals of pixel units (1, 1), (1, 2), ..., (h, k) in single-channel spectral unit C 13 , ..., single-channel spectral unit C Combination of analog domain signals of pixel units (1, 1), (1, 2), ..., (h, k) in mn .
  • the analog domain merging of two pixel units can be understood as the current signal after photoelectric conversion of each pixel unit can be transmitted on its own current path, but the analog signal is combined when transmitted to ADC2.
  • the analog signals of the pixel units involved in each spectrum array are combined at ADC2 to obtain the simulation of the all-pass channel after combining the analog signals of each single-channel spectrum unit in the same spectrum array.
  • the digital domain merging can be understood as the digital domain merging of multiple single-channel spectral units C 11 , C 12 , C 13 , ..., C mn , specifically the pixels in the single-channel spectral unit C 11 Digital domain signals of units (1,1), (1,2), ... ,(h,k), pixel units (1,1), (1,2),...,( h, k) digital domain signals, digital domain signals of pixel units (1, 1), (1, 2), ..., (h, k) in the single-channel spectral unit C 13 , ..., single-channel spectral unit Cmn Combination of digital domain signals of pixel units (1,1), (1,2),..., (h,k) in .
  • the digital domain merging of two pixel units can be understood as the current signal after photoelectric conversion of each pixel unit can be transmitted to ADC2 on its own current path, and the merging of digital signals after analog-to-digital conversion by ADC2 (for example, The result after performing a weighted average).
  • the current signals of the pixel units involved in each spectrum array are accumulated (combined) into the digital signals after analog-to-digital conversion in ADC2, and the digital signals of each single-channel spectrum unit in the same spectrum array can be obtained.
  • the accumulated digital signal of the all-pass channel is mapped to digital signals after analog-to-digital conversion by ADC2
  • the multispectral detection device determines that the combined signal read out by each all-pass channel is less than or equal to the preset threshold, it continues to divide multiple all-pass channels at equal intervals to obtain multiple grouped combined signals and read them out.
  • the logic control circuit in the multispectral detection device determines that the combined signal read out by each all-pass channel is less than the preset threshold, it can be considered that C 11 , C 12 ,..., C mn in an all-pass channel
  • the ambient light intensity judged by the result of signal merging is still very weak.
  • the multiple all-pass channels (1, 1), (1, 2), ..., (x, y) obtained in Figure 9 can be further processed, etc. Interval division, multiple groups are obtained.
  • the merging method is not limited to charge merging, analog domain merging, digital domain merging, etc.
  • the combined signals of the (x/k1, y/k2) groups (pixel areas) obtained after division can be quantified by at least one ADC2 in ADC group 2 to obtain the ambient light intensity of different groups or pixel areas, or It's called ambient light brightness.
  • Step 802 is equivalent to controlling the first switching device in each pixel unit to turn off and the second switching device to turn on when the logic control circuit is used to determine ambient light intensity detection.
  • the row and column control circuit is controlled to divide multiple spectrum arrays into regions, and the single-channel spectral units in all spectrum arrays in each region are regarded as one division Groups are used to combine signals, and the signal after combining signals in one group is used to indicate the ambient light intensity of a spectrum array area.
  • the logic control circuit of the present application when used to perform signal readout of each pixel unit at the bottom layer, can dynamically adapt low gain or high gain according to the ambient light intensity of the previous frame image, so as to Further meet the demand for dynamic range of light intensity detection.
  • the logic control circuit determines that the ambient light intensity of the previous frame image is greater than or equal to the preset threshold, when acquiring the signal of each pixel unit of the subsequent frame image, the exposure and/or gain of each pixel unit is reduced;
  • the exposure and/or gain of each pixel unit is increased.
  • the gains of multiple pixel units in multiple spectrum arrays are not exactly the same.
  • the logic control circuit determines that the ambient light intensity of the previous frame image is relatively strong, the next frame of image can be acquired before the next frame image is obtained.
  • the gain of the pixel unit can be appropriately lowered; conversely, when the ambient light intensity of the previous frame image is weak, the gain of the pixel unit can be appropriately increased.
  • This multi-gain reading method of the underlying pixel unit can further enhance the dynamic range capability of ambient light detection.
  • the ambient light intensity detection method provided by this application, for each pixel unit, part of the circuit (including the capacitor C) coupled to the first switching device TX1 in the pixel unit does not work, and no signal clamping ( The saturation problem caused by clamp) is suitable for high dynamic range light detection. Moreover, this application can also be used to detect ambient light of different intensities through different current addition methods (step 801 or step 802).
  • This multi-region, highly dynamic ambient light intensity information in the ambient light intensity detection solution provided by this application can be used to assist other cameras in accurately and quickly judging AE or accurately configuring AE in different HDR ratios of HDR modes, and can also assist HDR fusion. algorithm to adapt a more appropriate compression curve.
  • Embodiments of the present application provide a method for detecting ambient light types in ultra-high dynamic scenes, as shown in Figure 10.
  • the method includes:
  • the multispectral detection device determines to perform ambient light type detection, it controls the first switching device in each pixel unit to be turned off and the second switching device to be turned on.
  • the logic control unit determines to perform ambient light type detection, it can control the first switching device TX1 in each pixel unit to turn off, the second switching device TX2 to turn on, and photoelectrically convert the PD in each pixel unit.
  • the current signal flows to the row and column control circuit through TX2 and current mirror M.
  • the multispectral detection device divides multiple spectrum arrays into regions, and combines the same single-channel spectral units in all spectrum arrays in each area as a group to obtain the merger of each single-channel spectrum unit in each area. Signals, a grouped signal in each area, are combined to indicate an ambient light type.
  • the logic control circuit may instruct the row and column control circuit to divide the multiple spectrum arrays into regions.
  • the row and column control circuit does not need to combine multiple single-channel spectral units in each spectrum array, but combines the signals of the same single-channel spectral unit in each area.
  • a total of 16 spectrum arrays are divided into 4 partitions, or 4 groups: 1, 2, 3 and 4.
  • 1 partition includes (1,1), (1,2), (2,1) and (2,2) 4 spectrum arrays
  • 2 partition includes (1,3), (1,4), (2 , 3) and (2, 4) 4 spectrum arrays
  • 3 partition includes (3, 1), (3, 2), (4, 1) and (4, 2) 4 spectrum arrays
  • 4 partition includes (3 , 3), (3, 4), (4, 3) and (4, 4) 4 spectrum arrays.
  • the same single-channel spectral units in these four spectrum arrays can be combined as a group for signal merging. That is to say, the signals of the pixel units in the four single-channel spectral units C 11 in the four spectrum arrays are combined, and the signals of the pixel units in the four single-channel spectral units C 21 in the four spectrum arrays are combined. Merge, combine the signals of the pixel units in the four single-channel spectral units C 31 in the four spectrum arrays,..., combine the signals of the pixel units in the four single-channel spectral units C mn in the four spectrum arrays merge. In this way, the combined signal of each single-channel spectral unit in the 1 partition can be obtained. For the other three partitions, similar signal merging can be performed.
  • the merging method of the same single-channel spectral unit or the same spectrum channel in each partition is not limited to the above-mentioned charge merging, analog domain merging and digital domain merging.
  • the logic control circuit can adapt the division and merging method according to the requirements of dynamic range and ambient light type detection accuracy, that is, appropriately determine the values of k1 and k2. For example, when the last frame of image imaging was performed, the determined ambient light intensity was relatively strong, and the values of k1 and k2 could be appropriately increased, that is, the number of spectrum arrays in each partition was smaller. When the last frame image is imaged and the determined ambient light intensity is weak, the values of k1 and k2 can be appropriately reduced, that is, the number of spectrum arrays in each partition is larger to improve the ambient light type detection accuracy.
  • the logic control unit can also control the first switching device TX1 to turn on and the second switching device TX2 to turn off. In this way, the combined signal in step 1002 is quantized by ADC1. This is due to the strong quantification ability of ADC1 for dark light.
  • the signal readout method of each pixel unit at the bottom layer can dynamically adapt to low gain or high gain based on the ambient light brightness detection information of the previous frame image, further meeting the requirements for the dynamic range of light intensity detection.
  • the specific implementation is similar to the gain adaptation process in step 802.
  • the ambient light type detection method in ultra-high dynamic scenes provided by this application can be applied to different ambient light brightness scenarios by partitioning multiple spectrum arrays and using a flexible signal merging method to achieve ambient light type detection.
  • This multi-region, highly dynamic, multi-channel spectral information can be used to assist other cameras in regional environment type detection and regional AWB parameter adaptation, assisting in more accurate color restoration of single or mixed color temperature scenes.
  • Embodiments of the present application provide a multispectral imaging method, as shown in Figure 13.
  • the method includes:
  • the multispectral detection device determines to perform multispectral imaging, it controls the first switching device in each pixel unit to be turned on and the second switching device to be turned off.
  • the ADC that collects voltage that is, ADC1 in this application, can be used for correlated double sampling quantification of voltage signals
  • the logic control circuit determines that multispectral imaging is to be performed
  • the ADC in each single-channel spectral unit in the spectrum array can be The current signal after photoelectric conversion by the pixel unit flows to the ADC group 1 that collects the voltage.
  • Control the row and column control circuit to combine the multiple pixel units of each single-channel spectral unit of each spectrum array in the multiple spectrum arrays as a group to obtain the combined signal of each single-channel spectrum unit of each spectrum array.
  • the combined signal of each single-channel spectral unit of each spectral array is used to perform multispectral imaging.
  • the logic control circuit can control each pixel unit of each single-channel spectral unit of each spectrum array to perform photoelectric conversion and individually input the current signal to the ADC1 to be read out to obtain a digital signal. That is, the current signals after photoelectric conversion of the underlying pixel units of the same spectrum channel can be individually output to ADC1 and quantized to improve image resolution in high-brightness scenes.
  • the logic control circuit may control the row and column control circuit to convert multiple pixel units in each single-channel spectral unit of each spectrum array in the plurality of spectrum arrays. (pixel unit array) is grouped or partitioned, and signals are combined into the groups or groups.
  • the row and column control circuit can combine the signals of multiple pixels in each single-channel spectral unit into a group of four adjacent pixel units, as shown in Figure 14 for pixel units (1, 1), ( 1, 2) (2, 1) and (2, 2) are a group for signal merging, pixel units (1, k-1), (1, k) (2, k-1) and (2, k) Combine signals for a group, and the combining methods are not limited to charge combining, analog domain combining, or digital domain combining.
  • the logic control circuit may control the column control circuit to group multiple single-channel spectral units of each spectrum array in the multiple spectrum arrays, and each group Contains multiple adjacent single-channel spectral units.
  • the row and column control circuit is controlled to combine the signals of each group of each spectrum array to obtain a combined signal of each group of each spectrum array, and the combined signal of each group of each spectrum array is used for multispectral imaging.
  • the row and column control circuit can combine signals from different single-channel spectral units in each spectrum array.
  • the row and column control circuit can combine the signals of four adjacent single-channel spectrum units C 11 , C 12 , C 22 and C 22 in a spectrum array as a group.
  • the single-channel spectral units C 13 , C 14 , C 23 and C 24 are used as a group for signal merging. Although this will sacrifice a certain color gamut space, it can improve the color and dynamic range of the image.
  • the signal readout method of each pixel unit at the bottom layer can dynamically adapt low gain or high gain, or a fusion of high and low gain, for each pixel unit based on the brightness information of the previous frame image.
  • the underlying pixel units of the same single-channel spectral unit can also be configured with different exposures or gains, and perform high dynamic synthesis compression.
  • Bayer image remosaic of multiple single-channel spectral units can be accomplished on-chip or platform side.
  • the multispectral imaging method provided by this application can be used for hyperspectral imaging.
  • high-resolution and high-dynamic multispectral information color restoration and image effects of higher color gamut can be achieved.
  • this application can achieve high-resolution hyperspectral imaging, take into account resolution and color accuracy, achieve low noise, and is suitable for high-quality image restoration and night scenes. Improved effectiveness.
  • the multispectral detection device includes corresponding hardware and/or software modules for performing each function.
  • this application can It can be implemented in hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions in conjunction with the embodiments for each specific application, but such implementations should not be considered to be beyond the scope of this application.
  • This embodiment can divide the electronic device into functional modules according to the above method examples.
  • each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
  • the above integrated modules can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. In actual implementation, there may be other division methods.
  • Figure 16 shows a possible composition diagram of the multispectral detection device 160 involved in the above embodiment.
  • the multispectral detection device 160 can It includes: switch control unit 1601 and signal combining unit 1602.
  • the signal combining unit 1602 may be used to support the multispectral detection device 160 to perform the above steps 801, 802, 1002, 1302, etc., and/or other processes for the technology described herein.
  • the switch control unit 1601 may be used to support the multispectral detection device 160 to perform the above steps 1001, 1301, etc., and/or other processes for the technology described herein.
  • the multispectral detection device 160 provided in this embodiment is used to perform the above multispectral processing method, and therefore can achieve the same effect as the above implementation method.
  • the multispectral detection device 160 may include a processing module, a storage module, and a communication module.
  • the processing module may be used to control and manage the actions of the multispectral detection device 160. For example, it may be used to support the multispectral detection device 160 to perform the steps performed by the switch control unit 1601 and the signal combining unit 1602.
  • the storage module can be used to support the multispectral detection device 160 to store program codes, data, etc.
  • the communication module can be used to support communication between the multispectral detection device 160 and other devices.
  • the processing module may be a processor or a controller. It may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with this disclosure.
  • a processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc.
  • the storage module may be a memory.
  • the communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip and other devices that interact with other electronic devices.
  • Embodiments of the present application also provide a computer storage medium.
  • Computer instructions are stored in the computer storage medium.
  • the electronic device causes the electronic device to execute the above related method steps to achieve the antenna gain in the above embodiment. Adjustment method.
  • Embodiments of the present application also provide a computer program product.
  • the computer program product When the computer program product is run on a computer, it causes the computer to perform the above related steps to implement the antenna gain adjustment method performed by the electronic device in the above embodiment.
  • inventions of the present application also provide a device.
  • This device may be a chip, a component or a module.
  • the device may include a connected processor and a memory.
  • the memory is used to store computer execution instructions.
  • the processor can execute computer execution instructions stored in the memory, so that the chip executes the multispectral processing method executed by the electronic device in each of the above method embodiments.
  • the multispectral detection device, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects it can achieve can be referred to the above provided The beneficial effects of the corresponding methods will not be described again here.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of modules or units is only a logical function division.
  • there may be other division methods for example, multiple units or components may be The combination can either be integrated into another device, or some features can be omitted, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated.
  • the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. . Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the above integrated units can be implemented in the form of hardware or software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium.
  • the technical solutions of the embodiments of the present application are essentially or contribute to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium , including several instructions to cause a device (which can be a microcontroller, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program code.

Landscapes

  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Spectrometry And Color Measurement (AREA)

Abstract

本申请实施例提供一种多光谱处理方法和多光谱探测装置，涉及芯片技术领域，能够在多光谱处理中支持小像素、低噪声的高质量多光谱成像，实现高分辨率信号采集和提升暗光性能。具体方案为：在多个频谱阵列中，每个像素单元包括PD、电压探测电路和电流探测电路，电压探测电路包括第一开关器件，电流探测电路包括第二开关器件；逻辑控制电路，用于控制每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测；逻辑控制电路，还用于控制行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强。本申请实施例用于多光谱成像和多光谱环境光探测。

Description

一种多光谱处理方法和多光谱探测装置
本申请要求于2022年04月06日提交国家知识产权局、申请号为202210357214.9、申请名称为“一种多光谱处理方法和多光谱探测装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及芯片技术领域，尤其涉及一种多光谱处理方法和多光谱探测装置。
背景技术
多光谱技术逐渐被应用于环境光检测和物质识别，并且为精准的颜色还原提供更加丰富的信息。目前，多见基于标准的成像光子探测器(Imaging Photon Detector，IPD)结构进行光电转化，形成电流，并采集电流的模数转换器(Analog-to-Digital Converter，ADC)进行电流信号数据的量化，得到光谱信息。其中，每个IPD的Si表面进行了干涉滤波镀膜，并针对设计的带通频谱进行膜系层数和厚度设计和制造，形成最终的多光谱探测器。
但是，对于多光谱传感器所在的芯片来说，芯片的总面积是固定的，每个像素(pixel)占用的面积包括IPD结构，每个pixel需要占用较大的面积。因此，像素的数量受到限制，图像的分辨率较低，无法实现手机等架构的芯片可接受靶面下的高分辨率信号采集。而且，ADC的高增益是为暗光环境服务的，在暗光环境下，采集电流的ADC及上述光电转化过程，没有抑制噪声的手段(如相关双采样等)，暗电流等因素导致的噪声水平较差，暗光性能较差，无法实现超夜环境的有效亮度以及光谱信息的探测和高质量成像。
发明内容
本申请实施例提供一种多光谱处理方法和多光谱探测装置，能够在多光谱处理中支持小像素、低噪声的高质量多光谱成像，实现高分辨率信号采集和提升暗光性能。
[根据细则91更正 07.04.2023]
为达到上述目的，本申请实施例采用如下技术方案。
第一方面，提供一种多光谱探测装置，多光谱探测装置包括逻辑控制电路、多个频谱阵列、行列控制电路、多个采集电流的模数转换器ADC(积分型ADC2)和多个采集电压的ADC(阶跃，ADC1)；每个频谱阵列包括多个单通道光谱单元，多个单通道光谱单元中的每个单通道光谱单元包括多个像素单元；对于多个像素单元中的每个像素单元，每个像素单元与行列控制电路、多个采集电流的ADC中的一个采集电流的ADC和多个采集电压的ADC中的一个采集电压的ADC耦合；每个像素单元包括光电探测器PD、电压探测电路和电流探测电路，电压探测电路包括第一开关器件，电流探测电路包括第二开关器件；逻辑控制电路，用于控制每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测；逻辑控制电路，还用于控制行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式， 进行成像增强或环境光探测增强。
由此，在本申请中，通过新设计的一种像素单元的电路结构，可使得像素单元的的工作模式可根据需要进行切换，具体可通过第一开关器件和第二开关器件的导通和关断确定每个像素单元的工作模式用于成像或环境光探测。在此基础上，本申请通过对多个频率阵列中的像素单元通过行列控制电路进行合并，在多种合并方式下进行环境光探测增强或成像增强。这样，通过新像素单元在多个频率阵列中的多种合并方式读出，可实现高分辨率下的高质量成像和高动态的环境光探测，即在多光谱处理中支持小像素、低噪声的高质量多光谱成像，实现高分辨率信号采集和提升暗光性能。
在一种可能的设计中，逻辑控制电路，用于控制每个像素单元中的第一开关器件导通，第二开关器件关断，以便将每个像素单元中的PD进行光电转化后的电流信号通过第一开关器件传输至采集电压的ADC进行成像；或，逻辑控制电路，用于控制每个像素单元中的第一开关器件关断，第二开关器件导通，以便将每个像素单元中的PD进行光电转化后的电流信号通过第二开关器件传输至采集电流的ADC进行环境光探测。也就是说，在进行多光谱成像时，可通过电压探测电路中的第一开关器件的导通将光电转换后的电流信号进行合并输入到采集电压的ADC中；在进行多光谱的环境光探测时，可通过电流探测电路中的第二卡关器件的导通将光电转换后的电流信号进行合并输入到采集电流的ADC中，以实现过多光谱的高质量成像和高动态的环境光探测。
在一种可能的设计中，每个像素单元中的电流探测电路还包括电流镜，每个像素单元中的电压探测电路还包括复位管、源跟随管和行选管；对于每个像素单元，PD的输出端与第一开关器件的第一端和第二开关器件的第一端耦合，第一开关器件的第二端与复位管的第一端和源跟随管的第一端耦合，源跟随管的第二端与行选管的第一端耦合，行选管的第二端与行列控制电路的第一输入端耦合，行列控制电路的第一输出端与采集电压的ADC耦合，行列控制电路的第二输出端与采集电流的ADC耦合；第二开关器件的第二端与电流镜的第一端耦合，电流镜的第一输出端与行列控制电路的第二输入端耦合。对于每个像素单元来说，通过电流探测电路和电压探测电路的切换，可实现多光谱成像或环境光探测等多种目的。
在一种可能的设计中，逻辑控制电路，用于：在确定进行环境光强度探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通；以单个频谱阵列为单位，控制行列控制电路将每个频谱阵列中的所有单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。由此，通过本申请提供的环境光强探测方法，对于每个像素单元，该像素单元中与第一开关器件耦合的部分电路(包括电容C)并未工作，不会产生信号钳位(clamp)带来的饱和问题，适用于高动态范围的光感探测。而且，本申请还可以通过不同的电流加和方式，可用于不同强度的环境光的探测需要。
在一种可能的设计中，逻辑控制电路，用于：在确定进行环境光强度探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通；控制行列控制电路对多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中的单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。这样，通过对多个频谱阵列进一步的合并划分，可降噪或者增加光强度探 测感度。
在一种可能的设计中，逻辑控制电路，用于：在确定进行环境光类型探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通；控制行列控制电路对多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中相同的单通道光谱单元作为一个分组进行信号合并，得到每个区域中每种单通道光谱单元的合并信号，每个区域中的一个分组进行信号合并后的信号用于指示一种环境光类型。由此，本申请提供的超高动态场景的环境光类型探测方法，可通过对多个频谱阵列进行分区，借助灵活的信号合并方式，可适用于不同环境光亮度场景，达成环境光类型的探测精度和分辨率的平衡选择，且不会产生信号clamp带来的饱和问题，适用于高动态光感探测。这种多区域高动态多通道光谱信息，可以用于辅助其他camera进行分区域的环境类型检测和分区域的AWB参数适配，辅助更精准的单一或混合色温场景的颜色还原。
在一种可能的设计中，逻辑控制电路，用于：在确定进行多光谱成像时，控制每个像素单元中的第一开关器件导通，第二开关器件关闭；控制行列控制电路将多个频谱阵列中每个频谱阵列的每个单通道光谱单元的多个像素单元作为一个分组进行信号合并，得到每个频谱阵列的每个单通道光谱单元的合并信号，每个频谱阵列的每个单通道光谱单元的合并信号用于进行多光谱成像。
由此，本申请提供的多光谱成像方法，可用于高光谱成像，通过高分辨高动态多光谱信息，可实现更高色域的颜色还原和图像效果。相比现有的方案中，无法有效和灵活地兼顾多光谱和高动态效果，本申请可实现高分辨高光谱成像，兼顾解析力和颜色精度，实现低噪声，适用于高质量图像还原以及夜景效果提升。
在一种可能的设计中，逻辑控制电路，用于：在确定进行多光谱成像时，控制每个像素单元中的第一开关器件导通，第二开关器件关闭；控制行列控制电路将多个频谱阵列中每个频谱阵列的多个单通道光谱单元进行分组，每个分组包括相邻的多个单通道光谱单元；控制行列控制电路将每个频谱阵列的每个分组进行信号合并，得到每个频谱阵列的每个分组的合并信号，每个频谱阵列的每个分组的合并信号用于进行多光谱成像。这种通过不同频谱通道相邻再做合并的方式，虽然牺牲一定的色域空间，但是可以兼顾多光谱成像时颜色和动态范围的提升。
在一种可能的设计中，信号合并包括：电荷合并、或模拟域合并或数字域合并。但不限于这三种合并方式。
在一种可能的设计中，逻辑控制电路还用于：确定前一帧图像的环境光强度大于或等于预设阈值时，在获取后一帧图像的每个像素单元的信号时，降低每个像素单元的曝光和/或增益；确定前一帧图像的环境光强度小于预设阈值时，在获取后一帧图像的每个像素单元的信号时，抬高每个像素单元的曝光和/或增益。这样，最底层每像素单元的读出方式可以根据上一帧的亮度探测信息，动态的适配低增益或者高增益，进一步满足光强探测动态范围的需求。
第二方面，提供一种多光谱处理方法，应用于多光谱探测装置，多光谱探测装置包括逻辑控制电路、多个频谱阵列、行列控制电路、多个采集电流的模数转换器ADC和多个采集电压的ADC；每个频谱阵列包括多个单通道光谱单元，多个单通道光谱单元中的每个单通道光谱单元包括多个像素单元；对于多个像素单元中的每个像素单元， 每个像素单元与行列控制电路、多个采集电流的ADC中的一个采集电流的ADC和多个采集电压的ADC中的一个采集电压的ADC耦合；每个像素单元包括光电探测器PD、电压探测电路和电流探测电路，电压探测电路包括第一开关器件，电流探测电路包括第二开关器件。该方法包括：多光谱探测装置控制逻辑控制电路用于每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测；多光谱探测装置控制逻辑控制电路对行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强。
在一种可能的设计中，多光谱探测装置控制逻辑控制电路用于每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测包括：多光谱探测装置控制逻辑控制电路用于每个像素单元中的第一开关器件导通，第二开关器件关断，以便将每个像素单元中的PD进行光电转化后的电流信号通过第一开关器件传输至采集电压的ADC进行成像；或，多光谱探测装置控制逻辑控制电路用于每个像素单元中的第一开关器件关断，第二开关器件导通，以便将每个像素单元中的PD进行光电转化后的电流信号通过第二开关器件传输至采集电流的ADC进行环境光探测。
在一种可能的设计中，每个像素单元中的电流探测电路还包括电流镜，每个像素单元中的电压探测电路还包括复位管、源跟随管和行选管；对于每个像素单元，PD的输出端与第一开关器件的第一端和第二开关器件的第一端耦合，第一开关器件的第二端与复位管的第一端和源跟随管的第一端耦合，源跟随管的第二端与行选管的第一端耦合，行选管的第二端与行列控制电路的第一输入端耦合，行列控制电路的第一输出端与采集电压的ADC耦合，行列控制电路的第二输出端与采集电流的ADC耦合；第二开关器件的第二端与电流镜的第一端耦合，电流镜的第一输出端与行列控制电路的第二输入端耦合。
在一种可能的设计中，多对行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：在确定进行环境光强度探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通；以单个频谱阵列为单位，控制行列控制电路将每个频谱阵列中的所有单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。在一种可能的设计中，多对行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：在确定进行环境光强度探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通；控制行列控制电路对多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中的单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。
在一种可能的设计中，多对行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：在确定进行环境光类型探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通；控制行列控制电路对多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中相同的单通道光谱单元作为一个分组进行信号合并，得到每个区域中每种单通道光谱单元的合并信号，每个区域中的一个分组进行信号合并后的信号用于指示一种环境光类型。
在一种可能的设计中，多对行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：在确定进行多光谱成像时，控制每个像素单元中的第一开关器件导通，第二开关器件关闭；控制行列控制电路将多个频谱阵列中每个频谱阵列的每个单通道光谱单元的多个像素单元作为一个分组进行信号合并，得到每个频谱阵列的每个单通道光谱单元的合并信号，每个频谱阵列的每个单通道光谱单元的合并信号用于进行多光谱成像。
在一种可能的设计中，多对行列控制电路对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：在确定进行多光谱成像时，控制每个像素单元中的第一开关器件导通，第二开关器件关闭；控制行列控制电路将多个频谱阵列中每个频谱阵列的多个单通道光谱单元进行分组，每个分组包括相邻的多个单通道光谱单元；控制行列控制电路将每个频谱阵列的每个分组进行信号合并，得到每个频谱阵列的每个分组的合并信号，每个频谱阵列的每个分组的合并信号用于进行多光谱成像。
在一种可能的设计中，信号合并包括：电荷合并、或模拟域合并或数字域合并。
该方法还包括：确定前一帧图像的环境光强度大于或等于预设阈值时，在获取后一帧图像的每个像素单元的信号时，降低每个像素单元的曝光和/或增益；确定前一帧图像的环境光强度小于预设阈值时，在获取后一帧图像的每个像素单元的信号时，抬高每个像素单元的曝光和/或增益。
第三方面，本申请实施例提供了一种计算机可读存储介质，包括计算机指令，当计算机指令在电子设备上运行时，使得电子设备执行上述第二方面及第二方面任一项可能的实现方式中的多光谱处理方法。
第四方面，本申请实施例提供了一种计算机程序产品，当计算机程序产品在计算机或处理器上运行时，使得计算机或处理器执行上述第二方面及第二方面任一项可能的实现方式中的多光谱处理方法。
第五方面，提供一种芯片，包括如第一方面及第一方面任一项可能的实现方式中的多光谱探测装置。
可以理解的是，上述提供的任一多光谱探测装置、芯、计算机可读存储介质或计算机程序产品等均可以应用于上文所提供的对应的方法，因此，其所能达到的有益效果可参考对应的方法中的有益效果，此处不再赘述。
本申请的这些方面或其他方面在以下的描述中会更加简明易懂。
附图说明
图1为本申请实施例提供的一种拍摄图像的系统架构示意图；
图2为本申请实施例提供的一种图像传感器中的多光谱探测器部分电路结构示意图；
图3为本申请实施例提供的一种多光谱探测器部分电路结构示意图；
图4为本申请实施例提供的一种多光谱探测装置的结构示意图；
图5为本申请实施例提供的一种多光谱探测装置的结构示意图；
图6为本申请实施例提供的一种多光谱探测装置的结构示意图；
图7为本申请实施例提供的一种多光谱探测装置中每个像素单元的结构示意图；
图8为本申请实施例提供的一种环境光强探测方法流程示意图；
图9为本申请实施例提供的一种将每个频谱阵列中的单通道光谱单元合并为全通通道后的示意图；
图10为本申请实施例提供的一种环境光类型探测方法流程示意图；
图11为本申请实施例提供的一种将每个频谱阵列中的单通道光谱单元合并示意图；
图12为本申请实施例提供的一种相同的单通道光谱单元合并示意图；
图13为本申请实施例提供的一种多光谱成像方法流程示意图；
图14为本申请实施例提供的一种频谱阵列中像素单元的合并示意图；
图15为本申请实施例提供的一种频谱阵列中不同的单通道光谱单元进行信号合并的示意图；
图16为本申请实施例提供的一种多光谱探测装置架构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，在本申请实施例的描述中，“多个”是指两个或多于两个。
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在拍摄图像时，如图1所示，景物可通过透镜装置(Lens)生成的光学图像投射到图像传感器的表面，然后转化为电信号，经过预中放电路放大，自动色度控制(Automatic Chroma Control，ACC)和模拟数字转换器(analog to digital，A/D)转换后的数字图像信号传输给数字信号处理芯片进行加工处理，再经过输入/输出(Input/Output，I/O)接口传输到中央处理器(Central Processing Unit，CPU)中处理，最后将图像输出到显示屏上。
在图像拍摄过程中，需要进行白平衡，白平衡是指摄像头在不同光源照明条件下还原白色物体的能力，摄像头所拍摄的物理的影像还原应具有与人眼在相同照明条件下观察被测物体色彩相符合。因此，在白平衡过程中，需要进行环境光检测，以为图像提供精准的颜色还原。
目前，多光谱技术逐渐被应用于环境光检测，此外，还可以应用于物质识别等过程，为精准的颜色还原提供更加丰富的信息。例如，在科研物质检测设备、美妆行业肤质检测设备、产业界光谱测试设备和手机行业摄像头等领域，多光谱技术逐渐称为主流的解决方案。
例如，在一些多光谱技术中，可基于IPD和采集电流的ADC实现环境光检测，每个感光通道借助于干涉滤波实现特定频谱筛选。如图2所示，为图像传感器的部分电路，这部分电路可理解为多光谱探测器的核心电路，此外，还可以包括该核心电路 外围的逻辑控制电路。图中的PD用于响应接收到的透过的波段的光信号，即将筛选到的频谱打到IPD上，IPD可响应光信号，将光信号进行光电转换，得到电流信号传输到该部分电路中，当电流信号传输至采集电流的ADC时，采集电流的ADC可将电流信号转换为数字信号，数字信号被传输至数字信号处理芯片进行处理。在图像传感器中，每个像素位置可对应一个IPD，图2示出的电路中除IPD以外的电路可认为是采集电流的ADC电路，多光谱探测器包括多个IPD和多个采集电流的ADC电路，每个IPD与一个采集电流的ADC电路耦合，一个采集电流的ADC电路可与多个IPD耦合。
其中，每个IPD的硅(Si)表面可进行干涉滤波镀膜，针对设计的带通频谱进行膜系层数和厚度的设计与制造，形成最终的多光谱探测器。
但是，对于单个像素(pixel)来说，最为最小的感光单元，需要占用较大的面积，
且在暗光场景下，ADC的高增益噪声水平较差，因此无法实现手机等架构可接受靶面下的高分辨率信号采集，同时暗光性能交较差，无法实现超夜环境的有效亮度以及光谱信息的探测。
在另一些多光谱技术中，如图3所示，多光谱探测器可基于PIN光电二极管(pin-photodiode)、4T读出结构和ramp型ADC实现高分辨低噪声的光电转化和信号量化。但是，图3示出的多光谱探测器的动态范围受到FD电容的限制，即使依靠多电容切换设计，也无法在有限的面积下实现太阳光下不饱和的高动态能力。
其中，动态范围可理解为摄像头能够拍摄的从最黑到最白之间的最大的影调范围，表示摄像头记录影响灰阶等级的能力，动态范围越大，说明被捕捉下来的层次越丰富。
由此，本申请提出一种多光谱探测装置，单个像素的电路和像素外围的电路结构与现有的不同，可通过开关控制实现光电转换后的电流信号流向，支持小像素、低噪声的高质量多光谱成像，以及太阳光下不饱和的超高动态范围的环境光源类型探测和环境光强度探测，同时拓展了暗态的探测能力。
其次，本申请可提供一种更高分辨率的多光谱器件，且为多功能的多光谱器件，可兼具成像、光源类型探测和光源强度探测。具体可通过本申请提供的新型的像素结构搭配新型的组合读出方式实现。也就是说，在本申请提出的新型的像素组合读出方式，可实现多种功能，例如包括高分辨多光谱高动态成像，可实现成像时更好的颜色效果、安全和健康检测；高动态多区域环境光强检测，可用于辅助高动态范围图像(High-Dynamic Range，HDR)的效果升级；高动态多区域环境光类型检测，用于辅助分区颜色还原精度和效果升级。这样，可带来现有多光谱器件的升级，作为一颗独立的多功能摄像机(camera)，给系统级的高动态、准确颜色带来收益，并且成本可控。
本申请提供的多光谱探测装置可应用于在高分辨、多功能高光谱探测器，具体可应用于多种使用场景中。例如应用于手机应用的多光谱camera中，可实现高分辨的多光谱成像、提升颜色精度；可依据场景灵活配置的多分区的环境光类型检测，辅助实现精细分区的自动白平衡(Automatic white balance，AWB)，实现混合色温场景的精准颜色还原；可依据场景灵活配置的多分区的环境光强度探测，辅助实现更准确的自动曝光(Auto exposure，AE)控制，辅助实现高动态场景更准确的高动态范围比率(High Dynamic Range ratio，HDR)配置。还可在手机上，实现基于多光谱的肤质检测，人脸防伪检测等。
再例如，还可以应用于手机、手表、配件应用的多光谱camera，实现物质成分检测、血糖、血氧和血压等健康检测。
再例如，还可以应用于全屋智能以及车载领域，实现活体检测，辅助提供安全场景所需的信息。
本申请提供的多光谱探测装置可应用于图像传感器等感光类传感器件中。
下面对本申请提供的多光谱探测装置进行介绍。
如图4所示为本申请提供的一种多光谱探测装置40，该多光谱探测装置包括逻辑控制电路、多个频谱阵列、行列控制电路(SWAP SEL)、多个采集电流的ADC和多个采集电压的ADC。
本申请中采集电流的ADC例如可以为积分型ADC，或为其他采集电流类型的ADC；本申请中采集电压的ADC例如可以为阶跃式ADC，或为其他采集电压类型的ADC。
每个频谱阵列包括多个单通道光谱单元，多个单通道光谱单元中的每个单通道光谱单元包括多个像素单元。
本申请中，每个频谱阵列可理解为环境光类型探测最小单元和环境光强度探测最小单元。
如图5所示，C11、C12、C13、…、Cmn组成的阵列可理解为一个频谱阵列，其中的单个C11或C12、…、或Cmn等为一个单通道光谱单元，或者称为一个频谱通道，m和n为等于或等于1的整数。每个C11或C12、…、或Cmn包括多个相同频谱通道的像素单元：(1，1)、(1，2)、…、(h，k)，即(h，k)表示一个最小像素单元，指示像素单元的位置信息。
ADC组(group)1表示多个采集电压的ADC，ADC组(group)2表示多个采集电流的数模转换器ADC。
如图6所示，对于多个像素单元中的每个像素单元，每个像素单元与行列控制电路、多个采集电流的ADC中的一个采集电流的ADC(ADC2)和多个采集电压的ADC中的一个采集电压的ADC(ADC1)耦合。
每个像素单元包括光电探测器PD、电压探测电路和电流探测电路，所述电压探测电路包括第一开关器件TX1，电流探测电路包括第二开关器件TX2；
逻辑控制电路，用于控制每个像素单元中的第一开关器件TX1和第二开关器件TX2在不同的工作模式下进行成像或环境光探测；
逻辑控制电路，还用于控制行列控制电路SWAP SEL对多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强。
由此，在本申请中，通过新设计的一种像素单元的电路结构，可使得像素单元的的工作模式可根据需要进行切换，具体可通过第一开关器件TX1和第二开关器件TX2的导通和关断确定每个像素单元的工作模式用于成像或环境光探测。
在此基础上，本申请通过对多个频率阵列中的像素单元通过行列控制电路进行合并，在多种合并方式下进行环境光探测增强或成像增强。这样，通过新像素单元在多个频率阵列中的多种合并方式读出，可实现高分辨率下的高质量成像和高动态的环境 光探测，即在多光谱处理中支持小像素、低噪声的高质量多光谱成像，实现高分辨率信号采集和提升暗光性能。
在一些实施例中，逻辑控制电路，用于控制每个像素单元中的第一开关器件TX1导通，第二开关器件TX2关断，以便将每个像素单元中的PD进行光电转化后的电流信号通过第一开关器件TX1传输至采集电压的ADC(ADC1)进行成像；或，逻辑控制电路，用于控制每个像素单元中的第一开关器件TX1关断，第二开关器件TX2导通，以便将每个像素单元中的PD进行光电转化后的电流信号通过第二开关器件TX2传输至采集电流的ADC(ADC2)进行环境光探测。
也就是说，本申请可通过多个频谱阵列的排列，提升手机等架构可接受靶面下的高分辨率信号采集。而且，本申请提供的单个像素单元的电路结构与现有的单个像素的电路结构不同。本申请提供的单个像素单元中的PD进行光电转化后的电流信号可通过第一开关器件TX1向ADC1传输，或通过第二开关器件TX2向ADC2传输，并不是如现有技术中，单个像素单元的电路结构的电流信号只能通过一种类型的ADC传输。
并且，本申请中可通过对多个频谱阵列中的单通道光谱单元按照预设的分组类型进行信号合并，也就是说，结合本申请提供的单个像素单元的新型电路结构，得到基于单通道光谱单元的多种分组类型，可实现多种分组类型下对环境光类型或强度的探测，还可以实现超高动态范围的环境光探测，解决现有的无法进行高分辨率成像问题，和太阳光等超高动态范围的环境光探测问题。
在一些实施例中，在图6示出的像素单元的电路结构的基础上，本申请中，如图7所示，每个像素单元中的电流探测电路还包括电流镜M，每个像素单元中的电压探测电路还包括复位管RST、源跟随管SF、电容C和行选管SEL1。
对于每个像素单元，PD的输出端a与第一开关器件TX1的第一端b和第二开关器件TX2的第一端c耦合，第一开关器件TX1的第二端d与复位管RST的第一端e和源跟随管SF的第一端f耦合，源跟随管SF的第二端g与行选管SEL1的第一端h耦合，行选管SEL1的第二端i与行列控制电路(SWAP SEL)的第一输入端j耦合，行列控制电路(SWAP SEL)的第一输出端k与采集电压的ADC(ADC1)耦合，行列控制电路(SWAP SEL)的第二输出端l与采集电流的ADC(ADC2)耦合；第二开关器件TX2的第二端m与电流镜M的第一端n耦合，电流镜M的第一输出端o与行列控制电路(SWAP SEL)的第二输入端p耦合。电容C耦合在PD的输入端和第一开关器件TX1的第二端d间。
需要说明的是，图7中示出的两个行列控制电路SWAP SEL实际上为同一个，即同一个SWAP SEL与至少一个ADC1耦合和与至少一个ADC2耦合。图7中的bitLine是指与源跟随管SF的第二端g相连的列向信号线、iLine是指与电流镜M的o端相连的列向信号线。
也就是说，根据图7的电路结构，对于每个像素单元，包括PD、第一开关器件TX1和第二开关器件TX2以外，还包括电流镜M、复位管RST、源跟随管SF、电容C和行选管SEL1。行列控制电路(SWAP SEL)的数量也可以为至少一个，每个像素单元和至少一个行列控制电路(SWAP SEL)中的一个耦合，至少一个行列控制电路 (SWAP SEL)中的每个行列控制电路(SWAP SEL)与多个ADC1中的一个耦合，且与多个ADC2中的一个耦合。
对于单个像素单元来说，逻辑控制电路可控制像素单元中的PD经过光电转换后的电流信号流经第一开关器件TX1或第二开关器件TX2。当电流信号流经第一开关器件TX1时，电流信号可通过TX1传输至源跟随管SF，源跟随管SF可将接收到的电流信号无损的输出给行选管SEL1。行选管SEL1用于在频谱阵列中，根据逻辑控制电路的指示确定某一行的像素单元根据被选中时，该行的像素单元的电流信号可输出给行列控制电路(SWAP SEL)。行列控制电路(SWAP SEL)用于在频谱阵列中，根据逻辑控制电路的指示进行单通道频谱单元按照预设的组合类型进行信号合并。
当电流信号流经第二开关器件TX2时，电流信号可通过电流镜M镜像传输至行列控制电路(SWAP SEL)，以便行列控制电路(SWAP SEL)根据逻辑控制电路的指示进行单通道频谱单元按照预设的组合类型进行信号合并。
基于本申请提供的多光谱探测装置，下面对多光谱探测装置应用于超高动态场景的环境光强探测场景、超高动态场景的环境光类型探测场景和高分辨高动态多光谱成像的场景分别进行介绍。当然，本申请提供的多光谱探测装置不局限应用于这三种场景，还可以应用于其他场景中，本申请不做限定。
本申请实施例提供一种超高动态场景的环境光强探测方法，如图8所示，该方法包括：
801、多光谱探测装置确定执行环境光强探测时，将多个频谱阵列中每个频谱阵列中的单通道光谱单元合并为一个全通通道，得到多个全通通道的合并信号并读出。
在一些实施例中，多光谱探测装置中的逻辑控制电路接收到环境光强探测指示时，由于采集电流的数模转换器ADC具备更高动态范围的检测能力，逻辑控制电路可控制多个频谱阵列中，每个像素单元的第一开关器件TX1关断，第二开关器件TX2导通，并指示行列控制电路SWAP SEL以单个频谱阵列为单位，控制行列控制电路将每个频谱阵列中的所有单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频率阵列区域的环境光强度。
示例性的，假设将每个频谱阵列中的单通道光谱单元合并为全通通道后的示意图如图9所示，可理解，将一个频谱阵列中的单通道光谱单元C11、C12、C13、…、Cmn合并为一个全通通道，得到图9中的全通通道(1，1)，将另一个频谱阵列中的单通道光谱单元C11、C12、C13、…、Cmn合并为一个全通通道，得到图9中的全通通道(1，2)，这样在执行多个频谱阵列的单通道光谱单元C11、C12、C13、…、Cmn合并后，可得到的全通通道包括(1，1)、(1，2)、…、(x，y)。
在一些实施例中，合并方式不限于电荷合并、模拟域合并和数字域合并。
其中，以一个频谱阵列来说，电荷合并，可理解为C11、C12、C13、…、Cmn这多个单通道光谱单元的电荷合并，具体为单通道光谱单元C11中的像素单元(1，1)、(1，2)、…、(h，k)的电荷、单通道光谱单元C12中的像素单元(1，1)、(1，2)、…、(h，k)的电荷、单通道光谱单元C13中的像素单元(1，1)、(1，2)、…、(h，k)的电荷、…、单通道光谱单元Cmn中的像素单元(1，1)、(1，2)、…、(h，k)的电荷的合并。其中，两个像素单元的电荷的合并可理解为，将与两个像素单元耦合的第二开关器件TX2同时导通，使得两个像素单元的进行光电转换后的电流信号传输到同一个电流路径上传输至行列控制电路SWAP SEL。同理，将每个频谱阵列中涉及到的像素单元的电流信号进行电流信号的合并，可得到同一个频谱阵列中每个单通道光谱单元进行电荷合并后的全通通道的电流信号。
以一个频谱阵列来说，模拟域合并，可理解为C11、C12、C13、…、Cmn这多个单通道光谱单元的模拟域合并，具体为单通道光谱单元C11中的像素单元(1，1)、(1，2)、…、(h，k)的模拟域信号、单通道光谱单元C12中的像素单元(1，1)、(1，2)、…、(h，k)的模拟域信号、单通道光谱单元C13中的像素单元(1，1)、(1，2)、…、(h，k)的模拟域信号、…、单通道光谱单元Cmn中的像素单元(1，1)、(1，2)、…、(h，k)的模拟域信号的合并。其中，两个像素单元的模拟域合并可理解为，每个像素单元进行光电转换后的电流信号可在各自的电流路径上传输，但是在传输至ADC2时进行了模拟信号合并。同理，将每个频谱阵列中涉及到的像素单元的模拟信号在ADC2处进行模拟信号的合并，可得到同一个频谱阵列中每个单通道光谱单元进行模拟信号合并后的全通通道的模拟信号输入给至少一个ADC2。
以一个频谱阵列来说，数字域合并，可理解为C11、C12、C13、…、Cmn这多个单通道光谱单元的数字域合并，具体为单通道光谱单元C11中的像素单元(1，1)、(1，2)、…、(h，k)的数字域信号、单通道光谱单元C12中的像素单元(1，1)、(1，2)、…、(h，k)的数字域信号、单通道光谱单元C13中的像素单元(1，1)、(1，2)、…、(h，k)的数字域信号、…、单通道光谱单元Cmn中的像素单元(1，1)、(1，2)、…、(h，k)的数字域信号的合并。其中，两个像素单元的数字域合并可理解为，每个像素单元进行光电转换后的电流信号可在各自的电流路径上传输至ADC2，经过ADC2进行模数转换后的数字信号的合并(例如执行加权平均后的结果)。同理，将每个频谱阵列中涉及到的像素单元的电流信号在ADC2中经过模数转换后的数字信号进行累加(合并)，可得到同一个频谱阵列中每个单通道光谱单元进行数字信号累加后的全通通道的数字信号。
802、多光谱探测装置确定每个全通通道读出的合并后的信号小于或等于预设阈值时，对多个全通通道继续进行等间隔划分，得到多个分组合并后信号并读出。
如果多光谱探测装置中的逻辑控制电路在确定了每个全通通道读出的合并后的信号小于预设阈值时，可认为一个全通通道中的C11、C12、…、Cmn的信号合并后的结果判断的环境光强度依然很弱，还可以对图9中的得到的多个全通通道(1，1)、(1，2)、…、(x，y)进一步进行等间隔划分，得到多个分组，划分结果可示意为(x/k1，y/k2)，(k1，k2＝1，2，3……)个分组，或者称为(x/k1，y/k2)个像素区域，以达到降噪或者增加感度，合并方式不限于电荷合并、模拟域合并和数字域合并等。
这样，划分后得到的(x/k1，y/k2)个分组(像素区域)合并后的信号可通过ADC组2中的至少一个ADC2进行量化，得到不同分组或像素区域的环境光强度，或者称为环境光亮度。
步骤802也就相当于，在逻辑控制电路，用于确定进行环境光强度探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通。控制行列控制电路对多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中的单通道光谱单元作为一个分 组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。
在一些实施例中，本申请的逻辑控制电路，在用于执行底层每个像素单元的信号读出时，可根据上一帧图像的环境光强度，动态的适配低增益或高增益，以进一步满足光强探测动态范围的需求。
即逻辑控制电路确定前一帧图像的环境光强度大于或等于预设阈值时，在获取后一帧图像的每个像素单元的信号时，降低每个像素单元的曝光和/或增益；
确定前一帧图像的环境光强度小于预设阈值时，在获取后一帧图像的每个像素单元的信号时，抬高每个像素单元的曝光和/或增益。
也就是说，多个频谱阵列中的多个像素单元的增益不完全相同，当逻辑控制电路确定上一帧图像的环境光强度较强时，可在得到下一帧图像前，即采集下一帧图像所需的像素单元的光信号时，可将像素单元的增益适当调低；反之，上一帧图像的环境光强度较弱时，可将像素单元的增益适当增高。这种底层像素单元的多增益读取方式，可以进一步增强环境光可探测的动态范围能力。
由此，通过本申请提供的环境光强探测方法，对于每个像素单元，该像素单元中与第一开关器件TX1耦合的部分电路(包括电容C)并未工作，不会产生信号钳位(clamp)带来的饱和问题，适用于高动态范围的光感探测。而且，本申请还可以通过不同的电流加和方式(步骤801或步骤802)，可用于不同强度的环境光的探测需要。
相对目前现有的环境光强探测方案，现有的只能实现单点(单个像素单元)环境光强的检测，无法有效地探测到环境的实际动态范围，因此无法有效地辅助视频后期制作软件(After Effect，AE)精准迅速配置或者不同高动态范围(High Dynamic Range，HDR)比率(ratio)精准配置。而本申请可实现多种像素单元的分区(分组)，借助灵活的信号合并方式，适用于不同的环境光亮度场景，从而达成环境光探测精度和分辨率的平衡选择。
本申请提供的环境光强探测方案中这种多区域的高动态环境光强信息可以用于辅助其他camera进行AE的精准迅速判断或者HDR模式的不同HDR ratio的AE精准配置，还可以辅助HDR融合算法，适配更加合适的压缩曲线。
本申请实施例提供一种超高动态场景的环境光类型探测方法，如图10所示，该方法包括：
1001、多光谱探测装置确定执行环境光类型探测时，控制每个像素单元中的第一开关器件关断，第二开关器件导通。
即当逻辑控制单元确定执行环境光类型探测时，可控制每个像素单元中的第一开关器件TX1关断，第二开关器件TX2导通，将每个像素单元中的PD进行光电转换后的电流信号通过TX2和电流镜M流向行列控制电路。
1002、多光谱探测装置对多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中相同的单通道光谱单元作为一个分组进行信号合并，得到每个区域中每种单通道光谱单元的合并信号，每个区域中的一个分组进行信号合并后的信号用于指示一种环境光类型。
在一些实施例中，逻辑控制电路可指示行列控制电路将多个频谱阵列进行区域划 分，这里的划分可理解为是对图9示出的(1，1)、(1，2)、…、(x，y)个频谱阵列按照(x/k1，y/k2)(k1，k2＝1，2，3……)阵列合并读出。与步骤801不同的是，行列控制电路不需要将每个频谱阵列中的多个单通道光谱单元进行合并，而是将每个区域中相同的单通道光谱单元的信号合并。
如图11所示，假设x＝y＝4，k1＝k2＝2，共16个频谱阵列，被划分为4个分区，或者称为4个分组：①、②、③和④。其中，①分区包括(1，1)、(1，2)、(2，1)和(2,2)4个频谱阵列，②分区包括(1，3)、(1，4)、(2，3)和(2，4)4个频谱阵列，③分区包括(3，1)、(3，2)、(4，1)和(4，2)4个频谱阵列，④分区包括(3，3)、(3，4)、(4，3)和(4，4)4个频谱阵列。
如图12所示，以①分区的4个频谱阵列为例，可将这4个频谱阵列中相同的单通道光谱单元作为一个分组进行信号合并。也就是说，将这4个频谱阵列中的4个单通道光谱单元C11中的像素单元进行信号合并，将这4个频谱阵列中的4个单通道光谱单元C21中的像素单元进行信号合并，将这4个频谱阵列中的4个单通道光谱单元C31中的像素单元进行信号合并，…，将这4个频谱阵列中的4个单通道光谱单元Cmn中的像素单元进行信号合并。这样，可得到①分区中每种单通道光谱单元的合并信号。对于其他3个分区，也进行类似的信号合并即可。
其中，每个分区中相同的单通道光谱单元或者说相同的频谱通道的合并方式不限于上述电荷合并、模拟域合并和数字域合并。
在一些实施例中，逻辑控制电路可根据动态范围和环境光类型探测精度的需求，适配划分合并方式，即适当的确定k1和k2的值。例如，在上一帧图像成像时，确定的环境光强度较强，k1和k2的值可适当增大，即每个分区的频谱阵列的数量较少。当上一帧图像成像时，确定的环境光强度较弱时，k1和k2的值可适当减小，即每个分区的频谱阵列的数量较多，以提升环境光类型探测精度。
可理解，合并后的信号经过ADC2量化，可得到不同分区的频谱响应信息。
在一些实施例中，在暗光场景，或者动态范围较低的场景，或者光源复杂的场景，逻辑控制单元也可以控制第一开关器件TX1导通，第二开关器件TX2关断。这样，步骤1002合并后的信号经过ADC1量化。这是由于ADC1对于暗光的量化能力较强。
在一些实施例中，最底层每个像素单元的信号读出方式可根据上一帧图像的环境光亮度探测信息，动态地适配低增益或高增益，进一步满足光强探测动态范围的需求。具体实现与步骤802中的增益适配过程类似。
由此，本申请提供的超高动态场景的环境光类型探测方法，可通过对多个频谱阵列进行分区，借助灵活的信号合并方式，可适用于不同环境光亮度场景，达成环境光类型的探测精度和分辨率的平衡选择，且不会产生信号clamp带来的饱和问题，适用于高动态光感探测。这种多区域高动态多通道光谱信息，可以用于辅助其他camera进行分区域的环境类型检测和分区域的AWB参数适配，辅助更精准的单一或混合色温场景的颜色还原。
本申请实施例提供一种多光谱成像方法，如图13所示，该方法包括：
1301、多光谱探测装置在确定进行多光谱成像时，控制每个像素单元中的第一开关器件导通，第二开关器件关闭。
由于采集电压的ADC，即本申请中的ADC1可用于电压信号的相关双采样量化，因此，在逻辑控制电路确定要进行多光谱成像时，可使得频谱阵列中的每个单通道光谱单元中的像素单元进行光电转换后的电流信号流向采集电压的ADC组1。
1302、控制行列控制电路将多个频谱阵列中每个频谱阵列的每个单通道光谱单元的多个像素单元作为一个分组进行信号合并，得到每个频谱阵列的每个单通道光谱单元的合并信号，每个频谱阵列的每个单通道光谱单元的合并信号用于进行多光谱成像。
在一些实施例中，逻辑控制电路可控制每个频谱阵列的每个单通道光谱单元的每个像素单元进行光电转换后的电流信号单独输入到ADC1被读出，得到数字信号。即相同频谱通道的底层像素单元进行光电转换后的电流信号可单独输出至ADC1并量化，以在高亮场景提升图像分辨率。
在一些实施例中，为了实现更高色域的颜色还原和图像效果，逻辑控制电路可控制行列控制电路将多个频谱阵列中每个频谱阵列的每个单通道光谱单元中的多个像素单元(像素单元阵列)进行分组或分区，对分组或分组进行信号合并。如图14所示，行列控制电路可将每个单通道光谱单元中的多个像素每4个相邻的像素单元为一组进行信号合并，如图14中像素单元(1，1)、(1，2)(2，1)和(2，2)为一组进行信号合并，像素单元(1，k-1)、(1，k)(2，k-1)和(2，k)为一组进行信号合并，合并方式不限于电荷合并、模拟域合并或数字域合并。
在一些实施例中，为了实现更高色域的颜色还原和图像效果，逻辑控制电路可控制行列控制电路将多个频谱阵列中每个频谱阵列的多个单通道光谱单元进行分组，每个分组包括相邻的多个单通道光谱单元。
控制行列控制电路将每个频谱阵列的每个分组进行信号合并，得到每个频谱阵列的每个分组的合并信号，每个频谱阵列的每个分组的合并信号用于进行多光谱成像。
也就是说，行列控制电路可将每个频谱阵列中不同的单通道光谱单元进行信号合并。如图15所示，以一个频谱阵列来说，行列控制电路可将一个频谱阵列中的4个相邻的单通道光谱单元C11、C12、C22和C22为一组进行信号合并，单通道光谱单元C13、C14、C23和C24为一组进行信号合并。虽然这样会牺牲一定的色域空间，但是可以兼顾图像的颜色和动态范围的提升。
在一些实施例中，最底层的每个像素单元的信号读出方式可以根据上一帧图像的亮度信息，为每个像素单元动态地适配低增益或高增益，或者高低增益的融合。
在一些实施例中，相同的单通道光谱单元的底层像素单元还可以配置不同的曝光或增益，并进行高动态合成压缩。
在一些实施例中，多个单通道光谱单元的拜耳图像再生成(remosaic)可在片上或平台侧完成。
由此，本申请提供的多光谱成像方法，可用于高光谱成像，通过高分辨高动态多光谱信息，可实现更高色域的颜色还原和图像效果。相比现有的方案中，无法有效和灵活地兼顾多光谱和高动态效果，本申请可实现高分辨高光谱成像，兼顾解析力和颜色精度，实现低噪声，适用于高质量图像还原以及夜景效果提升。
可以理解的是，为了实现上述功能，多光谱探测装置包含了执行各个功能相应的硬件和/或软件模块。结合本文中所公开的实施例描述的各示例的算法步骤，本申请能 够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以结合实施例对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本实施例可以根据上述方法示例对电子设备进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块可以采用硬件的形式实现。需要说明的是，本实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图16示出了上述实施例中涉及的多光谱探测装置160的一种可能的组成示意图，如图16所示，该多光谱探测装置160可以包括：开关控制单元1601和信号合并单元1602。
其中，信号合并单元1602可以用于支持多光谱探测装置160执行上述步骤801、步骤802、步骤1002和步骤1302等，和/或用于本文所描述的技术的其他过程。
开关控制单元1601可以用于支持多光谱探测装置160执行上述步骤1001和步骤1301等，和/或用于本文所描述的技术的其他过程。
需要说明的是，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
本实施例提供的多光谱探测装置160，用于执行上述多光谱处理方法，因此可以达到与上述实现方法相同的效果。
在采用集成的单元的情况下，多光谱探测装置160可以包括处理模块、存储模块和通信模块。其中，处理模块可以用于对多光谱探测装置160的动作进行控制管理，例如，可以用于支持多光谱探测装置160执行上述开关控制单元1601和信号合并单元1602执行的步骤。存储模块可以用于支持多光谱探测装置160存储程序代码和数据等。通信模块，可以用于支持多光谱探测装置160与其他设备的通信。
其中，处理模块可以是处理器或控制器。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，数字信号处理(digital signal processing，DSP)和微处理器的组合等等。存储模块可以是存储器。通信模块具体可以为射频电路、蓝牙芯片、Wi-Fi芯片等与其他电子设备交互的设备。
本申请的实施例还提供一种计算机存储介质，该计算机存储介质中存储有计算机指令，当该计算机指令在电子设备上运行时，使得电子设备执行上述相关方法步骤实现上述实施例中的天线增益调整方法。
本申请的实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中电子设备执行的天线增益调整方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中电子设备执行的多光谱处理方法。
其中，本实施例提供的多光谱探测装置、计算机存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上实施方式的描述，所属领域的技术人员可以了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上内容，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (20)

  1. 一种多光谱探测装置，其特征在于，所述多光谱探测装置包括逻辑控制电路、多个频谱阵列、行列控制电路、多个采集电流的模数转换器ADC和多个采集电压的ADC；
    每个所述频谱阵列包括多个单通道光谱单元，所述多个单通道光谱单元中的每个单通道光谱单元包括多个像素单元；
    对于所述多个像素单元中的每个像素单元，所述每个像素单元与所述行列控制电路、所述多个采集电流的ADC中的一个采集电流的ADC和所述多个采集电压的ADC中的一个采集电压的ADC耦合；
    所述每个像素单元包括光电探测器PD、电压探测电路和电流探测电路，所述电压探测电路包括第一开关器件，所述电流探测电路包括第二开关器件；
    所述逻辑控制电路，用于控制所述每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测；
    所述逻辑控制电路，还用于控制所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强。
  2. 根据权利要求1所述的多光谱探测装置，其特征在于，
    所述逻辑控制电路，用于控制所述每个像素单元中的第一开关器件导通，第二开关器件关断，以便将所述每个像素单元中的PD进行光电转化后的电流信号通过第一开关器件传输至所述采集电压的ADC进行成像；或，所述逻辑控制电路，用于控制所述每个像素单元中的第一开关器件关断，第二开关器件导通，以便将所述每个像素单元中的PD进行光电转化后的电流信号通过第二开关器件传输至所述采集电流的ADC进行环境光探测。
  3. 根据权利要求2所述的多光谱探测装置，其特征在于，所述每个像素单元中的电流探测电路还包括电流镜，所述每个像素单元中的电压探测电路还包括复位管、源跟随管和行选管；
    对于所述每个像素单元，PD的输出端与所述第一开关器件的第一端和所述第二开关器件的第一端耦合，所述第一开关器件的第二端与所述复位管的第一端和所述源跟随管的第一端耦合，所述源跟随管的第二端与所述行选管的第一端耦合，所述行选管的第二端与所述行列控制电路的第一输入端耦合，所述行列控制电路的第一输出端与所述采集电压的ADC耦合，所述行列控制电路的第二输出端与所述采集电流的ADC耦合；所述第二开关器件的第二端与所述电流镜的第一端耦合，所述电流镜的第一输出端与所述行列控制电路的第二输入端耦合。
  4. 根据权利要求1-3任一项所述的多光谱探测装置，其特征在于，所述逻辑控制电路，用于：
    在确定进行环境光强度探测时，控制所述每个像素单元中的第一开关器件关断，第二开关器件导通；
    以单个频谱阵列为单位，控制所述行列控制电路将所述每个频谱阵列中的所有单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。
  5. 根据权利要求1-3任一项所述的多光谱探测装置，其特征在于，所述逻辑控制电路，用于：
    在确定进行环境光强度探测时，控制所述每个像素单元中的第一开关器件关断，第二开关器件导通；
    控制所述行列控制电路对所述多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中的单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。
  6. 根据权利要求1-3任一项所述的多光谱探测装置，其特征在于，所述逻辑控制电路，用于：
    在确定进行环境光类型探测时，控制所述每个像素单元中的第一开关器件关断，第二开关器件导通；
    控制所述行列控制电路对所述多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中相同的单通道光谱单元作为一个分组进行信号合并，得到每个区域中每种单通道光谱单元的合并信号，每个区域中的一个分组进行信号合并后的信号用于指示一种环境光类型。
  7. 根据权利要求1-3任一项所述的多光谱探测装置，其特征在于，所述逻辑控制电路，用于：
    在确定进行多光谱成像时，控制所述每个像素单元中的第一开关器件导通，第二开关器件关闭；
    控制所述行列控制电路将所述多个频谱阵列中每个频谱阵列的每个单通道光谱单元的多个像素单元作为一个分组进行信号合并，得到所述每个频谱阵列的每个单通道光谱单元的合并信号，所述每个频谱阵列的每个单通道光谱单元的合并信号用于进行多光谱成像。
  8. 根据权利要求7所述的多光谱探测装置，其特征在于，所述逻辑控制电路，用于：
    在确定进行多光谱成像时，控制所述每个像素单元中的第一开关器件导通，第二开关器件关闭；
    控制所述行列控制电路将所述多个频谱阵列中每个频谱阵列的多个单通道光谱单元进行分组，每个分组包括相邻的多个单通道光谱单元；
    控制所述行列控制电路将所述每个频谱阵列的每个分组进行信号合并，得到所述每个频谱阵列的每个分组的合并信号，所述每个频谱阵列的每个分组的合并信号用于进行多光谱成像。
  9. 根据权利要求4-8任一项所述的多光谱探测装置，其特征在于，所述信号合并包括：
    电荷合并、或模拟域合并或数字域合并。
  10. 根据权利要求4-8任一项所述的多光谱探测装置，其特征在于，所述逻辑控制电路还用于：
    确定前一帧图像的环境光强度大于或等于预设阈值时，在获取后一帧图像的每个像素单元的信号时，降低所述每个像素单元的曝光和/或增益；
    确定前一帧图像的环境光强度小于所述预设阈值时，在获取后一帧图像的每个像素单元的信号时，抬高所述每个像素单元的曝光和/或增益。
  11. 一种多光谱处理方法，其特征在于，应用于多光谱探测装置，所述多光谱探测装置包括逻辑控制电路、多个频谱阵列、行列控制电路、多个采集电流的模数转换器ADC和多个采集电压的ADC；
    每个所述频谱阵列包括多个单通道光谱单元，所述多个单通道光谱单元中的每个单通道光谱单元包括多个像素单元；
    对于所述多个像素单元中的每个像素单元，所述每个像素单元与所述行列控制电路、所述多个采集电流的ADC中的一个采集电流的ADC和所述多个采集电压的ADC中的一个采集电压的ADC耦合；
    所述每个像素单元包括光电探测器PD、电压探测电路和电流探测电路，所述电压探测电路包括第一开关器件，所述电流探测电路包括第二开关器件；
    所述方法包括：
    所述多光谱探测装置控制所述逻辑控制电路用于所述每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测；
    所述多光谱探测装置控制所述逻辑控制电路对所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强。
  12. 根据权利要求11所述的方法，其特征在于，所述多光谱探测装置控制所述逻辑控制电路用于所述每个像素单元中的第一开关器件和第二开关器件在不同的工作模式下进行成像或环境光探测包括：
    所述多光谱探测装置控制所述逻辑控制电路用于所述每个像素单元中的第一开关器件导通，第二开关器件关断，以便将所述每个像素单元中的PD进行光电转化后的电流信号通过第一开关器件传输至所述采集电压的ADC进行成像；
    或，所述多光谱探测装置控制所述逻辑控制电路用于所述每个像素单元中的第一开关器件关断，第二开关器件导通，以便将所述每个像素单元中的PD进行光电转化后的电流信号通过第二开关器件传输至所述采集电流的ADC进行环境光探测。
  13. 根据权利要求11所述的方法，其特征在于，所述每个像素单元中的电流探测电路还包括电流镜，所述每个像素单元中的电压探测电路还包括复位管、源跟随管和行选管；
    对于所述每个像素单元，PD的输出端与所述第一开关器件的第一端和所述第二开关器件的第一端耦合，所述第一开关器件的第二端与所述复位管的第一端和所述源跟随管的第一端耦合，所述源跟随管的第二端与所述行选管的第一端耦合，所述行选管的第二端与所述行列控制电路的第一输入端耦合，所述行列控制电路的第一输出端与所述采集电压的ADC耦合，所述行列控制电路的第二输出端与所述采集电流的ADC耦合；所述第二开关器件的第二端与所述电流镜的第一端耦合，所述电流镜的第一输出端与所述行列控制电路的第二输入端耦合。
  14. 根据权利要求11-13任一项所述的方法，其特征在于，所述多对所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：
    在确定进行环境光强度探测时，控制所述每个像素单元中的第一开关器件关断，第二开关器件导通；
    以单个频谱阵列为单位，控制所述行列控制电路将所述每个频谱阵列中的所有单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。
  15. 根据权利要求11-13任一项所述的方法，其特征在于，所述多对所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：
    在确定进行环境光强度探测时，控制所述每个像素单元中的第一开关器件关断，第二开关器件导通；
    控制所述行列控制电路对所述多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中的单通道光谱单元作为一个分组进行信号合并，一个分组进行信号合并后的信号用于指示一个频谱阵列区域的环境光强度。
  16. 根据权利要求11-13任一项所述的方法，其特征在于，所述多对所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：
    在确定进行环境光类型探测时，控制所述每个像素单元中的第一开关器件关断，第二开关器件导通；
    控制所述行列控制电路对所述多个频谱阵列进行区域划分，将每个区域的所有频谱阵列中相同的单通道光谱单元作为一个分组进行信号合并，得到每个区域中每种单通道光谱单元的合并信号，每个区域中的一个分组进行信号合并后的信号用于指示一种环境光类型。
  17. 根据权利要求11-13任一项所述的方法，其特征在于，所述多对所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：
    在确定进行多光谱成像时，控制所述每个像素单元中的第一开关器件导通，第二开关器件关闭；
    控制所述行列控制电路将所述多个频谱阵列中每个频谱阵列的每个单通道光谱单元的多个像素单元作为一个分组进行信号合并，得到所述每个频谱阵列的每个单通道光谱单元的合并信号，所述每个频谱阵列的每个单通道光谱单元的合并信号用于进行多光谱成像。
  18. 根据权利要求17所述的方法，其特征在于，所述多对所述行列控制电路对所述多个频谱阵列中的多个像素单元采用不同的信号合并方式，进行成像增强或环境光探测增强包括：
    在确定进行多光谱成像时，控制所述每个像素单元中的第一开关器件导通，第二开关器件关闭；
    控制所述行列控制电路将所述多个频谱阵列中每个频谱阵列的多个单通道光谱单元进行分组，每个分组包括相邻的多个单通道光谱单元；
    控制所述行列控制电路将所述每个频谱阵列的每个分组进行信号合并，得到所述 每个频谱阵列的每个分组的合并信号，所述每个频谱阵列的每个分组的合并信号用于进行多光谱成像。
  19. 根据权利要求14-18任一项所述的方法，其特征在于，所述信号合并包括：
    电荷合并、或模拟域合并或数字域合并；
    所述方法还包括：确定前一帧图像的环境光强度大于或等于预设阈值时，在获取后一帧图像的每个像素单元的信号时，降低所述每个像素单元的曝光和/或增益；
    确定前一帧图像的环境光强度小于所述预设阈值时，在获取后一帧图像的每个像素单元的信号时，抬高所述每个像素单元的曝光和/或增益。
  20. 一种计算机可读存储介质，其特征在于，包括计算机指令，当计算机指令在电子设备上运行时，使得电子设备执行上述权利要求11-19中的任一项所述的方法。
PCT/CN2023/086214 2022-04-06 2023-04-04 一种多光谱处理方法和多光谱探测装置 WO2023193712A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210357214.9A CN116929554A (zh) 2022-04-06 2022-04-06 一种多光谱处理方法和多光谱探测装置
CN202210357214.9 2022-04-06

Publications (1)

Publication Number Publication Date
WO2023193712A1 true WO2023193712A1 (zh) 2023-10-12

Family

ID=88244036

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/086214 WO2023193712A1 (zh) 2022-04-06 2023-04-04 一种多光谱处理方法和多光谱探测装置

Country Status (2)

Country Link
CN (1) CN116929554A (zh)
WO (1) WO2023193712A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104081528A (zh) * 2012-01-10 2014-10-01 软动力学传感器公司 多光谱传感器
CN108426639A (zh) * 2017-02-14 2018-08-21 芯视达系统公司 多光谱传感系统和方法
US20200120293A1 (en) * 2017-05-22 2020-04-16 Washington University Multispectral imaging sensors and systems
CN111579497A (zh) * 2019-02-19 2020-08-25 艾迪悌科技有限公司 片上光谱仪
CN113048907A (zh) * 2021-02-08 2021-06-29 浙江大学 一种基于宏像素分割的单像素多光谱成像方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104081528A (zh) * 2012-01-10 2014-10-01 软动力学传感器公司 多光谱传感器
CN108426639A (zh) * 2017-02-14 2018-08-21 芯视达系统公司 多光谱传感系统和方法
US20200120293A1 (en) * 2017-05-22 2020-04-16 Washington University Multispectral imaging sensors and systems
CN111579497A (zh) * 2019-02-19 2020-08-25 艾迪悌科技有限公司 片上光谱仪
CN113048907A (zh) * 2021-02-08 2021-06-29 浙江大学 一种基于宏像素分割的单像素多光谱成像方法及装置

Also Published As

Publication number Publication date
CN116929554A (zh) 2023-10-24

Similar Documents

Publication Publication Date Title
EP2179591B1 (en) Multiple component readout of image sensor
US9287316B2 (en) Systems and methods for mitigating image sensor pixel value clipping
US8164651B2 (en) Concentric exposure sequence for image sensor
US7119317B2 (en) Wide dynamic range imager with selective readout
US8368792B2 (en) Imager methods, apparatuses, and systems providing a skip mode with a wide dynamic range operation
US8405748B2 (en) CMOS image sensor with improved photodiode area allocation
US10798311B2 (en) Methods and apparatus for reducing spatial flicker artifacts
US20100328490A1 (en) Imaging control apparatus, imaging apparatus and imaging control method
JP7099446B2 (ja) 固体撮像装置および電子機器
CN101647271A (zh) 固态成像装置及其驱动方法、成像系统
CN111343396A (zh) 具有可控转换增益的图像传感器
JP2013198160A (ja) 透明フィルタピクセルを備えるイメージングシステム
US20200396406A1 (en) Image sensors having boost current control circuitry for column settling speedup
US10863101B2 (en) Image capturing apparatus and method of controlling the same
KR20180094837A (ko) 이미지 센서, 제어 방법 및 전자 장치
JP2003304544A (ja) 撮像装置
EP1971131B1 (en) Analog image signal processing circuit for cmos image sensor
CN114449188A (zh) 暗电流校准方法及相关联像素电路系统
WO2023193712A1 (zh) 一种多光谱处理方法和多光谱探测装置
JP2018182459A (ja) 撮像装置及び撮像装置の制御方法
US8692899B2 (en) Imaging apparatus with an input/output unit and a lens moving mechanism
WO2020049867A1 (ja) 撮像装置、および撮像素子
US20180048871A1 (en) Methods and devices for increasing the spectral response of an image sensor
US8884210B2 (en) Spectrum information measurement method, color sensor and virtual slide device
JPWO2019216072A1 (ja) 画像処理装置、画像処理方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23784275

Country of ref document: EP

Kind code of ref document: A1