WO2021128533A1 - 双模态仿生视觉传感器 - Google Patents

双模态仿生视觉传感器 Download PDF

Info

Publication number
WO2021128533A1
WO2021128533A1 PCT/CN2020/073523 CN2020073523W WO2021128533A1 WO 2021128533 A1 WO2021128533 A1 WO 2021128533A1 CN 2020073523 W CN2020073523 W CN 2020073523W WO 2021128533 A1 WO2021128533 A1 WO 2021128533A1
Authority
WO
WIPO (PCT)
Prior art keywords
type
target
signal
photosensitive device
current
Prior art date
Application number
PCT/CN2020/073523
Other languages
English (en)
French (fr)
Inventor
施路平
杨哲宇
赵蓉
裴京
徐海峥
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Priority to JP2022536912A priority Critical patent/JP7335020B2/ja
Priority to US17/788,909 priority patent/US11943550B2/en
Publication of WO2021128533A1 publication Critical patent/WO2021128533A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/133—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements including elements passing panchromatic light, e.g. filters passing white light
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/53—Control of the integration time
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/702—SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components

Definitions

  • the present invention relates to the field of integrated circuit technology, and more specifically, to a dual-mode bionic vision sensor.
  • bionic vision sensors are playing an increasingly important role in many application fields such as industrial manufacturing, intelligent transportation, and intelligent robots.
  • the bionic vision sensor mainly simulates the modalities of the retina of the human eye.
  • the retina of the human eye mainly includes two visual perception cells, namely cone cells and rod cells, corresponding to two different modalities respectively.
  • the mode of cone cells is mainly sensitive to absolute light intensity information and color information, and has high image restoration accuracy, but the restoration speed is slow; contrary to the mode of cone cells, rod cells mainly respond to light.
  • the change amount of strong information is sensed, which has a faster perception speed and a larger dynamic range of perception, but it cannot perceive absolute light intensity information and color information.
  • bionic vision sensors in the prior art can only simulate one of the modalities of the retina of the human eye, forming a single perception mode, and thus can only perceive a certain type of information.
  • traditional cameras similar to cones, mainly perceive color information.
  • Dynamic Vision Sensor (DVS) similar to rod cells, mainly perceives the amount of change in light intensity information.
  • DVS Dynamic Vision Sensor
  • the application scenarios of single-modal vision sensors are limited. For example, for a bionic vision sensor similar to a cone cell, since its shooting obtains absolute light intensity information rather than the amount of change in light intensity information, although it is widely used in home entertainment electronic equipment, it is often used in the field of industrial control. Faced with the problem of insufficient speed and too small dynamic range, it is difficult to apply.
  • the bionic vision sensor similar to the rod cell although the sensing speed is very fast, it is only sensitive to moving targets, which makes it difficult to capture images, or the captured images are of poor quality, which is difficult to meet the needs of entertainment electronic devices. Moreover, because the bionic vision sensor only contains a single perception mode, the bionic vision sensor will fail when this perception mode fails, which has great limitations for unmanned, unmanned aerial vehicles and other robots that have high requirements for stability. In addition, the current main indicators for evaluating the performance of the bionic vision sensor include image quality, dynamic range and shooting speed.
  • these three indicators are often mutually exclusive: for example, when the shooting speed increases, the dynamic range of the bionic vision sensor will decrease; when the image quality increases, the shooting speed will generally be It is difficult to take both into consideration at the same time.
  • embodiments of the present invention provide a dual-mode bionic vision sensor.
  • the embodiment of the present invention provides a dual-mode bionic vision sensor, which is characterized by comprising: a first-type current mode active pixel sensor circuit and a voltage mode active pixel sensor circuit;
  • the first-type current mode active pixel sensor circuit includes a target first-type photosensitive device; the target first-type photosensitive device is used to obtain a target light signal and convert the target light signal into a first-type current signal The first-type current mode active pixel sensor circuit is used to convert the first-type current signal and a first preset number of non-target first-type photosensitive devices around the target first-type photosensitive device. The difference between the sum of the two types of current signals, and output a designated digital signal that characterizes the light intensity gradient information in the target light signal;
  • the voltage mode active pixel sensor circuit includes a second type of photosensitive device, the second type of photosensitive device is used to obtain the target light signal, extract the light signal of a specified frequency band from the target light signal, and combine the The optical signal of the designated frequency band is converted into a third-type current signal, and the voltage mode active pixel sensor circuit is configured to output a target voltage signal that represents light intensity information in the target optical signal based on the third-type current signal;
  • each non-target first-type photosensitive device is respectively connected in series with a first-type control switch.
  • the first-type current mode active pixel sensor circuit further includes: a first current amplifier, a comparator, an adder, and a digital-to-analog converter;
  • the target first-type photosensitive device is connected to the first current amplifier, and the first current amplifier is connected to an input terminal of the comparator;
  • the input terminals of the adder are respectively connected to the first-type control switch, and the output terminal of the adder is connected to the other input terminal of the comparator;
  • the output terminal of the comparator is connected to the digital-to-analog converter, and the digital-to-analog converter converts the input specified digital signal into a specified analog signal, and outputs the specified analog signal to the first current amplifier or The adder until the output terminal of the comparator outputs an event pulse signal, the first-type current mode active pixel sensor circuit outputs the designated digital signal, and the designated digital signal is used to represent the target light signal The light intensity gradient information in.
  • the first-type current mode active pixel sensor circuit further includes: a three-state gate circuit;
  • the tri-state gate circuit is respectively connected with the output terminal of the comparator and the input terminal of the digital-to-analog converter;
  • the three-state gate circuit is used to output the designated digital signal when the event pulse signal is output at the output terminal of the comparator.
  • the voltage-mode active pixel sensor circuit specifically includes: a second preset number of the second-type photosensitive devices around the target first-type photosensitive device, each of the second-type photosensitive devices respectively It is connected in series with a second-type control switch, and only one second-type control switch is in the conducting state at the same time;
  • the voltage mode active pixel sensor circuit further includes: a current integrator, a shutter, and an analog-to-digital converter.
  • a current integrator a current integrator
  • a shutter a shutter
  • an analog-to-digital converter Each of the second-type photosensitive devices and the second-type control switches connected in series form a device branch, and all the device branches are connected in parallel Connecting and sharing the current integrator, the shutter, and the analog-to-digital converter;
  • the current integrator is used to obtain the voltage analog signal of the target capacitor in the voltage mode active pixel sensor circuit; the shutter is used to control the integration time of the current integrator; the analog-to-digital converter is used to convert the target The voltage analog signal of the capacitor is converted into the target voltage signal.
  • the dual-mode bionic vision sensor further includes: a second-type current mode active pixel sensor circuit;
  • the second-type current mode active pixel sensor circuit includes one non-target first-type photosensitive device and the second preset number of current mirrors;
  • Each current mirror is respectively connected in series with one of the target first-type photosensitive devices around the non-target first-type photosensitive device.
  • the pixel array of the dual-mode bionic vision sensor is formed by arranging the target first-type photosensitive device, the non-target first-type photosensitive device, and the second-type photosensitive device;
  • the second-type photosensitive device and the target first-type photosensitive device are alternately arranged, or the second-type photosensitive device and the non-target first-type photosensitive device are alternately arranged.
  • the second type of photosensitive device specifically includes an optical filter and a photodiode
  • the optical filter is used to obtain the target optical signal, and extract the optical signal of a specified frequency band from the target optical signal; the photodiode is used to convert the optical signal of the specified frequency band into the third type current signal .
  • the first-type current mode active pixel sensor circuit further includes: a second current amplifier;
  • the second current amplifier is connected between the target second-type photosensitive device and the first current amplifier.
  • the target voltage signal and the designated digital signal jointly form an image.
  • the dual-mode bionic vision sensor further includes: two storage units; the two storage units are respectively used to store the target voltage signal and the designated digital signal.
  • the dual-mode bionic vision sensor provided by the embodiment of the present invention, on the one hand, simulates the role of excitatory rod cells through the first-type current mode active pixel sensor circuit, and realizes the evaluation of the light intensity gradient information in the target light signal. Sensing function, thereby increasing the dynamic range of the image of the bionic vision sensor, and increasing the shooting speed. Moreover, a first-type control switch is introduced for each non-target first-type photosensitive device, which can control the obtained light intensity gradient information, realize the adjustment of the dynamic range of the bionic vision sensor image, and then realize the adjustment of the shooting speed , To achieve the effect of reconfiguration.
  • the voltage-mode active pixel sensor simulates the role of cone cells, and can output a target voltage signal that characterizes the light intensity information in the target light signal, and realizes the perception of the light intensity information in the target light signal.
  • the target voltage signal has higher accuracy in characterizing light intensity information, and higher quality images can be obtained, that is, the images have a higher image signal-to-noise ratio.
  • FIG. 1 is a schematic structural diagram of a dual-mode bionic vision sensor provided by an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of a first-type current mode active pixel sensor circuit in a dual-mode bionic vision sensor according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a change form of a designated digital signal input to a digital-to-analog converter in a first-type current mode active pixel sensor circuit in a dual-mode bionic vision sensor according to an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of a first-type current mode active pixel sensor circuit in a dual-mode bionic vision sensor according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of a specific structure of a first-type current mode active pixel sensor circuit in a dual-mode bionic vision sensor according to an embodiment of the present invention
  • FIG. 6 is a schematic structural diagram of a voltage mode active pixel sensor in a dual-mode bionic vision sensor according to an embodiment of the present invention
  • FIG. 7 is a schematic diagram of a specific structure of a voltage mode active pixel sensor in a dual-mode bionic vision sensor according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a specific structure of a second-type current mode active pixel sensor circuit in a dual-mode bionic vision sensor according to an embodiment of the present invention
  • FIG. 9 is a schematic structural diagram of the arrangement of pixel arrays in a dual-mode bionic vision sensor according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of the arrangement of pixel arrays in a dual-mode bionic vision sensor according to an embodiment of the present invention.
  • FIG. 11 is a schematic diagram of the final output image of a dual-mode bionic vision sensor according to an embodiment of the present invention.
  • connection should be interpreted broadly. For example, they may be fixed connections or Removable connection or integral connection; it can be mechanical connection or electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication between two components.
  • connection should be interpreted broadly. For example, they may be fixed connections or Removable connection or integral connection; it can be mechanical connection or electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication between two components.
  • the embodiment of the present invention provides a dual-mode bionic vision sensor, including: a first-type current mode active pixel sensor circuit and a voltage mode active pixel sensor circuit;
  • the first-type current mode active pixel sensor circuit includes a target first-type photosensitive device; the target first-type photosensitive device is used to obtain a target light signal and convert the target light signal into a first-type current signal The first-type current mode active pixel sensor circuit is used to convert the first-type current signal and a first preset number of non-target first-type photosensitive devices around the target first-type photosensitive device. The difference between the sum of the two types of current signals, and output a designated digital signal that characterizes the light intensity gradient information in the target light signal;
  • the voltage mode active pixel sensor circuit includes a second type of photosensitive device, the second type of photosensitive device is used to obtain the target light signal, extract the light signal of a specified frequency band from the target light signal, and combine the The optical signal of the designated frequency band is converted into a third-type current signal, and the voltage mode active pixel sensor circuit is configured to output a target voltage signal that represents light intensity information in the target optical signal based on the third-type current signal;
  • each non-target first-type photosensitive device is respectively connected in series with a first-type control switch.
  • an embodiment of the present invention provides a dual-mode bionic vision sensor, the pixel array of which is composed of photosensitive devices, and the photosensitive devices are respectively controlled by a control circuit.
  • the control circuit includes a current mode active pixel sensor circuit 1 and a voltage mode active pixel sensor circuit 2.
  • the photosensitive devices in the biomimetic vision sensor can be divided into the first type of photosensitive device and the second type of photosensitive device according to the sensing of different types of light.
  • the first type of photosensitive device is used to directly sense the target light signal, and the second type of photosensitive device It is used to sense the color components in the target light signal.
  • the color component in the target light signal is recorded as the light signal of the specified frequency band, that is, the first type photosensitive device is used to obtain the target light signal and convert the target light signal into a current signal, and the second type photosensitive device It is used to obtain a target optical signal, extract the optical signal of a specified frequency band from the target optical signal, and convert the optical signal of the specified frequency band into a current signal.
  • the target light signal refers to the light signal reflected on the surface of the target object.
  • the target light signal can be directly irradiated on the first type of photosensitive device or the second type of photosensitive device, or it can be irradiated on the first type of photosensitive device or through a collimating lens.
  • the second type of photosensitive device can also be irradiated on the first type of photosensitive device or the second type of photosensitive device through the cover.
  • the waveband of the target light signal may be a visible light waveband, that is, the target light signal may be a visible light signal.
  • the target object refers to an object that needs to be observed by the human eye, which may be a real object, an image, or other forms, and the specific form of the target object is not limited in the present invention.
  • the number of photosensitive devices of the first type and the number of photosensitive devices of the second type can be set as required.
  • the first type of photosensitive device and its control circuit can simulate rod cells, and the second type of photosensitive device and its control circuit can simulate cone cells.
  • the first type of photosensitive device may specifically include the target type 1 photosensitive device and the non-target type 1 photosensitive device except the target type 1 photosensitive device.
  • the target type 1 photosensitive device and its control circuit can simulate excitatory rod cells,
  • the non-target type I photosensitive device and its control circuit can simulate suppressed rod cells.
  • the current signal obtained by the conversion of the target first type photosensitive device is recorded as the first type current signal, and the non-target first type
  • the current signal obtained by the conversion of the type photosensitive device is recorded as the second type of current signal, and the current signal obtained by the conversion of the second type of photosensitive device is recorded as the third type of circuit signal.
  • the first type of photosensitive device in the dual-mode bionic vision sensor is controlled by a current mode active pixel sensor circuit, and the number of current mode active pixel sensor circuits can be determined according to the target number of the first type of photosensitive device.
  • the control circuit of the target first-type photosensitive device in order to distinguish the control circuit of the target first-type photosensitive device and the non-target first-type photosensitive device, the control circuit of the target first-type photosensitive device is recorded as the first-type current mode active pixel sensor circuit, and The control circuit of the non-target first type photosensitive device is recorded as the second type current mode active pixel sensor circuit.
  • Each target first type photosensitive device corresponds to a first type current mode active pixel sensor circuit
  • each non-target first type photosensitive device corresponds to a second type current mode active pixel sensor circuit.
  • the second type of photosensitive device is controlled by the voltage mode active pixel sensor circuit.
  • the number of voltage mode active pixel sensor circuits can be less than or equal to the number of the second type of photosensitive device, which is specifically determined according to the number of the second type of photosensitive device and the multiplexing situation This is not specifically limited in the embodiment of the present invention.
  • Current mode active pixel sensor circuit refers to the active pixel sensor (Active Pixel Sensor, APS) circuit whose working mode is current mode, that is, the target first-type photosensitive device in it converts the first-type current signal, and There is no need to directly integrate the first-type current signal, but based on the sum of the second-type current signal converted from the first-type current signal and a preset number of non-target first-type photosensitive devices around the target first-type photosensitive device Difference, output a designated digital signal that characterizes the light intensity gradient information in the target light signal.
  • each non-target first-type photosensitive device is respectively connected in series with a first-type control switch.
  • the first type of control switches can be specifically MOS transistors. All the first type of control switches can be turned on at the same time, or they can be turned off at the same time, and they can also be partially turned on and partially turned off. Specifically, they can be set as required. This is not specifically limited.
  • the on and off of the first type of control switch can be set as required, so the first type of control switch is a configurable first type of control switch. Since the first type of control switch realizes whether the non-target type 1 photosensitive device around the target type 1 photosensitive device is effective, it can be understood that the first type of control switch can be used as a parameter configurable 1bit convolution check to check the first type of photosensitive device.
  • the current signal converted by the device is subjected to convolution operation, the completion speed is very high, and the 1-bit convolution operation in the pixel can be completed to realize high-speed feature extraction.
  • the voltage mode active pixel sensor circuit refers to the APS circuit whose working mode is voltage mode, that is, after the second type photosensitive device converts to obtain the third type current signal, the third type current signal needs to be integrated to obtain the target voltage Signal, through the target voltage signal to characterize the light intensity information in the target light signal, this light intensity information is a kind of absolute light intensity information, which also contains color information.
  • the embodiment of the present invention provides a dual-mode bionic vision sensor.
  • the function of the excitatory rod cell is simulated by the first-type current mode active pixel sensor circuit to realize the light intensity gradient information in the target light signal.
  • the sensing effect of the bionic vision sensor thereby increasing the dynamic range of the image of the bionic vision sensor, and increasing the shooting speed.
  • a first-type control switch is introduced for each non-target first-type photosensitive device, which can control the obtained light intensity gradient information, realize the adjustment of the dynamic range of the bionic vision sensor image, and then realize the adjustment of the shooting speed , To achieve the effect of reconfiguration.
  • the voltage-mode active pixel sensor simulates the role of cone cells, and can output a target voltage signal that characterizes the light intensity information in the target light signal, and realizes the perception of the light intensity information in the target light signal.
  • the target voltage signal has higher accuracy in characterizing light intensity information, and higher quality images can be obtained, that is, the images have a higher image signal-to-noise ratio.
  • the dual-mode bionic vision sensor provided in the embodiment of the present invention, when the illuminance of the target light signal is greater than a first preset value, all the first-type control switches are turned on at the same time, When the intensity of the target optical signal is less than the second preset value, all the first-type control switches are turned off at the same time.
  • all the control switches of the first type are independent of each other, and one is turned on and off does not affect the other.
  • the number of turns on and the number of turns off can be selected according to needs, and all of them can be turned on or off.
  • all the control switches of the first type in order to obtain a better effect, when the illuminance of the target light signal is greater than the first preset value, all the control switches of the first type can be turned on at the same time, and when the intensity of the target light signal is less than the second preset value When the value is set, all the first type control switches are turned off at the same time.
  • the first preset value and the second preset value may be determined according to the type, parameter, and ambient light intensity of the photosensitive device.
  • the first preset value may be 10klux
  • the second preset value may be 50lux. That is, when the illuminance of the target light signal is greater than the first preset value, it is indicated as strong illumination.
  • all the first-type control switches are switched at the same time
  • the designated digital signal output by the type 1 current mode active pixel sensor circuit is a differential mode signal, which enables the bionic vision sensor to obtain edge information of the image.
  • the intensity of the target light signal is less than the second preset value, it is indicated as weak illumination.
  • the designated digital signal output by the first-type current mode active pixel sensor circuit is a common mode signal, which can make the bionic vision sensor obtain The original information of the image.
  • the first-type current mode active pixel sensor circuit provided in the embodiment of the present invention better simulates the Gap Junction connection of the human eye, thereby achieving an improvement in the image dynamic range of the bionic vision sensor.
  • the control switches of the first type may be partially turned on and partially turned off.
  • the designated digital signal output by the first type current mode active pixel sensor circuit is a differential mode signal.
  • the first type current mode has The specified digital signal output by the source pixel sensor circuit is a common mode signal.
  • the first-type current mode active pixel sensor circuit further includes: a first current amplifier, a comparator, an adder, and a digital-to-analog converter;
  • the target first-type photosensitive device is connected to the first current amplifier, and the first current amplifier is connected to an input terminal of the comparator;
  • the input terminals of the adder are respectively connected to the first-type control switch, and the output terminal of the adder is connected to the other input terminal of the comparator;
  • the output terminal of the comparator is connected to the digital-to-analog converter, and the digital-to-analog converter converts the input specified digital signal into a specified analog signal, and outputs the specified analog signal to the first current amplifier or The adder until the output terminal of the comparator outputs an event pulse signal, the first-type current mode active pixel sensor circuit outputs the designated digital signal, and the designated digital signal is used to represent the target light signal The light intensity gradient information in.
  • the first-type current mode active pixel sensor circuit for controlling a target first-type photosensitive device provided in an embodiment of the present invention.
  • the first-type current mode active pixel sensor circuit includes a target first-type photosensitive device 11, a first current amplifier 12, a comparator 13, an adder 14, and a digital to analog converter (DAC) 15 ,
  • the target first-type photosensitive device 11 is connected to the first current amplifier 12, and the first current amplifier 12 is used to amplify the first-type current signal I 0 converted by the target first-type photosensitive device 11, and the amplification factor is the first
  • the preset number that is, the magnification is equal to the number of non-target first-type photosensitive devices around the target first-type photosensitive device 11, so as to ensure that the amplified first-type current signal is the same as the first-type first-type photosensitive device around the target first-type photosensitive device 11.
  • the sum of the second-type current signals converted by the preset number of non-target second-type photosensitive devices is on the same order of magnitude. It should be noted that there is no color filter (CF) in the first type of photosensitive device provided in the embodiment of the present invention, so the response band of the first type of photosensitive device is related to itself.
  • CF color filter
  • the first current amplifier 12 is connected to an input terminal of the comparator 13 and inputs the amplified first-type current signal into the comparator 13.
  • the four non-target first-type photosensitive devices around the target first-type photosensitive device 11 are all connected to the input ends of the adder 14 respectively. Because each non-target first-type photosensitive device is connected in series with a first-type control switch. In the embodiment of the present invention, only the first-type control switches M 1 , M 2 , M 3 , and M 4 connected in series with each non-target first-type photosensitive device are shown.
  • the output terminal of the adder 14 is connected to the other input terminal of the comparator 13.
  • the current signals I 1 , I 2 , I 3 , and I 4 obtained by the conversion of the four non-target type 1 photosensitive devices are input to the adder 14 respectively, and the adder 14 calculates I 1 , I 2 , I 3 , and I 4 And input the sum result to the comparator 13.
  • the comparator 13 compares the amplified first-type current signal and the sum result of the adder 14. If the comparison result between the current moment and the current moment is consistent, no output is made.
  • the DAC15 converts the input designated digital signal into a designated analog signal, and outputs the designated analog signal to the first current amplifier 12 or the adder 14, and then outputs it to the first current amplifier 12 or the adder 14.
  • the designated analog signal of a current amplifier 12 is denoted as I DA2
  • the designated analog signal output to the adder 14 is denoted as I DA1 .
  • the comparator 13 is used for comparison.
  • the output terminal of the comparator 13 outputs the event pulse signal, that is, the comparator 13 is in the edge-triggered state.
  • the first type of current The pattern active pixel sensor circuit outputs a designated digital signal, and the designated digital signal is used to characterize the light intensity gradient information in the target light signal.
  • the designated digital signal output by the first-type current mode active pixel sensor circuit is a digital signal represented by 0 and 1.
  • the designated digital signal input to the DAC15 can be a designated digital signal that is manually inputted periodically.
  • the change form of the designated digital signal is shown in Fig. 3.
  • the designated digital signal increases stepwise with time.
  • N*step the designated digital signal value is ⁇ I
  • the comparator 13 outputs an event pulse signal, that is, the comparator 13 is in an edge-triggered state, and the ⁇ I at this time is used as the output of the first-type current mode active pixel sensor circuit.
  • N is the number of steps passed before, and step is the time length of each step.
  • the adder in the embodiment of the present invention can be an actual device or a functional module that realizes the addition function.
  • it can be achieved by combining the lines where the current signals I 1 , I 2 , I 3 , and I 4 are located. Realize as a line.
  • the first current amplifier may also be an actual device or a functional module that realizes the current amplifying function, which is not specifically limited in the embodiment of the present invention.
  • the first-type current mode active pixel sensor circuit further includes: a three-state gate circuit
  • the tri-state gate circuit is respectively connected with the output terminal of the comparator and the input terminal of the digital-to-analog converter;
  • the three-state gate circuit is used to output the designated digital signal when the event pulse signal is output at the output terminal of the comparator.
  • the first-type current mode active pixel sensor circuit further includes: a tri-state gate circuit 41.
  • the tri-state gate circuit 41 is respectively connected to the output terminal of the comparator 13 and the input terminal of the DAC 15; the tri-state gate circuit 41 is used to output the event pulse signal at the output terminal of the comparator 13, that is, the comparator 13 is in the edge trigger state and outputs Specify a digital signal.
  • FIG. 5 it is a schematic diagram of the specific structure of the first type of current mode active pixel sensor circuit provided in the embodiment of the present invention.
  • the circuit structure 51 simulates a rod cell circuit
  • the circuit structure 52 simulates a ganglion cell and a bipolar cell.
  • Vcc is the power supply of the control circuit.
  • the target first-type photosensitive device 53 is connected to Vcc.
  • the current mirror 54 in FIG. 5 is the first current amplifier.
  • Fig. 5 does not show the four non-target first-type photosensitive devices around the target first-type photosensitive device 53, only the first-type control switches M 1 , which are connected in series with each non-target first-type photosensitive device. M 2 , M 3 , M 4 .
  • the lines where I 1 , I 2 , I 3 , and I 4 are located are merged into one line to realize the function of an adder.
  • the combined line is connected to the input terminal of CP56.
  • the CP56 compares the amplified first-type current signal and the sum of I 1 , I 2 , I 3 , and I 4 .
  • the DAC55 converts the input designated digital signal into a designated analog signal, and outputs the designated analog signal to the target type 1 photosensitive device 53 or a non-target A type of photosensitive device.
  • the CP56 is used for comparison.
  • the output terminal of the CP56 outputs the event pulse signal, that is, the CP56 is in the edge-triggered state.
  • the three-state gate circuit 57 outputs the specified digital signal .
  • a capacitor 58 is also connected between CP56 and ground.
  • the capacitor 58 can be an actual capacitor or a virtual parasitic capacitor in the first-type current mode active pixel sensor circuit. This is in the embodiment of the present invention. There is no specific limitation.
  • the first-type current mode active pixel sensor circuit further includes: a storage unit.
  • the storage unit is connected to the output terminal of the tri-state gate circuit and is used to store the designated digital signal output by the first-type current mode active pixel sensor circuit.
  • the storage unit may specifically be a register, a latch, SRAM, DRAM, memristor, etc. Taking a register as an example, the number of bits of the register can be selected according to the accuracy of the DAC. In the embodiment of the present invention, a 4-bit register can be selected here.
  • the voltage-mode active pixel sensor circuit specifically includes: a second preset number of light-sensitive devices around the target first type For the second type of photosensitive devices, each of the second type of photosensitive devices is respectively connected in series with a second type of control switch, and only one type of second control switch is in a conducting state at the same time;
  • the voltage mode active pixel sensor circuit further includes: a current integrator, a shutter, and an analog-to-digital converter.
  • a current integrator a current integrator
  • a shutter a shutter
  • an analog-to-digital converter Each of the second-type photosensitive devices and the second-type control switches connected in series form a device branch, and all the device branches are connected in parallel Connecting and sharing the current integrator, the shutter, and the analog-to-digital converter;
  • the current integrator is used to obtain the voltage analog signal of the target capacitor in the voltage mode active pixel sensor circuit; the shutter is used to control the integration time of the current integrator; the analog-to-digital converter is used to convert the target The voltage analog signal of the capacitor is converted into the target voltage signal.
  • each voltage mode active pixel sensor circuit includes the target first A second preset number of second-type photosensitive devices around the second-type photosensitive device. It can be understood that the target first type photosensitive device and the surrounding second preset number of second type photosensitive devices constitute a group, and the target first type photosensitive device in the group is controlled by a current mode active pixel sensor. All non-target type 1 photosensitive devices in the group are controlled by a voltage mode active pixel sensor.
  • FIG. 6 it is a schematic structural diagram of a voltage mode active pixel sensor provided in an embodiment of the present invention.
  • FIG. 6 includes: four second-type photosensitive devices 61, a current integrator (Current Integrator, CI) 62, a shutter 64, and an analog-to-digital converter (Analog-to-Digital Converter, ADC) 63.
  • Each second-type photosensitive device 61 is connected in series with a second-type control switch 65, and only one second-type control switch 65 is in an on state at the same time.
  • Each second-type photosensitive device 61 and the second-type control switch 65 connected in series form a device branch, and the four device branches are connected in parallel, sharing CI62, shutter 64 and ADC63.
  • the second type of control switch 65 may specifically be a MOS tube.
  • CI62 is used to obtain the voltage analog signal of the target capacitor in the voltage mode active pixel sensor circuit; ADC63 is used to convert the voltage analog signal of the target capacitor into the target voltage signal.
  • the shutter 64 is used to control the integration time of the CI62. For example, if the integration time of the CI62 controlled by the shutter 64 is 33ms, after 33ms, the shutter 64 is closed, and the CI62 obtains the voltage analog signal of the target capacitor, which is read by the ADC63.
  • a storage unit may also be connected after the ADC 63, and the voltage analog signal of the target capacitor read by the ADC 63 is stored in the storage unit.
  • the storage unit 413 may specifically be a register, a latch, an SRAM, a DRAM, a memristor, and so on. Taking a register as an example, the number of bits of the register can be selected according to the accuracy of the ADC63. In the embodiment of the present invention, an 8-bit register can be selected to store the voltage analog signal of the target capacitor. After the ADC63 readout operation is completed, the shutter 64 can also be opened, and the CI62 continues to integrate the current of the target capacitor. Repeat the above steps to complete the acquisition of the video signal.
  • FIG. 7 it is a schematic diagram of a specific structure of a voltage mode active pixel sensor circuit provided in an embodiment of the present invention.
  • a total of 4 second-type photosensitive devices are included, namely 71, 72, 73, 74.
  • the second-type photosensitive device 71 and the second-type control switch 75 are connected in series to form the first device branch, and the second-type photosensitive device 72 is connected in series with the second type of control switch 76 to form a second device branch, the second type of photosensitive device 73 is connected in series with the second type of control switch 78 to form a third device branch, and the second type of photosensitive device 74 and the second type of control switch 77
  • the fourth device branch is formed in series.
  • the first device branch, the second device branch, the third device branch, and the fourth device branch are connected in parallel to the MOS transistors 79 and 710, and the MOS transistor 710 is connected to the MOS transistor 711.
  • the MOS tube 79 is used for biasing
  • the MOS tube 710 is used for switching
  • the MOS tube 711 is used for current integration of the third type current signal converted by the second type photosensitive device on a certain device branch to obtain the target
  • the voltage signal represents the light intensity information in the target light signal.
  • the voltage mode active pixel sensor circuit provided in the embodiment of the present invention realizes the control of the device branch through the second type control switch connected in series with each second type photosensitive device, thereby realizing the control of a voltage mode active pixel sensor circuit
  • the functions of multiple second-type photosensitive devices improve the integration of dual-mode bionic vision sensors.
  • the dual-mode bionic vision sensor provided in the embodiment of the present invention further includes: a second-type current mode active pixel sensor circuit;
  • the second-type current mode active pixel sensor circuit includes one non-target first-type photosensitive device and the second preset number of current mirrors;
  • Each current mirror is respectively connected in series with one of the second-type photosensitive devices around the non-target first-type photosensitive device.
  • the second-type current mode active pixel sensor circuit for controlling non-target first-type photosensitive devices specifically includes: non-target first-type photosensitive devices and a second preset number of current mirrors, Each current mirror is respectively connected in series with a target first-type photosensitive device around the non-target first-type photosensitive device. That is to say, each second-type current mode active pixel sensor circuit in the embodiment of the present invention controls a non-target first-type photosensitive device.
  • the second-type current mode active pixel sensor circuit includes a non-target first-type photosensitive device 81 and four first-type current mirrors 82, 83, 84, 85.
  • Each first-type current mirror is connected in series with a target first-type photosensitive device around the non-target first-type photosensitive device 81, that is, the current signal I 1 obtained by the conversion of the non-target first-type photosensitive device 81 is copied into four I 1 , which are respectively used for the first-type current mode active pixel sensor circuit of each target first-type photosensitive device around the non-target first-type photosensitive device 81 to obtain the light intensity gradient information in the target light signal to achieve the non-target
  • the multiplexing of the first type of photosensitive device improves the pixel fill factor of the dual-mode bionic vision sensor.
  • the first type of photosensitive device is specifically a photodiode (Photo-Diode, PD), or other types that can convert light signals. It is a current signal device, which is not specifically limited in the embodiment of the present invention. It should be noted that the first type of photosensitive device does not include a filter.
  • the second type of photosensitive device specifically includes an optical filter and a photodiode
  • the optical filter is used to obtain the target optical signal, and extract the optical signal of a specified frequency band from the target optical signal; the photodiode is used to convert the optical signal of the specified frequency band into the third type current signal .
  • the second type of photosensitive device is used to sense color components in the target light signal.
  • the second type of photosensitive device may specifically include a PD and a color filter (CF) arranged on the PD.
  • the final image obtained by the bionic vision sensor is a color image.
  • CF is used to obtain the target optical signal, extract the optical signal of the specified frequency band from the target optical signal, and the PD converts the optical signal of the specified frequency band into the third type current signal.
  • the color filter may specifically be a filter or a lens, which is used to transmit light signals of a specified wavelength. When the color filter is a lens, a Byron lens can be used specifically, and other types of lenses can also be used.
  • the color filter can be divided into a red color filter, a blue color filter and a green color filter according to the wavelength of the transmitted light signal, and the transmitted light signals are respectively a red light signal, a blue light signal and a green light signal.
  • the second type of photosensitive device can also be directly composed of photodiodes.
  • the target light signal can be obtained, and the light signal of the specified band can be extracted from the target light signal, and the light of the specified waveband can be obtained.
  • the signal is converted into the role of the third type of current signal.
  • the pixel array of the dual-mode bionic vision sensor is composed of the target first type photosensitive device and the non-target first type
  • the photosensitive device and the second type of photosensitive device are arranged and formed;
  • the second-type photosensitive device and the target first-type photosensitive device are alternately arranged, or the second-type photosensitive device and the non-target first-type photosensitive device are alternately arranged.
  • FIG. 9 a schematic structural diagram of the arrangement of the pixel array can be shown in FIG. 9, which includes a first type of photosensitive device 91 and a second type of photosensitive device 92, each of the first type of photosensitive device and each of the second type of photosensitive device Each constitutes a pixel.
  • the target first-type photosensitive device in the first-type photosensitive device 91 is marked as "+”, and the non-target first-type photosensitive device is marked as "-”.
  • the second type photosensitive device 92 containing a red color filter is marked as "R”
  • the second type photosensitive device 92 containing a blue color filter is marked as "B”
  • the second type photosensitive device containing a green color filter is marked as "R”.
  • Device 92 is labeled "G”.
  • Each target type 1 photosensitive device is surrounded by 4 non-target type 1 photosensitive devices and 4 type 2 photosensitive devices, and each non-target type 1 photosensitive device is surrounded by 4 target type 1 photosensitive devices and 4 The second type of photo
  • the structure diagram of the arrangement of the pixel array can also be shown in Figure 10, which includes the first type of photosensitive device 101 and the second type of photosensitive device 102.
  • the target first type of photosensitive device in the first type of photosensitive device 101 is marked as "+ ", non-target type 1 photosensitive devices are marked as "-”.
  • the second type photosensitive device 102 containing a red color filter is marked as "R”
  • the second type photosensitive device 102 containing a blue color filter is marked as "B”
  • the second type photosensitive device containing a green color filter is marked as "R”.
  • the device 102 is labeled "G”.
  • Each target type 1 photosensitive device has 6 non-target type 1 photosensitive devices and 2 type 2 photosensitive devices, and each non-target type 1 photosensitive device is surrounded by 2 target type 1 photosensitive devices and 4 The second type photosensitive device, or each non-target first type photosensitive device is surrounded by 4 target first type photosensitive devices and 2 second type photosensitive devices.
  • the arrangement of the pixel array may also be in other forms, which is not specifically limited in the embodiment of the present invention.
  • the pixel array shown in FIG. 9 corresponds to the first preset number and the second preset number of 4, and the pixel array shown in FIG. 10 corresponds to the first preset number of 6, and the second preset number is 6.
  • the pixel array shown in FIG. 9 is taken as an example for description.
  • the second type photosensitive device 71 in FIG. 7 may be the second type photosensitive device marked as "G”
  • the second type photosensitive device 72, 73 may be the second type photosensitive device marked as "R”
  • the second type photosensitive device is marked as "R”.
  • the type photosensitive device 74 may be a second type of photosensitive device labeled "B".
  • the first-type current mode active pixel sensor circuit further includes: a second current amplifier;
  • the second current amplifier is connected between the target second-type photosensitive device and the first current amplifier.
  • a second current amplifier may be connected between the first current amplifier and the target first-type photosensitive device to serve as the target
  • the first-type current signal converted by the first-type photosensitive device is preliminarily amplified.
  • the second current amplifier may be an actual device or a functional module that realizes the current amplification function, which is not specifically limited in the embodiment of the present invention.
  • a second current amplifier is also provided between the non-target first-type photosensitive device around the target first-type photosensitive device and the adder, so that the current of the branch where each non-target first-type photosensitive device is located before the adder is The signal is in the same order of magnitude as the current signal of the branch where the target type 1 photosensitive device is located.
  • the target voltage signal and the designated digital signal jointly form an image.
  • the target voltage signal and the designated digital signal are combined to form an image.
  • the output speed of the target voltage signal is about 30ms.
  • the asynchronous event address is then used to output, specifically (X, Y, P, T).
  • "X, Y" is the event address
  • "P” is the 4-value event output (including the first sign bit)
  • "T” is the time when the event is generated.
  • the final output image is shown in Fig. 11, the two frames of pictures are color pictures, which are formed by the successively output target voltage signals, and the edge points between the two frames of pictures are formed by the output designated digital signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Vascular Medicine (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Photometry And Measurement Of Optical Pulse Characteristics (AREA)

Abstract

本发明实施例提供了一种双模态仿生视觉传感器，通过第一类电流模式有源像素传感器电路模拟兴奋型视杆细胞的作用，实现对目标光信号中的光强梯度信息的感知作用，进而提高仿生视觉传感器图像的动态范围，提高拍摄速度。而且，为每个非目标第一类感光器件引入一个第一类控制开关，可以对得到的光强梯度信息进行控制，实现对仿生视觉传感器图像的动态范围的调整，进而实现对拍摄速度的调整，实现了可重配的效果。通过电压模式有源像素传感器模拟视锥细胞的作用，可以输出表征目标光信号中的光强信息的目标电压信号，实现对目标光信号中的光强信息的感知作用，得到的目标电压信号表征光强信息的精度更高，可以保证得到的图像质量。

Description

双模态仿生视觉传感器 技术领域
本发明涉及集成电路技术领域，更具体地，涉及双模态仿生视觉传感器。
背景技术
随着对图像传感器与图像处理识别算法研究地不断深入，仿生视觉传感器在工业制造、智能交通、智能机器人等多个应用领域扮演着越来越重要的作用。
仿生视觉传感器主要是对人眼视网膜的模态进行仿真，人眼视网膜主要包括两种视觉感知细胞，即视锥细胞和视杆细胞，分别对应两种不同的模态。其中，视锥细胞的模态主要是对绝对光强信息与颜色信息敏感，具有很高的图像还原精度，但是还原速度较慢；与视锥细胞的模态相反，视杆细胞主要是对光强信息的变化量进行感知，具有感知速度较快且感知的动态范围较大，但是其无法感知绝对光强信息与颜色信息。
但是，现有技术中存在的仿生视觉传感器均只能对人眼视网膜的其中一种模态进行仿真，形成单一的感知模式，进而只能对某一类信息进行感知。如传统相机，类似于视锥细胞，主要对颜色信息进行感知。如动态视觉传感器(Dynamic Vision Sensor，DVS)，类似于视杆细胞，主要对光强信息的变化量进行感知。而单一模态的视觉传感器应用场景有限。例如，对于类似于视锥细胞的仿生视觉传感器，由于其拍摄得到的是绝对光强信息而非光强信息的变化量，虽然在家用娱乐电子设备中应用非常广泛，但在工业控制领域，往往面临速度不够动态范围太小等问题，因而很难应用。对于类似于视杆细胞的仿生视觉传感器，虽然感知速度很快，但是由于只对运动目标敏感，导致难以拍摄到图像，或者拍摄到的图像质量较差，难以满足娱乐电子设备的需求。而且由于仿生视觉传感器只包含单一的感知模式，在这种感知模式失效时仿生视觉传感器则失效，这对于对稳定性有高要求的无人驾驶、无人机等机器人有很大的局限。另外，目前评价仿生视觉传感器性能的主要指标有图像质量，动态范围与拍摄速度。由上述内容可知，在传统的仿生视觉传感器的框架下，这三个指标往往互斥：如当拍摄速度提高时，仿生视觉传感器的动态范围就会降低；当图像质量提高时拍摄速度一般就会降低，很难同时兼顾。
因此，现急需提供一种双模态仿生视觉传感器。
发明内容
为克服上述问题或者至少部分地解决上述问题，本发明实施例提供了一种双模态仿生视觉传感器。
本发明实施例提供了一种双模态仿生视觉传感器，其特征在于，包括：第一类电流模式有源像素传感器电路和电压模式有源像素传感器电路；
所述第一类电流模式有源像素传感器电路包括一个目标第一类感光器件；所述目标第一类感光器件用于获取目标光信号，并将所述目标光信号转换为第一类电流信号，所述第一类电流模式有源像素传感器电路用于基于所述第一类电流信号与所述目标第一类感光器件周围第一预设数量个非目标第一类感光器件转换得到的第二类电流信号之和的差值，输出表征所述目标光信号中的光强梯度信息的指定数字信号；
所述电压模式有源像素传感器电路包括第二类感光器件，所述第二类感光器件用于获取所述目标光信号，从所述目标光信号中提取指定频段的光信号，并将所述指定频段的光信号转换为第三类电流信号，所述电压模式有源像素传感器电路用于基于所述第三类电流信号，输出表征所述目标光信号中的光强信息的目标电压信号；
其中，每个非目标第一类感光器件分别与一个第一类控制开关串联。
优选地，所述第一类电流模式有源像素传感器电路还包括：第一电流放大器、比较器、加法器和数模转换器；
所述目标第一类感光器件与所述第一电流放大器相连，所述第一电流放大器与所述比较器的一个输入端相连；
所述加法器的输入端分别与所述第一类控制开关连接，所述加法器的输出端与所述比较器的另一个输入端相连；
所述比较器的输出端与所述数模转换器相连，所述数模转换器将输入的指定数字信号转换为指定模拟信号，并将所述指定模拟信号输出至所述第一电流放大器或者所述加法器，直至所述比较器的输出端输出事件脉冲信号，所述第一类电流模式有源像素传感器电路输出所述指定数字信号，所述指定数字信号用于表征所述目标光信号中的光强梯度信息。
优选地，所述第一类电流模式有源像素传感器电路还包括：三态门电路；
所述三态门电路分别与所述比较器的输出端以及所述数模转换器的输入端连接；
所述三态门电路用于在所述比较器的输出端输出所述事件脉冲信号时，输出所述指定数字信号。
优选地，所述电压模式有源像素传感器电路具体包括：所述目标第一类感光 器件周围的第二预设数量个所述第二类感光器件，每个所述第二类感光器件均分别与一个第二类控制开关串联，同一时刻仅存在一个第二类控制开关处于导通状态；
所述电压模式有源像素传感器电路还包括：电流积分器、快门以及模数转换器，每个所述第二类感光器件以及串联的第二类控制开关构成器件支路，所有器件支路并联连接且共用所述电流积分器、所述快门以及所述模数转换器；
所述电流积分器用于获取所述电压模式有源像素传感器电路中目标电容的电压模拟信号；所述快门用于控制所述电流积分器的积分时间；所述模数转换器用于将所述目标电容的电压模拟信号转换为所述目标电压信号。
优选地，所述的双模态仿生视觉传感器，还包括：第二类电流模式有源像素传感器电路；
所述第二类电流模式有源像素传感器电路包括一个所述非目标第一类感光器件和所述第二预设数量个电流镜；
每个电流镜分别与所述非目标第一类感光器件周围的一个所述目标第一类感光器件串联。
优选地，所述双模态仿生视觉传感器的像素阵列由所述目标第一类感光器件、所述非目标第一类感光器件以及所述第二类感光器件排列形成；
所述像素阵列的每一行中，所述第二类感光器件与所述目标第一类感光器件相间排列，或者所述第二类感光器件与所述非目标第一类感光器件相间排列。
优选地，所述第二类感光器件具体包括滤光器和光电二极管；
所述滤光器用于获取所述目标光信号，从所述目标光信号中提取指定频段的光信号；所述光电二极管用于将所述指定频段的光信号转换为所述第三类电流信号。
优选地，所述第一类电流模式有源像素传感器电路还包括：第二电流放大器；
所述第二电流放大器连接在所述目标第二类感光器件与所述第一电流放大器之间。
优选地，所述目标电压信号和所述指定数字信号共同形成图像。
优选地，所述的双模态仿生视觉传感器，还包括：两个存储单元；所述两个存储单元分别用于存储所述目标电压信号以及所述指定数字信号。
本发明实施例提供的一种双模态仿生视觉传感器，一方面，通过第一类电流模式有源像素传感器电路模拟兴奋型视杆细胞的作用，实现对目标光信号中的光强梯度信息的感知作用，进而提高仿生视觉传感器图像的动态范围，提高拍摄速度。而且，为每个非目标第一类感光器件引入一个第一类控制开关，可以对得到的光强梯度信息进行控制，实现对仿生视觉传感器图像的动态范围的调整，进而实现对拍摄速度的调整，实现了可重配的效果。另一方面，通过电压模式有源像 素传感器模拟视锥细胞的作用，可以输出表征目标光信号中的光强信息的目标电压信号，实现对目标光信号中的光强信息的感知作用，得到的目标电压信号表征光强信息的精度更高，可以得到更高质量的图像，即图像具有更高的图像信噪比。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种双模态仿生视觉传感器的结构示意图；
图2为本发明实施例提供的一种双模态仿生视觉传感器中第一类电流模式有源像素传感器电路的结构示意图；
图3为本发明实施例提供的一种双模态仿生视觉传感器中第一类电流模式有源像素传感器电路中输入至数模转换器的指定数字信号的变化形式示意图；
图4为本发明实施例提供的一种双模态仿生视觉传感器中第一类电流模式有源像素传感器电路的结构示意图；
图5为本发明实施例提供的一种双模态仿生视觉传感器中第一类电流模式有源像素传感器电路的具体结构示意图；
图6为本发明实施例提供的一种双模态仿生视觉传感器中电压模式有源像素传感器的结构示意图；
图7为本发明实施例提供的一种双模态仿生视觉传感器中电压模式有源像素传感器的具体结构示意图；
图8为本发明实施例提供的一种双模态仿生视觉传感器中第二类电流模式有源像素传感器电路的具体结构示意图；
图9为本发明实施例提供的一种双模态仿生视觉传感器中像素阵列的排布方式结构示意图；
图10为本发明实施例提供的一种双模态仿生视觉传感器中像素阵列的排布方式结构示意图；
图11为本发明实施例提供的一种双模态仿生视觉传感器最终输出的图像示意图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在本发明实施例的描述中，需要说明的是，术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明实施例和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明实施例的限制。此外，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。
在本发明实施例的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本发明实施例中的具体含义。
本发明实施例提供了一种双模态仿生视觉传感器，包括：第一类电流模式有源像素传感器电路和电压模式有源像素传感器电路；
所述第一类电流模式有源像素传感器电路包括一个目标第一类感光器件；所述目标第一类感光器件用于获取目标光信号，并将所述目标光信号转换为第一类电流信号，所述第一类电流模式有源像素传感器电路用于基于所述第一类电流信号与所述目标第一类感光器件周围第一预设数量个非目标第一类感光器件转换得到的第二类电流信号之和的差值，输出表征所述目标光信号中的光强梯度信息的指定数字信号；
所述电压模式有源像素传感器电路包括第二类感光器件，所述第二类感光器件用于获取所述目标光信号，从所述目标光信号中提取指定频段的光信号，并将所述指定频段的光信号转换为第三类电流信号，所述电压模式有源像素传感器电路用于基于所述第三类电流信号，输出表征所述目标光信号中的光强信息的目标电压信号；
其中，每个非目标第一类感光器件分别与一个第一类控制开关串联。
具体地，本发明实施例中提供了一种双模态仿生视觉传感器，其像素阵列由感光器件构成，感光器件分别由控制电路进行控制。如图1所示，控制电路包括电流模式有源像素传感器电路1和电压模式有源像素传感器电路2。仿生视觉传感器中的感光器件可以根据对不同类别的光进行感应分为第一类感光器件和第二类感光器件，第一类感光器件用于直接对目标光信号进行感应，第二类感光器件用 于对目标光信号中的彩色分量进行感应。本发明实施例中，将目标光信号中的彩色分量记为指定频段的光信号，即第一类感光器件用于获取目标光信号，并将目标光信号转换为电流信号，第二类感光器件用于获取目标光信号，从所述目标光信号中提取指定频段的光信号，并将指定频段的光信号转换为电流信号。
其中，目标光信号是指目标物体表面反射的光信号，目标光信号可以直接照射在第一类感光器件或第二类感光器件上，也可以是通过准直透镜照射在第一类感光器件或第二类感光器件上，还可以是透过覆盖物照射在第一类感光器件或第二类感光器件上。目标光信号的波段可以是可见光波段，即目标光信号可以为可见光信号。目标物体是指需要人眼观察的物体，可以是实物，也可以是图像，或者其他形式，本发明中不对目标物体的具体形态进行限定。
第一类感光器件和第二类感光器件的数量可以根据需要进行设定。第一类感光器件及其控制电路可以模拟视杆细胞，第二类感光器件及其控制电路可以模拟视锥细胞。第一类感光器件具体可以包括目标第一类感光器件和除目标第一类感光器件外的非目标第一类感光器件，目标第一类感光器件及其控制电路可以模拟兴奋型视杆细胞，非目标第一类感光器件及其控制电路可以模拟抑制型视杆细胞。本发明实施例中，为区分第一类感光器件、第二类感光器件转换得到的电流信号，将目标第一类感光器件转换得到的电流信号记为第一类电流信号，将非目标第一类感光器件转换得到的电流信号记为第二类电流信号，将第二类感光器件转换得到的电流信号记为第三类电路信号。
双模态仿生视觉传感器中第一类感光器件由电流模式有源像素传感器电路进行控制，电流模式有源像素传感器电路的数量可以根据目标第一类感光器件的数量确定。本发明实施例中，为区分目标第一类感光器件和非目标第一类感光器件的控制电路，将目标第一类感光器件的控制电路记为第一类电流模式有源像素传感器电路，将非目标第一类感光器件的控制电路记为第二类电流模式有源像素传感器电路。每个目标第一类感光器件对应一个第一类电流模式有源像素传感器电路，每个非目标第一类感光器件对应一个第二类电流模式有源像素传感器电路。第二类感光器件由电压模式有源像素传感器电路进行控制，电压模式有源像素传感器电路的数量可以小于等于第二类感光器件的数量，具体根据第二类感光器件的数量以及复用情况确定，本发明实施例中对此不作具体限定。
电流模式有源像素传感器电路是指工作模式为电流模式的有源像素传感器(Active Pixel Sensor，APS)电路，也即在其中的目标第一类感光器件在转换得到第一类电流信号后，并不需要直接对第一类电流信号进行积分，而是根据第一类电流信号与目标第一类感光器件周围预设数量个非目标第一类感光器件转换得到的第二类电流信号之和的差值，输出表征目标光信号中的光强梯度信息的指定数字信号。其中，每个非目标第一类感光器件分别与一个第一类控制开关串联。第 一类控制开关具体可以是MOS管，所有第一类控制开关可以同时导通，也可以同时断开，还可以部分导通部分断开，具体可以根据需要进行设置，本发明实施例中对此不作具体限定。
第一类控制开关的导通与关闭可以根据需要进行设置，因此第一类控制开关为可配置的第一类控制开关。由于第一类控制开关实现的是控制目标第一类感光器件周围的非目标第一类感光器件是否有效，可以理解为第一类控制开关可以作为参数可配的1bit卷积核对第一类感光器件转换得到的电流信号进行卷积运算，完成速度很高，可以完成像素内的1bit卷积运算，实现高速的特征提取。
电压模式有源像素传感器电路是指工作模式为电压模式的APS电路，也即在其中的第二类感光器件转换得到第三类电流信号后，需要对第三类电流信号进行积分，得到目标电压信号，通过目标电压信号表征目标光信号中的光强信息，这种光强信息是一种绝对光强信息，其中还包含有颜色信息。
本发明实施例中提供了一种双模态仿生视觉传感器，一方面，通过第一类电流模式有源像素传感器电路模拟兴奋型视杆细胞的作用，实现对目标光信号中的光强梯度信息的感知作用，进而提高仿生视觉传感器图像的动态范围，提高拍摄速度。而且，为每个非目标第一类感光器件引入一个第一类控制开关，可以对得到的光强梯度信息进行控制，实现对仿生视觉传感器图像的动态范围的调整，进而实现对拍摄速度的调整，实现了可重配的效果。另一方面，通过电压模式有源像素传感器模拟视锥细胞的作用，可以输出表征目标光信号中的光强信息的目标电压信号，实现对目标光信号中的光强信息的感知作用，得到的目标电压信号表征光强信息的精度更高，可以得到更高质量的图像，即图像具有更高的图像信噪比。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，当所述目标光信号的光照度大于第一预设值时，所有所述第一类控制开关同时导通，当所述目标光信号的强度小于第二预设值时，所有所述第一类控制开关同时断开。
具体地，所有的第一类控制开关均相互独立，一个导通与断开并不会影响另一个，可以根据需要选择导通的数量和断开的数量，也可以全部导通或全部断开。本发明实施例中，为获得更好的效果，可以在目标光信号的光照度大于第一预设值时，使所有第一类控制开关同时导通，当目标光信号的强度小于第二预设值时，使所有第一类控制开关同时断开。其中第一预设值、第二预设值可以根据感光器件的类型、参数以及环境光照度进行确定。例如，第一预设值可以为10klux，第二预设值可以为50lux。即，当目标光信号的光照度大于第一预设值时，说明为强光照，此时为防止第一类电流模式有源像素传感器电路内DAC与比较器饱和，将所有第一类控制开关同时导通，此时所有非目标第一类感光器件有效，第一类电流模式有源像素传感器电路输出的指定数字信号为差模信号，可以使仿生视觉传 感器得到图像的边缘信息。当目标光信号的强度小于第二预设值时，说明为弱光照，此时经目标第一类感光器件转换得到的第一类电流信号I 1很小。因此将所有第一类控制开关同时断开，此时所有非目标第一类感光器件无效，第一类电流模式有源像素传感器电路输出的指定数字信号为共模信号，可以使仿生视觉传感器得到图像的原始信息。本发明实施例中提供的第一类电流模式有源像素传感器电路更好的仿真人眼的Gap Junction连接，从而实现对于仿生视觉传感器的图像动态范围的提高。
需要说明的是，当目标光信号的光照度大于第一预设值且小于第二预设值时，说明光照适中，此时所有第一类控制开关中可以部分导通部分断开。当至少一个第一类控制开关导通时，第一类电流模式有源像素传感器电路输出的指定数字信号均为差模信号，当所有第一类控制开关断开时，第一类电流模式有源像素传感器电路输出的指定数字信号为共模信号。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述第一类电流模式有源像素传感器电路还包括：第一电流放大器、比较器、加法器和数模转换器；
所述目标第一类感光器件与所述第一电流放大器相连，所述第一电流放大器与所述比较器的一个输入端相连；
所述加法器的输入端分别与所述第一类控制开关连接，所述加法器的输出端与所述比较器的另一个输入端相连；
所述比较器的输出端与所述数模转换器相连，所述数模转换器将输入的指定数字信号转换为指定模拟信号，并将所述指定模拟信号输出至所述第一电流放大器或者所述加法器，直至所述比较器的输出端输出事件脉冲信号，所述第一类电流模式有源像素传感器电路输出所述指定数字信号，所述指定数字信号用于表征所述目标光信号中的光强梯度信息。
具体地，如图2所示，为本发明实施例中提供的用于控制目标第一类感光器件的第一类电流模式有源像素传感器电路。图2中，第一类电流模式有源像素传感器电路包括目标第一类感光器件11、第一电流放大器12、比较器13、加法器14和数模转换器(Digital to Analog Converter，DAC)15，目标第一类感光器件11与第一电流放大器12相连，第一电流放大器12用于对目标第一类感光器件11转换得到的第一类电流信号I 0进行放大，放大的倍数为第一预设数量，即放大的倍数与目标第一类感光器件11周围的非目标第一类感光器件的数量相等，以保证放大后的第一类电流信号与目标第一类感光器件11周围第一预设数量个非目标第二类感光器件转换得到的第二类电流信号之和在同一量级上。需要说明的是，本发明实施例中提供的第一类感光器件中不存在滤光器(Colorfilter，CF)，因此第一类感光器件的响应波段与自身相关。
第一电流放大器12与比较器13的一个输入端相连，将放大后的第一类电流信号输入至比较器13中。目标第一类感光器件11周围的4个非目标第一类感光器件均分别与加法器14的输入端相连。由于每个非目标第一类感光器件均与一个第一类控制开关串联。本发明实施例中仅示出了与每个非目标第一类感光器件串联的第一类控制开关M 1、M 2、M 3、M 4。
加法器14的输出端与比较器13的另一个输入端相连。4个非目标第一类感光器件转换得到的电流信号I 1、I 2、I 3、I 4分别输入至加法器14，由加法器14对I 1、I 2、I 3、I 4进行求和，并将求和结果输入至比较器13。由比较器13对放大后的第一类电流信号以及加法器14的求和结果进行比较。当前一时刻与当前时刻的比较结果一致，则不做输出，由DAC15将输入的指定数字信号转换为指定模拟信号，并将指定模拟信号输出至第一电流放大器12或者加法器14，输出至第一电流放大器12的指定模拟信号记为I DA2，输出至加法器14的指定模拟信号记为I DA1。输出后再通过比较器13进行比较，当前一时刻与后一时刻的比较结果相反时，由比较器13的输出端输出事件脉冲信号，即比较器13处于边沿触发状态，此时第一类电流模式有源像素传感器电路输出指定数字信号，指定数字信号用于表征目标光信号中的光强梯度信息。其中，第一类电流模式有源像素传感器电路输出的指定数字信号是一种用0和1表示的数字信号。
其中，输入至DAC15的指定数字信号可以是人为输入的周期性增加的指定数字信号，指定数字信号的变化形式具体如图3所示，指定数字信号具体随时间呈阶梯型递增，当某一时刻N*step时，指定数字信号取值为ΔI，比较器13输出事件脉冲信号，即比较器13处于边沿触发状态，则将此时的ΔI作为第一类电流模式有源像素传感器电路的输出。其中，N为此前经过的台阶数，step为每一台阶经过的时长。
需要说明的是，本发明实施例中的加法器可以是实际的器件，也可以是实现加法功能的功能模块，例如可以通过将电流信号I 1、I 2、I 3、I 4所在的线路合并成一条线路实现。而且，第一电流放大器也可以是实际的器件，也可以是实现电流放大功能的功能模块，本发明实施例中对此不作具体限定。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述第一类电流模式有源像素传感器电路还包括：三态门电路；
所述三态门电路分别与所述比较器的输出端以及所述数模转换器的输入端连接；
所述三态门电路用于在所述比较器的输出端输出所述事件脉冲信号时，输出所述指定数字信号。
具体地，如图4所示，本发明实施例中，在第一类电流模式有源像素传感器电路中还包括：三态门电路41。三态门电路41分别与比较器13的输出端以及 DAC15的输入端连接；三态门电路41用于在比较器13的输出端输出事件脉冲信号时，即比较器13处于边沿触发状态，输出指定数字信号。
如图5所示，为本发明实施例中提供的第一类电流模式有源像素传感器电路的具体结构示意图。图5中，电路结构51模拟视杆细胞电路，电路结构52模拟节细胞与双极细胞。Vcc为控制电路的电源，目标第一类感光器件53与Vcc连接，目标第一类感光器件53转换得到的第一类电流信号I 0经电流镜54放大4倍后与比较器(Comparer，CP)56的输入端连接，目标第一类感光器件53周围的4个非目标第一类感光器件转换得到的电流信号分别为I 1、I 2、I 3、I 4。
需要说明的是，图5中电流镜54为第一电流放大器。图5中并未画出目标第一类感光器件53周围的4个非目标第一类感光器件，仅仅画出了与每个非目标第一类感光器件串联的第一类控制开关M 1、M 2、M 3、M 4。I 1、I 2、I 3、I 4所在的线路合并成一条线路，实现加法器的作用。合并的一条线路与CP56的输入端连接。由CP56对放大后的第一类电流信号以及I 1、I 2、I 3、I 4之和进行比较。当前一时刻与当前时刻的比较结果一致，则不做输出，由DAC55将输入的指定数字信号转换为指定模拟信号，并将指定模拟信号输出至目标第一类感光器件53或者某一个非目标第一类感光器件。输出后再通过CP56进行比较，当前一时刻与后一时刻的比较结果相反时，由CP56的输出端输出事件脉冲信号，即CP56处于边沿触发状态，此时由三态门电路57输出指定数字信号。
在图5中，CP56与地之间还连接有电容58，电容58可以是实际电容，也可以是第一类电流模式有源像素传感器电路中虚拟出来的寄生电容，本发明实施例中对此不作具体限定。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，第一类电流模式有源像素传感器电路还包括：存储单元。存储单元与三态门电路的输出端连接，用于存储第一类电流模式有源像素传感器电路输出的指定数字信号。存储单元具体可以是寄存器、锁存器、SRAM、DRAM、忆阻器等。以寄存器为例，寄存器的位数可以根据DAC的精度进行选择，本发明实施例中在此可选择4位寄存器。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述电压模式有源像素传感器电路具体包括：所述目标第一类感光器件周围的第二预设数量个所述第二类感光器件，每个所述第二类感光器件均分别与一个第二类控制开关串联，同一时刻仅存在一个第二类控制开关处于导通状态；
所述电压模式有源像素传感器电路还包括：电流积分器、快门以及模数转换器，每个所述第二类感光器件以及串联的第二类控制开关构成器件支路，所有器件支路并联连接且共用所述电流积分器、所述快门以及所述模数转换器；
所述电流积分器用于获取所述电压模式有源像素传感器电路中目标电容的电 压模拟信号；所述快门用于控制所述电流积分器的积分时间；所述模数转换器用于将所述目标电容的电压模拟信号转换为所述目标电压信号。
具体地，本发明实施例中提供的双模态仿生视觉传感器中，可以多个第二类感光器件共用一个电压模式有源像素传感器，即每个电压模式有源像素传感器电路中包括目标第一类感光器件周围的第二预设数量个第二类感光器件。可以理解为，由目标第一类感光器件和周围的第二预设数量个第二类感光器件构成一组，该组内的目标第一类感光器件由一个电流模式有源像素传感器控制，该组内的所有非目标第一类感光器件由一个电压模式有源像素传感器控制。
如图6所示，为本发明实施例中提供的电压模式有源像素传感器的结构示意图。图6中，包括：4个第二类感光器件61、电流积分器(Current Integrator，CI)62、快门64以及模数转换器(Analog-to-Digital Converter，ADC)63。每个第二类感光器件61分别与一个第二类控制开关65串联，同一时刻仅存在一个第二类控制开关65处于导通状态。每个第二类感光器件61与串联的第二类控制开关65构成一个器件支路，4个器件支路并联，共用CI62、快门64以及ADC63。第二类控制开关65具体可以是MOS管。
CI62用于获取电压模式有源像素传感器电路中目标电容的电压模拟信号；ADC63用于将所述目标电容的电压模拟信号转换为所述目标电压信号。快门64用于控制CI62的积分时间。例如，快门64控制CI62的积分时间为33ms，则33ms后，快门64闭合，CI62得到目标电容的电压模拟信号，并由ADC63读出。本发明实施例中还可以在ADC63后连接一存储单元，将ADC63读出的目标电容的电压模拟信号存储在存储单元中。存储单元413具体可以是寄存器、锁存器、SRAM、DRAM、忆阻器等。以寄存器为例，寄存器的位数可以根据ADC63的精度进行选择，本发明实施例中在此可选择8位寄存器存储目标电容的电压模拟信号。在ADC63读出动作完成后，还可以断开快门64，继续使CI62对目标电容的电流进行积分。循环上述步骤可以完成对视频信号的获取。
如图7所示，为本发明实施例中提供的电压模式有源像素传感器电路的具体结构示意图。图7中，共包括4个第二类感光器件，分别为71、72、73、74，第二类感光器件71与第二类控制开关75串联形成第一器件支路，第二类感光器件72与第二类控制开关76串联形成第二器件支路，第二类感光器件73与第二类控制开关78串联形成第三器件支路，第二类感光器件74与第二类控制开关77串联形成第四器件支路。第一器件支路、第二器件支路、第三器件支路以及第四器件支路并联后与MOS管79、710连接，MOS管710与MOS管711连接。MOS管79用于起偏置作用，MOS管710用于起开关作用，MOS管711用于对某一器件支路上的第二类感光器件转换得到的第三类电流信号进行电流积分，得到目标电压信号，表征目标光信号中的光强信息。
本发明实施例中提供的电压模式有源像素传感器电路，通过与每个第二类感光器件串联的第二类控制开关实现对器件支路的控制，进而实现一个电压模式有源像素传感器电路控制多个第二类感光器件的功能，提高了双模态仿生视觉传感器的集成度。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，还包括：第二类电流模式有源像素传感器电路；
所述第二类电流模式有源像素传感器电路包括一个所述非目标第一类感光器件和所述第二预设数量个电流镜；
每个电流镜分别与所述非目标第一类感光器件周围的一个所述第二类感光器件串联。
具体地，本发明实施例中，用于控制非目标第一类感光器件的第二类电流模式有源像素传感器电路具体包括：非目标第一类感光器件和第二预设数量个电流镜，每个电流镜分别与非目标第一类感光器件周围的一个目标第一类感光器件串联。也就是说，本发明实施例中每个第二类电流模式有源像素传感器电路均控制一个非目标第一类感光器件。
如图8所示，第二类电流模式有源像素传感器电路包括一个非目标第一类感光器件81和4个第一类电流镜82、83、84、85。每个第一类电流镜分别与非目标第一类感光器件81周围的一个目标第一类感光器件串联，即将非目标第一类感光器件81转换得到的电流信号I 1复制成4个I 1，分别用于包括非目标第一类感光器件81周围的每个目标第一类感光器件的第一类电流模式有源像素传感器电路获取到目标光信号中的光强梯度信息，以实现非目标第一类感光器件的复用，提高双模态仿生视觉传感器的像素填充因子。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述第一类感光器件具体为光电二极管(Photo-Diode，PD)，也可以是其他可以将光信号转换为电流信号的器件，本发明实施例中对此不作具体限定。需要说明的是，第一类感光器件中不包含有滤光器。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述第二类感光器件具体包括滤光器和光电二极管；
所述滤光器用于获取所述目标光信号，从所述目标光信号中提取指定频段的光信号；所述光电二极管用于将所述指定频段的光信号转换为所述第三类电流信号。
具体地，本发明实施例中，第二类感光器件用于对目标光信号中的彩色分量进行感应，第二类感光器件具体可以包括PD和设置在PD上的滤色器(Colour Filter，CF)，最终由仿生视觉传感器得到的图像是彩色图像。其中，CF用于获取目标光信号，从目标光信号中提取指定频段的光信号，PD将指定频段的光信号光 信号转换为第三类电流信号。滤色器具体可以是滤光片或者透镜，用于透过指定波长的光信号。当滤色器为透镜时，具体可选用拜伦透镜，也可以选用其他类型的透镜。滤色器按透过光信号的波长大小可分为红色滤色器、蓝色滤色器以及绿色滤色器，其透过的光信号分别为红光信号、蓝光信号和绿光信号。
需要说明的是，第二类感光器件还可以直接由光电二极管组成，通过选取不同响应曲线的光电二极管实现获取目标光信号，从目标光信号中提取指定波段的光信号，并将指定波段的光信号转换为第三类电流信号的作用。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述双模态仿生视觉传感器的像素阵列由所述目标第一类感光器件、所述非目标第一类感光器件以及所述第二类感光器件排列形成；
所述像素阵列的每一行中，所述第二类感光器件与所述目标第一类感光器件相间排列，或者所述第二类感光器件与所述非目标第一类感光器件相间排列。
具体地，像素阵列的排布方式结构示意图可以如图9所示，其中包括第一类感光器件91和第二类感光器件92，每个第一类感光器件和每个第二类感光器件均分别构成一个像素。第一类感光器件91中的目标第一类感光器件标记为“+”，非目标第一类感光器件标记为“-”。包含有红色滤色器的第二类感光器件92标记为“R”，包含有蓝色滤色器的第二类感光器件92标记为“B”，包含有绿色滤色器的第二类感光器件92标记为“G”。每个目标第一类感光器件周围具有4个非目标第一类感光器件以及4个第二类感光器件，每个非目标第一类感光器件周围具有4个目标第一类感光器件及4个第二类感光器件。
像素阵列的排布方式结构示意图也可以如图10所示，其中包括第一类感光器件101和第二类感光器件102，第一类感光器件101中的目标第一类感光器件标记为“+”，非目标第一类感光器件标记为“-”。包含有红色滤色器的第二类感光器件102标记为“R”，包含有蓝色滤色器的第二类感光器件102标记为“B”，包含有绿色滤色器的第二类感光器件102标记为“G”。每个目标第一类感光器件周围具有6个非目标第一类感光器件以及2个第二类感光器件，每个非目标第一类感光器件周围具有2个目标第一类感光器件以及4个第二类感光器件，或者每个非目标第一类感光器件周围具有4个目标第一类感光器件以及2个第二类感光器件。像素阵列的排布方式也可以是其他形式，本发明实施例中对此不作具体限定。
相应地，图9中示出的像素阵列，对应的第一预设数量和第二预设数量均为4，图10示出的像素阵列，对应的第一预设数量为6，第二预设数量为2或4。本发明实施例中均以图9中示出的像素阵列为例进行说明。例如，图7中的第二类感光器件71可以是标记为“G”的第二类感光器件，第二类感光器件72、73可以是标记为“R”的第二类感光器件，第二类感光器件74可以是标记为“B”的第二类感光器件。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述第一类电流模式有源像素传感器电路还包括：第二电流放大器；
所述第二电流放大器连接在所述目标第二类感光器件与所述第一电流放大器之间。
具体地，本发明实施例中，由于第一类感光器件转换得到的电流信号较小，因此可以在第一电流放大器与目标第一类感光器件之间连接有第二电流放大器，用于为目标第一类感光器件转换得到的第一类电流信号进行初步放大。第二电流放大器可以是实际的器件，也可以是实现电流放大功能的功能模块，本发明实施例中对此不作具体限定。相应地，目标第一类感光器件周围的非目标第一类感光器件与加法器之间也设置有第二电流放大器，使加法器之前每个非目标第一类感光器件所处支路的电流信号与目标第一类感光器件所处支路的电流信号处于同一量级。
在上述实施例的基础上，本发明实施例中提供的双模态仿生视觉传感器，所述目标电压信号和所述指定数字信号共同形成图像。
具体地，本发明实施例中，将目标电压信号和指定数字信号共同形成图像。需要注意的是，目标电压信号与指定数字信号的输出形式与速度并不相同。目标电压信号的输出速度约为30ms。而由于第一类电流模式有源像素传感器电路中数模转换器的扫描速度约为1ms，其后采用异步事件地址表示方式输出，具体为(X,Y,P,T)。其中，“X,Y”为事件地址，“P”为4值事件输出(包括第一位符号位)，“T”为事件产生的时间。
最终输出的图像如图11所示，两帧图片为彩色图片，由先后输出的目标电压信号形成，而两帧图片之间的边缘点由输出的指定数字信号形成。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (10)

  1. 一种双模态仿生视觉传感器，其特征在于，包括：第一类电流模式有源像素传感器电路和电压模式有源像素传感器电路；
    所述第一类电流模式有源像素传感器电路包括一个目标第一类感光器件；所述目标第一类感光器件用于获取目标光信号，并将所述目标光信号转换为第一类电流信号，所述第一类电流模式有源像素传感器电路用于基于所述第一类电流信号与所述目标第一类感光器件周围第一预设数量个非目标第一类感光器件转换得到的第二类电流信号之和的差值，输出表征所述目标光信号中的光强梯度信息的指定数字信号；
    所述电压模式有源像素传感器电路包括第二类感光器件，所述第二类感光器件用于获取所述目标光信号，从所述目标光信号中提取指定频段的光信号，并将所述指定频段的光信号转换为第三类电流信号，所述电压模式有源像素传感器电路用于基于所述第三类电流信号，输出表征所述目标光信号中的光强信息的目标电压信号；
    其中，每个非目标第一类感光器件分别与一个第一类控制开关串联。
  2. 根据权利要求1所述的双模态仿生视觉传感器，其特征在于，所述第一类电流模式有源像素传感器电路还包括：第一电流放大器、比较器、加法器和数模转换器；
    所述目标第一类感光器件与所述第一电流放大器相连，所述第一电流放大器与所述比较器的一个输入端相连；
    所述加法器的输入端分别与所述第一类控制开关连接，所述加法器的输出端与所述比较器的另一个输入端相连；
    所述比较器的输出端与所述数模转换器相连，所述数模转换器将输入的指定数字信号转换为指定模拟信号，并将所述指定模拟信号输出至所述第一电流放大器或者所述加法器，直至所述比较器的输出端输出事件脉冲信号，所述第一类电流模式有源像素传感器电路输出所述指定数字信号，所述指定数字信号用于表征所述目标光信号中的光强梯度信息。
  3. 根据权利要求2所述的双模态仿生视觉传感器，其特征在于，所述第一类电流模式有源像素传感器电路还包括：三态门电路；
    所述三态门电路分别与所述比较器的输出端以及所述数模转换器的输入端连接；
    所述三态门电路用于在所述比较器的输出端输出所述事件脉冲信号时，输出所述指定数字信号。
  4. 根据权利要求1所述的双模态仿生视觉传感器，其特征在于，所述电压模 式有源像素传感器电路具体包括：所述目标第一类感光器件周围的第二预设数量个所述第二类感光器件，每个所述第二类感光器件均分别与一个第二类控制开关串联，同一时刻仅存在一个第二类控制开关处于导通状态；
    所述电压模式有源像素传感器电路还包括：电流积分器、快门以及模数转换器，每个所述第二类感光器件以及串联的第二类控制开关构成器件支路，所有器件支路并联连接且共用所述电流积分器、所述快门以及所述模数转换器；
    所述电流积分器用于获取所述电压模式有源像素传感器电路中目标电容的电压模拟信号；所述快门用于控制所述电流积分器的积分时间；所述模数转换器用于将所述目标电容的电压模拟信号转换为所述目标电压信号。
  5. 根据权利要求4所述的双模态仿生视觉传感器，其特征在于，还包括：第二类电流模式有源像素传感器电路；
    所述第二类电流模式有源像素传感器电路包括一个所述非目标第一类感光器件和所述第二预设数量个电流镜；
    每个电流镜分别与所述非目标第一类感光器件周围的一个所述目标第一类感光器件串联。
  6. 根据权利要求1所述的双模态仿生视觉传感器，其特征在于，所述双模态仿生视觉传感器的像素阵列由所述目标第一类感光器件、所述非目标第一类感光器件以及所述第二类感光器件排列形成；
    所述像素阵列的每一行中，所述第二类感光器件与所述目标第一类感光器件相间排列，或者所述第二类感光器件与所述非目标第一类感光器件相间排列。
  7. 根据权利要求1所述的双模态仿生视觉传感器，其特征在于，所述第二类感光器件具体包括滤光器和光电二极管；
    所述滤光器用于获取所述目标光信号，从所述目标光信号中提取指定频段的光信号；所述光电二极管用于将所述指定频段的光信号转换为所述第三类电流信号。
  8. 根据权利要求1所述的双模态仿生视觉传感器，其特征在于，所述第一类电流模式有源像素传感器电路还包括：第二电流放大器；
    所述第二电流放大器连接在所述目标第二类感光器件与所述第一电流放大器之间。
  9. 根据权利要求1-8中任一项所述的双模态仿生视觉传感器，其特征在于，所述目标电压信号和所述指定数字信号共同形成图像。
  10. 根据权利要求1-8中任一项所述的双模态仿生视觉传感器，其特征在于，还包括：两个存储单元；
    所述两个存储单元分别用于存储所述目标电压信号以及所述指定数字信号。
PCT/CN2020/073523 2019-12-24 2020-01-21 双模态仿生视觉传感器 WO2021128533A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022536912A JP7335020B2 (ja) 2019-12-24 2020-01-21 バイモーダル生体模倣視覚センサ
US17/788,909 US11943550B2 (en) 2019-12-24 2020-01-21 Dual-modality neuromorphic vision sensor

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911348653.8A CN111083402B (zh) 2019-12-24 2019-12-24 双模态仿生视觉传感器
CN201911348653.8 2019-12-24

Publications (1)

Publication Number Publication Date
WO2021128533A1 true WO2021128533A1 (zh) 2021-07-01

Family

ID=70317305

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/073523 WO2021128533A1 (zh) 2019-12-24 2020-01-21 双模态仿生视觉传感器

Country Status (4)

Country Link
US (1) US11943550B2 (zh)
JP (1) JP7335020B2 (zh)
CN (1) CN111083402B (zh)
WO (1) WO2021128533A1 (zh)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112188092B (zh) * 2020-09-24 2022-09-30 北京灵汐科技有限公司 双模态信号处理系统和方法
CN112188093B (zh) * 2020-09-24 2022-09-02 北京灵汐科技有限公司 双模态信号融合系统和方法
TWI802015B (zh) * 2020-09-24 2023-05-11 大陸商北京靈汐科技有限公司 雙模態訊號處理系統和方法
CN112200757A (zh) * 2020-09-29 2021-01-08 北京灵汐科技有限公司 图像处理方法、装置、计算机设备及存储介质
CN112543271B (zh) * 2020-12-03 2022-09-02 清华大学 双模态紫外仿生视觉传感器
CN112600996B (zh) * 2020-12-03 2022-12-09 清华大学 紫外仿生视觉传感器
CN112532898B (zh) * 2020-12-03 2022-09-27 北京灵汐科技有限公司 双模态红外仿生视觉传感器
CN112584016B (zh) * 2020-12-03 2022-07-19 北京灵汐科技有限公司 红外仿生视觉传感器
WO2022116991A1 (zh) * 2020-12-03 2022-06-09 北京灵汐科技有限公司 仿生视觉传感器
CN112702588B (zh) * 2020-12-21 2023-04-07 北京灵汐科技有限公司 双模态图像信号处理器和双模态图像信号处理系统
CN112714301B (zh) * 2020-12-21 2023-07-21 北京灵汐科技有限公司 双模态图像信号处理器和图像传感器
WO2022135359A1 (zh) * 2020-12-21 2022-06-30 北京灵汐科技有限公司 双模态图像信号处理器和双模态图像信号处理系统
CN114697578B (zh) * 2020-12-31 2024-03-15 清华大学 基于三维堆叠技术的双模态图像传感器芯片及成像系统
CN113038046B (zh) * 2021-03-23 2023-07-25 北京灵汐科技有限公司 像素传感阵列和视觉传感器

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102984471A (zh) * 2012-12-10 2013-03-20 天津大学 由四管有源像素与数字像素组成的像素阵列
CN105163048A (zh) * 2015-09-11 2015-12-16 天津大学 基于放大器复用的动态视觉传感器
CN108200362A (zh) * 2017-12-19 2018-06-22 清华大学 基于空间对比度的仿生视网膜摄像电路及子电路
CN108566524A (zh) * 2018-01-31 2018-09-21 深圳市光微科技有限公司 像素单元、图像传感器芯片、成像系统、像素单元的形成方法以及深度信息测算方法
WO2019221580A1 (en) * 2018-05-18 2019-11-21 Samsung Electronics Co., Ltd. Cmos-assisted inside-out dynamic vision sensor tracking for low power mobile platforms

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002026304A (ja) 2000-07-03 2002-01-25 Sony Corp 固体撮像素子
JP2006279324A (ja) 2005-03-28 2006-10-12 Sanyo Electric Co Ltd 光検出装置
JP5655355B2 (ja) 2009-11-02 2015-01-21 ソニー株式会社 画素欠陥補正装置、撮像装置、画素欠陥補正方法、およびプログラム
US8266098B2 (en) * 2009-11-18 2012-09-11 International Business Machines Corporation Ranking expert responses and finding experts based on rank
FR2975251B1 (fr) * 2011-05-12 2015-12-04 Univ Pierre Et Marie Curie Paris 6 Procede et dispositif de commande d'un dispositif d'aide a la vision
US9047568B1 (en) * 2012-09-20 2015-06-02 Brain Corporation Apparatus and methods for encoding of sensory data using artificial spiking neurons
JP6166562B2 (ja) * 2013-03-21 2017-07-19 キヤノン株式会社 撮像素子及びその駆動方法、及び撮像装置
US10726337B1 (en) * 2015-04-30 2020-07-28 Hrl Laboratories, Llc Method and apparatus for emulation of neuromorphic hardware including neurons and synapses connecting the neurons
US10133944B2 (en) * 2016-12-21 2018-11-20 Volkswagen Ag Digital neuromorphic (NM) sensor array, detector, engine and methodologies
US10235565B2 (en) * 2016-12-21 2019-03-19 Volkswagen Ag System and methodologies for occupant monitoring utilizing digital neuromorphic (NM) data and fovea tracking
CN107333040B (zh) * 2017-07-13 2020-02-21 中国科学院半导体研究所 仿生视觉成像与处理装置
CN110324548B (zh) * 2019-06-27 2021-11-09 Oppo广东移动通信有限公司 一种像素单元电路、信号处理方法及存储介质
CN111083404B (zh) * 2019-12-24 2021-01-08 清华大学 视锥视杆双模态仿生视觉传感器
US20210312257A1 (en) * 2020-04-07 2021-10-07 Microsoft Technology Licensing, Llc Distributed neuromorphic infrastructure

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102984471A (zh) * 2012-12-10 2013-03-20 天津大学 由四管有源像素与数字像素组成的像素阵列
CN105163048A (zh) * 2015-09-11 2015-12-16 天津大学 基于放大器复用的动态视觉传感器
CN108200362A (zh) * 2017-12-19 2018-06-22 清华大学 基于空间对比度的仿生视网膜摄像电路及子电路
CN108566524A (zh) * 2018-01-31 2018-09-21 深圳市光微科技有限公司 像素单元、图像传感器芯片、成像系统、像素单元的形成方法以及深度信息测算方法
WO2019221580A1 (en) * 2018-05-18 2019-11-21 Samsung Electronics Co., Ltd. Cmos-assisted inside-out dynamic vision sensor tracking for low power mobile platforms

Also Published As

Publication number Publication date
US20230033544A1 (en) 2023-02-02
US11943550B2 (en) 2024-03-26
CN111083402B (zh) 2020-12-01
CN111083402A (zh) 2020-04-28
JP7335020B2 (ja) 2023-08-29
JP2023507363A (ja) 2023-02-22

Similar Documents

Publication Publication Date Title
WO2021128533A1 (zh) 双模态仿生视觉传感器
WO2021128531A1 (zh) 视锥视杆双模态仿生视觉传感器
WO2021196554A1 (zh) 图像传感器、处理系统及方法、电子设备和存储介质
CN103327342B (zh) 具有透明滤波器像素的成像系统
WO2021128532A1 (zh) 视锥视杆复用式仿生视觉传感器
WO2021128536A1 (zh) 像素阵列及仿生视觉传感器
CN112532898B (zh) 双模态红外仿生视觉传感器
WO2021128534A1 (zh) 视杆仿生视觉传感器
WO2021128535A1 (zh) 双模态仿生视觉传感器像素读出系统
WO2021016900A1 (zh) 一种图像传感器和图像感光的方法
CN103227928A (zh) 白平衡的调整方法及装置
US20210266475A1 (en) Image sensor and electronic camera
CN105430360B (zh) 成像方法、图像传感器、成像装置及电子装置
KR102285756B1 (ko) 전자 시스템 및 영상 처리 방법
KR20200098032A (ko) 이미지 센서의 픽셀 어레이 및 이를 포함하는 이미지 센서
CN107613183A (zh) 一种摄像头系统和摄像头系统的应用方法
JP2001016598A (ja) カラー撮像素子及び撮像装置
CN207589006U (zh) 一种广角摄像头模组以及终端
CN112543271B (zh) 双模态紫外仿生视觉传感器
KR20190100833A (ko) Hdr 이미지 생성 장치
CN112584016B (zh) 红外仿生视觉传感器
JP7164579B2 (ja) 画像センサー及び電子デバイス
WO2022116991A1 (zh) 仿生视觉传感器
CN216649833U (zh) 一种像素读出电路及图像传感器
CN116249024A (zh) 一种像素读出电路及图像传感器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20908238

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022536912

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20908238

Country of ref document: EP

Kind code of ref document: A1