WO2020063332A1 - 一种动态视觉传感器 - Google Patents

一种动态视觉传感器 Download PDF

Info

Publication number
WO2020063332A1
WO2020063332A1 PCT/CN2019/105187 CN2019105187W WO2020063332A1 WO 2020063332 A1 WO2020063332 A1 WO 2020063332A1 CN 2019105187 W CN2019105187 W CN 2019105187W WO 2020063332 A1 WO2020063332 A1 WO 2020063332A1
Authority
WO
WIPO (PCT)
Prior art keywords
order
voltage
order differential
photoinductive
differential voltage
Prior art date
Application number
PCT/CN2019/105187
Other languages
English (en)
French (fr)
Inventor
肖朝蕾
周建同
方运潭
高山
吴振华
肖晶
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP19864611.9A priority Critical patent/EP3849168A4/en
Publication of WO2020063332A1 publication Critical patent/WO2020063332A1/zh
Priority to US17/215,729 priority patent/US11310445B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array

Definitions

  • the invention relates to the technical field of image acquisition, in particular to a device and method for dynamic visual image acquisition.
  • a general camera can capture 25-30 frames per second, and its frame rate is not sufficient to support capturing images of high-speed moving objects.
  • high-speed cameras that collect data based on "frames”
  • DVS dynamic vision sensors
  • the high-speed camera can collect about 1000 frames per second.
  • DVS imaging equipment is widely used in the field of moving image acquisition due to its fast data acquisition, low acquisition volume, and low power consumption.
  • the pixel acquisition circuit structure in DVS mimics the imaging principle of the retina.
  • the photodiode in the DVS converts the perceived light signal into an electric signal, and amplifies and outputs the electric signal change caused by the brightness change through a first-order differential circuit.
  • the change is defined as an "event signal”. Only when the event signal occurs, that is, when the pixel experiences a change in brightness, the sensor will generate pulses and perform data communication to generate an image. See Figure 1 for the difference between the DVS imaging effect and the ordinary camera imaging effect. As shown in FIG.
  • the existing DVS sensor structure generally includes an image receiver, a differential circuit, and a comparison device, which respectively correspond to photoreceptors, bipolar cells, and ganglion cells in a biological vision system.
  • the image receiver the photodiode converts the perceived light signal into an electrical signal to form a logarithmic voltage.
  • the logarithmic representation method is convenient for measuring a wide range of light brightness; the differential circuit takes the logarithmic voltage as an input and passes through a differential amplifier.
  • the differential circuit After outputting the amplified voltage difference V diff , the differential circuit has the characteristics of “suppressing common mode signals and amplifying differential mode signals”, and can amplify and output electrical signal changes caused by brightness changes; finally, the comparison device compares the input V diff with the design
  • the threshold voltage is compared for comparison, and a pulse signal is output. For example, a positive pulse is output when V diff is greater than a positive threshold, and a negative pulse is output when the value is less than a negative threshold.
  • the pulse signal is transmitted to an external circuit through a handshake protocol for data communication and is also fed back to the differential.
  • the reset switch in the circuit resets the voltage for data communication when the next "event" arrives.
  • TCON time contrast ratio
  • the comparison device compares V diff with a threshold voltage and outputs a positive pulse or a negative pulse. It can be said that what the DVS imaging device itself records is not the actual scene picture, but the change in brightness in the scene. At the same time, the asynchronous output mode can output pixel changes quickly, preventing bus congestion. Therefore, DVS imaging equipment greatly reduces the amount of data acquisition, optimizes the use of bus efficiency, and reduces energy consumption.
  • the existing DVS uses first-order differential circuits to capture moving images and is very sensitive to changes in brightness.
  • the imaging device itself is also in motion, it is equivalent to every pixel in the entire screen to experience the brightness change.
  • the captured picture is very noisy, it is difficult to filter out background interference and affect the key information in the recognition scene; at the same time, asynchronous
  • the advantages of the output are also not reflected, which may cause bus congestion.
  • the current DVS imaging equipment is used for road condition detection of the automatic driving system, road conditions and surrounding scenes are continuously recorded during driving. If an obstacle suddenly appears in front, the image captured by the camera will be obstructed in the surrounding scene. Panorama, the machine is difficult to distinguish and identify obstacles, prone to security risks.
  • the present application provides a dynamic vision imaging sensor and sensing device, and an imaging method applied by the sensor and sensing device.
  • an imaging method applied by the sensor and sensing device By making a second-order difference of the light-sensing voltage and generating a second-order event signal according to the second-order differential voltage, the camera to which the sensor or sensing device is applied can effectively reduce background information during imaging and effectively capture changes in the speed of light change Emergency.
  • the present application provides a dynamic vision imaging sensor.
  • the sensor may include a photoelectric conversion unit, a second-order difference unit, and a comparison unit.
  • the photoelectric conversion unit can be used to convert the received optical signal into an electrical signal to generate a photoinductive voltage
  • the second-order difference unit can be used to perform a second-order difference on the photoinductive voltage to generate a second-order differential voltage
  • the comparison unit can be used The second-order differential voltage is compared with a second-order comparison threshold, and a second-order event signal is generated according to the comparison result.
  • the sensor responds to the second-order difference result of the photoinductive voltage, so that the output of the second-order event signal represents the change in the speed of light change, that is, the sensor responds to the acceleration of light change, thereby reducing the continuity of the sensor due to its own motion Meaningless response output improves imaging quality and reduces bus congestion.
  • the second-order differential voltage may include a non-zero second-order differential voltage and a zero second-order differential voltage; the aforementioned second-order difference of the photoinductive voltage to generate the second-order differential voltage may include: the speed of change of the photoinductive voltage at the current moment.
  • the second-order difference unit is used to generate a non-zero second-order difference voltage; when the speed of the light-sensing voltage at the current time is the same as the speed of the light-sensing voltage at the previous time, the second-order difference The unit is used to generate a zero second-order differential voltage.
  • the speed of change is the same, which means that the voltage at the current time and the previous time are both changing, and the speed of change is equal, or that the voltage at the current time and the previous time are unchanged, that is, the speed of change is 0.
  • the second-order differential circuit responds differently to the relationship between the current time and the previous speed of the light-sensitive voltage, thereby achieving the effect that the output reflects the change in the speed of light.
  • the second-order comparison threshold may include a positive second-order comparison threshold and a negative second-order comparison threshold, and the positive second-order comparison threshold and the negative second-order comparison threshold may be set symmetrically or asymmetrically, which is not limited here.
  • the comparison unit can generate a positive pulse signal when the second-order differential voltage is not lower than the positive second-order comparison threshold, and can generate a negative pulse signal when the second-order differential voltage is not higher than the negative second-order comparison threshold.
  • a zero-pulse signal is generated when the first-order comparison threshold is higher than the negative second-order comparison threshold.
  • the positive and negative thresholds are set, and different signals are output according to the different relationship between the second-order differential voltage and the positive and negative thresholds, which can more accurately reflect the change of the change rate of the light-sensitive voltage and improve the output accuracy of the sensor.
  • the second-order differential unit may include a sample-and-hold circuit and a second-order differential circuit.
  • the sample and hold circuit can be used to convert the photoinductive voltage into a differential input voltage; the second-order differential circuit can be used to perform a second-order difference on the photoinductive voltage according to the differential input voltage to generate a second-order differential voltage.
  • the differential input voltage usually needs to include multiple voltages, and these voltages correspond to the photoinductive voltages at different times.
  • the photoinductive voltage usually includes three voltages, which correspond to the photoinductive voltages at three different times, respectively.
  • the second-order differential unit includes a sample-and-hold circuit so that the sensor does not need to introduce additional devices to acquire the photoinductive voltage at three different times to be input to the second-order differential circuit at the same time, which simplifies the structure of the sensor.
  • the second-order differential circuit may include a first first-order differential circuit and a second first-order differential circuit, and the foregoing two differential circuits are connected in series.
  • the first first-order differential circuit may be used to generate a first first-order differential voltage and a second first-order differential voltage according to the differential input voltage
  • the second first-order differential circuit may be used to generate a second-order according to the foregoing two first-order differential voltages.
  • Differential voltage The first-order differential circuit is connected in series to increase the flexibility of the sensor function.
  • the output of the first-order differential circuit can be used as part of the final output result to obtain the first-order differential data of the photoinductive voltage. Functions of first-order event signals and second-order event signals.
  • the second-order differential circuit may not adopt the form of the first-order differential circuit connected in series, but adopts a second-order differential circuit designed as a whole.
  • the overall design of the second-order differential circuit reduces the complexity of the circuit and increases the reliability of the device.
  • the first first-order differential voltage or the second first-order differential voltage may be sent to the comparison unit, and the comparison unit may be further configured to compare the received first-order differential voltage with a first-order comparison threshold, and according to the comparison, The result is a first-order event signal.
  • the first first-order differential voltage and the second first-order differential voltage can also be sent to the comparison unit at the same time.
  • the comparison unit compares the two first-order differential voltages with the first-order comparison threshold respectively, and according to the first first-order difference
  • the comparison result of the voltage generates a first-order event signal, and the second-order event signal is generated according to the comparison result of the second-order differential voltage.
  • the output of the first-order differential circuit is used as part of the output result to obtain the first-order differential data of the photoinductive voltage, so that the sensor has both the function of obtaining the first-order event signal and the second-order event signal, and increases the flexibility of the sensor function .
  • the sensor may further include a gating unit configured to control the second first-order differential circuit to generate a second-order differential voltage according to the first first-order differential voltage and the second first-order differential voltage when a preset condition is satisfied.
  • the preset condition may be determining whether the sensor is moving, and when determining that the sensor is moving, the first first-order differential voltage and the second first-order differential voltage are input to the second first-order differential circuit to generate a second-order differential voltage; when the sensor is judged When stationary, the first first-order differential voltage and the second first-order differential voltage are not input to the second first-order differential circuit.
  • the output of the first-order event signal when the sensor is stationary ensures the accuracy of the output content, and the second-order output when the sensor moves
  • the event signal reduces image noise caused by the sensor's own motion.
  • the senor may further include a readout circuit, and the photoelectric conversion unit is further configured to send the photoinductive voltage to the readout circuit, and the readout circuit is configured to integrate the photoinductive voltage to generate an integrated voltage signal.
  • the integrated voltage signal is used to generate a frame-based image, that is, an image generated by a conventional camera.
  • the sensor outputs a second-order event signal or a first-order event signal, it also outputs an integrated voltage signal, so that the output of the sensor can be used to generate dynamic visual images and regular images at the same time, which makes it possible to fuse the two to improve image quality. .
  • the present application provides a dynamic vision imaging sensing device, including a dynamic vision sensor array and a peripheral circuit.
  • the dynamic vision sensor array includes a plurality of sensors, and these sensors are the dynamic vision sensors described in any one of the possible implementation manners of the first aspect.
  • These dynamic vision sensors are used to generate second-order event signals based on the received light signals and send the second-order event signals to peripheral circuits.
  • the peripheral circuit is used to generate the address event representation according to the second-order event signal.
  • the address event represents the position information of the second-order event signal and the dynamic vision sensor that generates the second-order event signal.
  • the position information may be the coordinate information of the sensor, or any other information that can indicate the position of the sensor, which is not limited here.
  • the sensing device responds to the second-order difference result of the photoinductive voltage, so that the output address event representation characterizes the change in the speed of light change, that is, the sensor responds to the acceleration of the light change, thereby reducing the sensing device's own motion-induced
  • the continuous and meaningless response output improves imaging quality and prevents bus congestion.
  • the sensor in the dynamic vision sensor array adopts a structure in which a first-order differential circuit is connected in series to obtain a second-order differential circuit
  • the sensor can also output a first-order event signal to a peripheral circuit, and the peripheral circuit generates a first-order event signal based on the Address event representation.
  • the sensing device can obtain both the address event representation based on the first-order event signal and the address event representation based on the second-order event signal, which increases the flexibility of the function of the sensing device.
  • the output of the address event table based on the first-order event signal or the address event table based on the second-order event signal is selected according to different scenarios, so that the output of the sensing device better meets the needs of the scene.
  • the address event representation of the first-order event signal ensures the accuracy of the output content.
  • the address event representation based on the second-order event signal is output, which reduces the image noise caused by the sensor's own movement.
  • the sensor in the dynamic vision sensor array when the sensor in the dynamic vision sensor array includes a readout circuit, the sensor may also output an integrated voltage signal to a peripheral circuit. While outputting the address event representation, the sensing device can also output an integrated voltage signal for generating a frame-based image, so that the output of the sensing device can be used to generate a dynamic visual image and a conventional image at the same time, thereby merging the two. It became possible to improve the picture quality.
  • the present application provides a dynamic vision camera, which includes the dynamic vision sensing device described in any one of the embodiments of the second aspect, and further includes a working circuit that can be used to The address event of the first-order event signal indicates that an image is generated.
  • the image generated in this way records the information of the pixels that change the speed of light change, that is, only the key events that change the speed of light change are recorded, thereby reducing the background noise caused by the camera's own movement and improving the imaging quality.
  • the present application provides a mobile terminal device, including the dynamic vision camera described in the third aspect, and further including a communication unit, which can be used to send the generated image to other devices.
  • the image generated by the terminal device records the information of pixels that change the speed of light change, that is, only the key events that change the speed of light change are recorded, thereby reducing the background noise caused by the terminal device's own movement and improving the imaging quality.
  • images can also be sent to other devices, enabling sharing of the image among multiple devices.
  • the present application provides a real-time positioning and map construction system, including the dynamic visual sensing device described in any one of the embodiments of the second aspect, and further includes a working circuit for Scene recognition is performed based on the address event representation of the second-order event signal.
  • the output address event representation characterizes the change in the speed of light change, that is, the sensor responds to the acceleration of the light change, thereby reducing the number of errors caused by the system's own motion. Background noise interference improves the accuracy of object recognition.
  • the instant positioning and map building system can be implemented as a head-mounted display device, a drone, an intelligent road condition recognition system, and the like.
  • the present application provides a dynamic vision sensing method, including converting an optical signal into an electrical signal to generate a photoinductive voltage; performing a second-order difference on the photoinductive voltage to generate a second-order differential voltage; and combining the second-order differential voltage with two
  • the order comparison threshold value is compared, and a second order event signal is generated according to the comparison result.
  • the output second-order event signal represents the change in the speed of light change, that is, the output second-order event signal responds to the acceleration of the light change, thereby reducing the acquisition of optical signals.
  • the continuous and meaningless response output caused by the motion of the device itself reduces noise, reduces bus congestion, and improves imaging quality.
  • the second-order comparison threshold may include a positive second-order comparison threshold and a negative second-order comparison threshold, and the positive second-order comparison threshold and the negative second-order comparison threshold may be set symmetrically or asymmetrically, which is not limited here.
  • a positive pulse signal can be generated when the second-order differential voltage is not lower than the positive second-order comparison threshold.
  • a negative pulse signal can be generated when the second-order differential voltage is not higher than the negative second-order comparison threshold.
  • the second-order differential voltage can be lower than the positive second-order comparison.
  • a zero-pulse signal is generated when the threshold is higher than the negative second-order comparison threshold.
  • the positive and negative thresholds are set, and different signals are output according to the different relationship between the second-order differential voltage and the positive and negative thresholds, which can more accurately reflect the change of the change rate of the light-sensitive voltage and improve the output accuracy.
  • the speed of change is the same, which means that the voltage at the current time and the previous time are both changing, and the speed of change is equal, or that the voltage at the current time and the previous time are unchanged, that is, the speed of change is 0.
  • the second-order differential circuit responds differently to the relationship between the current time and the previous speed of the light-sensitive voltage, thereby achieving the effect that the output reflects the change in the speed of light.
  • performing a second-order difference on the photoinductive voltage may include: firstly performing a first-order difference on the photoinductive voltage to generate a first-order differential voltage; and performing a first-order difference on the aforementioned first-order differential voltage to generate a second-order differential voltage.
  • the form of two first-order differential series is used to increase the flexibility of the output function.
  • the first-order differential voltage can be used as a part of the output result to obtain the first-order differential data of the photoinductive voltage, thereby obtaining the first-order event signal and the second First-order event signal.
  • the first-order differential voltage can be compared with a first-order comparison threshold, and a first-order event signal is generated according to the comparison result.
  • a first-order comparison threshold refer to the foregoing description of the second-order comparison threshold, and details are not described herein again.
  • the first-order event signal is used as part of the final output result to obtain the first-order differential data of the photoinductive voltage, so that the first-order event signal and / or the second-order event signal can be obtained, which increases the flexibility of the method.
  • the light-induced voltage can also be integrated to generate an integrated voltage signal.
  • the integrated voltage signal is used to generate a frame-based image, that is, an image generated by a conventional camera.
  • an integrated voltage signal is also output, so that the final output can be used to generate a dynamic visual image and a conventional image at the same time, thereby making it possible to fuse the two to improve the image quality.
  • FIG. 1 is a comparison diagram of imaging effects of a conventional DVS camera
  • FIG. 2 is a schematic structural diagram of a prior art DVS sensor
  • FIG. 3 is a schematic diagram of a logical structure of a dynamic vision sensor
  • FIG. 4 is a schematic structural diagram of a photoelectric conversion circuit
  • FIG. 5 is a schematic structural diagram of a sample-and-hold circuit
  • FIG. 6 is a schematic diagram of timing control of a sample-and-hold circuit
  • FIG. 7 is a schematic structural diagram of a second-order differential circuit
  • FIG. 8 is a schematic diagram of a logic structure of another second-order differential circuit
  • FIG. 10 is a schematic diagram of a logical structure of another dynamic vision sensor
  • FIG. 11 is a schematic diagram of a logical structure of a dynamic vision sensing device
  • FIG. 12 is a schematic structural diagram of a dynamic vision sensing device
  • FIG. 13 is a schematic flowchart of a dynamic vision sensing method
  • 15 is a schematic diagram of a logical structure of a dynamic vision camera
  • 16 is a schematic diagram of a logical structure of a mobile terminal
  • FIG. 17 is a schematic diagram of a logical structure of an instant positioning and map construction system.
  • FIG. 3 is a schematic diagram of a logical structure of a possible embodiment of a dynamic vision sensor according to the present invention.
  • the dynamic vision sensor 100 includes a photoelectric conversion unit 110, a second-order difference unit 120, and a comparison unit 130.
  • the second-order difference unit includes a sample-and-hold circuit 121 and a second-order difference circuit 122.
  • the dynamic vision sensor 100 may further include a readout circuit 140.
  • the sample-and-hold circuit 121 may also be considered as a component part of the photoelectric conversion unit 110.
  • this embodiment only uses the sample-and-hold circuit 121 as the second-order differential unit 120 as an example for description. It must be noted that no matter which functional unit the sample-and-hold circuit is assigned to, this type of structural division belongs to the technical solution of the present application. Protective structure.
  • the photoelectric conversion unit 110 is configured to convert the received optical signal into an electrical signal, generate a corresponding photoinductive voltage, and send it to the second-order difference unit 120.
  • the photoelectric conversion unit 110 also sends a light-sensitive voltage to the readout circuit 140.
  • the photoelectric conversion unit 110 may be composed of a conventional photoelectric conversion circuit, and is configured to convert a received optical signal into a corresponding electrical signal, that is, a photo-induced voltage.
  • the magnitude of the photoinductive voltage is usually linearly proportional to the intensity of the light. See Figure 4 for a possible implementation of the photoelectric conversion circuit.
  • the input of the optical signal passes through the photodiode and is converted into a photoinductive voltage by a typical photoelectric conversion circuit as shown in the figure.
  • the photoelectric conversion unit 110 may also be implemented as any other form of device or circuit having a photoelectric conversion function, which is not limited herein.
  • the second-order difference unit 120 is configured to perform a second-order difference on the photoinductive voltage to generate a second-order differential voltage.
  • the sample-and-hold circuit 121 is configured to convert the photoinductive voltage into a differential input voltage
  • the second-order differential circuit 122 is configured to perform a second-order difference on the photoinductive voltage according to the differential input voltage.
  • the photoinductive voltage is a continuous voltage output by the photoelectric conversion unit 110
  • the second-order difference circuit 122 performs second-order difference, which requires multiple photoinductive voltages corresponding to different times at the same time as the input of the second-order difference circuit 122.
  • the photoinductive voltages corresponding to a plurality of different times that are simultaneously input as the second-order differential circuit 122 are called differential input voltages.
  • the differential input voltage of the second-order differential circuit is based on an example that includes the photoinductive voltages corresponding to three different times. It should be noted that, in some possible implementation manners, as the requirements for the differential order of the photoinductive voltage are different (such as third-order differential or fourth-order differential), the number of photoinductive voltages that may be included in the differential input voltage at corresponding moments will also be different. It is not limited here.
  • the sample-and-hold circuit 121 includes three sample-and-hold 1211 for holding the respective input voltages for a certain time; three selectors 1212 for selecting one of the three corresponding inputs of A 1 , A 2 and A 3 For output.
  • the continuous input of the photoinductive voltage is U x
  • the three voltages included in the differential input voltage are the output voltages U 1 , U 2, and U 3 of the three selectors, respectively.
  • s1, s2, and s3 are the timing control inputs of the three sample-and-hold, respectively
  • M1 and M2 are the timing control inputs of the selector.
  • FIG. 6 shows a timing control method that the sample-and-hold circuit 121 can adopt.
  • the timing diagram in Figure 6 shows the two cycle cycles of the sequence control. In practical applications, this cycle can continue to cycle during the use of the circuit.
  • the working principle of the sample-and-hold circuit will be described by example.
  • the timing control method shown in FIG. 6 is adopted, for example, at the initial time, the sample-and-hold s3 and all switches A 1 are turned on, and U 1 , U 2 , and U 3 have initial voltages.
  • the sample holder s1 and all switches A 2 are turned on, so that the voltage at the original U 3 is output to U 2 , the voltage at the original U 2 is output to U 1 , and the U x at time 1 is output to U 3 ;
  • the sample holder s2 and all switches A 3 are turned on, the voltage at the original U 3 is output to U 2 , the voltage at the original U 2 is output to U 1 , and the U x at time 2 is output to U 3 ;
  • the sample and holder s3 and all switches A 1 are turned on, the voltage at the original U 3 is output to U 2 , the voltage at the original U 2 is output to U 1 , the U x at time 3 is output to U 3 , and so on.
  • the sample-and-hold circuit may also take any other circuit form with a sample-and-hold function, such as a capacitor, a digital phase shifter that implements a delayed phase function, and the like, which are not limited here
  • U 1 , U 2 and U 3 are differential input voltages
  • GND is the ground point
  • U 0 is the second-order differential voltage.
  • the second-order differential voltage U 0 is output.
  • the relationship between U 0 and U 1 , U 2 , U 3 satisfies the following equation:
  • FIG. 8 is a schematic diagram of the logical structure of the implementation of the second-order differential circuit.
  • the second-order differential circuit 122 includes a first first-order differential circuit 1221 and a second first-order differential circuit 1222, a first first-order differential circuit, and a second first-order differential circuit. Connect in series.
  • the first first-order differential circuit 1221 receives a differential input voltage, and generates a first first-order differential voltage and a second first-order differential voltage according to the differential input voltage.
  • the second first-order differential circuit is based on the first-order differential voltage and the second first-order voltage.
  • the differential voltage generates a second-order differential voltage.
  • U 9 is a circuit diagram of an embodiment of the second-order differential circuit.
  • U 1 , U 2 and U 3 are differential input voltages
  • GND is the ground point
  • U 0 is the second-order differential voltage of the output.
  • the first first-order differential circuit 1221 generates a first first-order differential voltage U A and a second first-order differential voltage U B according to U 1 , U 2 , and U 3
  • the second first-order differential circuit 1222 generates two second-order differential voltages according to U A and U B.
  • Order differential voltage U 0 The relationship between U A , U B and U 1 , U 2 , U 3 satisfies the following formula:
  • the second-order differential circuit 122 A positive or negative voltage is generated, that is, a non-zero second-order differential voltage; when the speed of change of the photoinductive voltage at the current time is the same as the speed of change of the photoinductive voltage at the previous time, the output voltage of the second-order differential circuit 122 is 0, that is, Generates a zero second-order differential voltage.
  • the comparison unit 130 is configured to compare the second-order differential voltage with a second-order comparison threshold, and generate a second-order event signal according to the comparison result.
  • the comparison unit 130 may be implemented as any device capable of comparing voltages and outputting a signal according to the comparison result, such as, but not limited to, a window comparator, a dedicated voltage comparison chip, and the like.
  • the second-order comparison threshold may include a positive second-order comparison threshold and a negative second-order comparison threshold.
  • the positive second-order comparison threshold may be set to positive 300 mV, and the negative second-order comparison threshold may be set to negative 300 mV.
  • the positive and negative thresholds may not be set symmetrically, and are not limited here.
  • the comparison unit 130 may output a positive pulse when the second-order differential voltage is higher than or equal to a positive second-order comparison threshold, and may output a negative pulse when the second-order differential voltage is lower than or equal to a negative second-order comparison threshold, or may be low when the second-order differential voltage is low.
  • a zero pulse is output when the positive second-order comparison threshold is higher than the negative second-order comparison threshold.
  • the second-order comparison threshold may only have a positive threshold, and only generate a positive pulse when the second-order differential voltage is greater than the positive threshold; or only have a negative threshold, and generate a negative pulse only when the second-order differential voltage is less than the negative threshold. , Not limited here.
  • the readout circuit 140 is configured to generate an integrated voltage signal according to the received photoinductive voltage.
  • the readout circuit integrates the received photoinductive voltage and outputs an integrated voltage signal according to the integration result over a period of time (such as 1 millisecond).
  • the readout circuit 140 may be composed of a transistor.
  • the integrated voltage signal generated by the readout circuit 140 is used to generate a frame-based image.
  • the second-order differential circuit 122 adopts a structure as shown in FIG. 9, the first first-order differential voltage U A and the second first-order differential voltage U B generated by the first first-order differential circuit 1221. It may be directly sent to the comparison unit 130, and the comparison unit 130 may be further configured to compare U A and / or U B with a first-order difference threshold, and generate a first-order event signal according to the comparison result.
  • the comparison unit 130 may be further configured to compare U A and / or U B with a first-order difference threshold, and generate a first-order event signal according to the comparison result.
  • FIG. 10 is a schematic diagram of a logical structure of another possible embodiment of a dynamic vision sensor according to the present invention.
  • the dynamic vision sensor 200 includes a photoelectric conversion unit 210, a second-order difference unit 220, a first-order comparison unit 231, and a second-order comparison unit 232.
  • the second-order differential unit includes a sample-and-hold circuit 221, a first first-order differential circuit 222, a first gating unit 223, a second first-order differential circuit 224, and a second gating unit 225.
  • the dynamic vision sensor may further include a readout circuit 240.
  • the sample-and-hold circuit 221 may also be considered as a component part of the photoelectric conversion unit 210.
  • this embodiment only uses the sample-and-hold circuit 221 as the second-order difference unit 220 as an example for description. It should be noted that no matter which functional unit the sample-and-hold circuit is assigned to, this type of structural division belongs to the technical solution of the present application. Protective structure.
  • the second-order difference unit 220 is configured to perform second-order difference on the photoinductive voltage to generate a second-order differential voltage.
  • the sample-and-hold circuit 221 is used to convert the photoinductive voltage into a differential input voltage.
  • the photoinductive voltage is a continuous voltage output by the photoelectric conversion unit 210, and the first first-order differential circuit 222 requires multiple different voltages.
  • the photoinductive voltage corresponding to the time is also used as an input.
  • the differential input voltage of the first first-order differential circuit 222 includes the photoinductive voltage corresponding to three different times as an example, and is provided by the sample-and-hold circuit 221.
  • FIG. 5 and FIG. 6 Will not repeat them here.
  • the number of photoinductive voltages at the corresponding moment that the differential input voltage may include, for example, in the case of a third order difference
  • the differential input voltage may include 5 photo-sensitive voltages at corresponding times, which is not limited here.
  • the first first-order differential circuit 222 receives a differential input voltage and generates a first first-order differential voltage and a second first-order differential voltage according to the differential input voltage.
  • the first gating unit 223 determines whether to perform a second first-order differential.
  • the second-order differential circuit when performing the second-order first-order difference, the second-order differential circuit generates a second-order differential voltage according to the first-order differential voltage and the second-order differential voltage, and the first-order differential voltage and the second-order differential The voltage is also sent to the second gating unit; when it is determined that the second first-order difference is not performed, the first first-order differential voltage and the second first-order differential voltage are directly sent to the second gating unit.
  • the first first-order differential voltage, the second first-order differential voltage, and the second-order differential voltage refer to the descriptions in FIG. 8 and FIG. 9, and details are not described herein again.
  • the first gating unit 223 is used to control whether the second-order first-order difference is performed on the photoinductive voltage.
  • the control condition may be to determine whether the camera is moving. If it is judged that the camera is in motion, the second-order first-order difference is determined to be stationary. Do not do the second first-order difference.
  • the second gating unit 225 is used for outputting the first-order differential voltage to the first-order comparison unit 231 and / or outputting the second-order differential voltage to the second-order comparison unit when both the first-order differential voltage and the second-order differential voltage are used as inputs. 232; When there is only a first-order differential voltage or only a second-order differential voltage as an input, the received voltage is output to a corresponding comparison unit.
  • the first gating unit and the second gating unit may be specifically implemented by a multiplexer or a matrix switch, which is not limited herein.
  • the first-order comparison unit 231 is configured to compare the received first-order differential voltage with a first-order differential threshold, and generate a first-order event signal according to the comparison result.
  • the second-order comparison unit 232 is configured to compare the received second-order differential voltage with a second-order comparison threshold, and generate a second-order event signal according to the comparison result.
  • the second-order differential unit 220 may include more first-order differential circuits and gating units to implement further multi-order differential of the photoinductive voltage.
  • the second-order differential unit 220 may include more first-order differential circuits and gating units to implement further multi-order differential of the photoinductive voltage.
  • FIG. 11 is a schematic diagram of a logical structure of a possible embodiment of a dynamic vision sensing device according to the present invention.
  • the dynamic vision sensing device 300 includes a sensor array 310 and a peripheral circuit 320.
  • the peripheral circuit 320 includes an address encoder 321 and a first encoder 322.
  • the peripheral circuit 320 further includes a first cache 323.
  • the peripheral circuit 320 further includes a second encoder 324 and a second buffer 325.
  • the sensor array 310 includes a plurality of any of the dynamic vision sensors described in the embodiments of the present application.
  • the dynamic vision sensor in the sensor array 310 is used to generate a second-order event signal or a first-order event signal according to the light signal.
  • a second-order event signal or a first-order event signal according to the light signal.
  • the second-order event signal and the first-order event signal as event signals.
  • the peripheral circuit 320 is configured to generate an address event representation according to the event signal. Specifically, the sensor array 310 sends the event signal to the address encoder 321, and the address encoder 321 sends the coordinates of the sensor generating the event signal to the first encoder 322 together with the event signal.
  • the first encoder 322 is configured to generate an address event representation according to the received event signal and coordinates, and encode the address event representation to generate binary code information.
  • the peripheral circuit 320 further includes a first buffer 323 for storing the address event representation encoded by the first encoder 322 for the external device to call.
  • the first cache 323 may also be used to store information output by the address encoder 321, which is not limited herein.
  • the peripheral circuit 320 further includes a second encoder 324 and a second buffer 325.
  • the second encoder is used for receiving and encoding the integrated voltage signal sent by the sensor array, and generating binary code information for the external device to call.
  • the binary code information generated from the integrated voltage signal can be used to generate frame-based images.
  • the second buffer 325 may be used to store the information generated by the second encoder 324 encoding.
  • FIG. 12 is a schematic structural diagram of another possible embodiment of a dynamic vision sensing device according to the present invention.
  • the dynamic vision sensing device 400 includes a sensor array 410, a row address encoder 420, a column address encoder 430, a first encoder 440, and a first buffer 450.
  • a second encoder 460 and a second cache 470 may also be included.
  • the sensor array 410 includes a plurality of any of the dynamic vision sensors described in the embodiments of the present application.
  • the dynamic vision sensor in the sensor array 410 is used to generate an event signal according to a light signal.
  • the event signal includes a second-order event signal and a first-order event signal.
  • the sensors in the sensor array 410 send the event signals to the row address encoder 420 and the column address encoder 430, respectively.
  • the row address encoder 420 is used to send the row coordinates of the sensor that generates the event signal to the first encoder 440 after receiving the event signal sent by the sensor; the column address encoder 430 is used to receive the sensor signal The column coordinates of the sensor generating the event signal are sent to the first encoder 440 together with the event signal.
  • the first encoder 440 is configured to generate an address event representation according to the received event signal and coordinates, and encode the address event representation to generate binary code information.
  • the first buffer 450 is used to store the address event representation encoded by the encoder for the external device to call.
  • the first buffer 450 is also used to store information output by the column address encoder 430.
  • the first buffer 450 may also be used to store information output by the row address encoder 420, which is not limited herein.
  • the second encoder 460 is configured to receive and encode the integrated voltage signal sent by the sensor array, and generate binary code information for the external device to call.
  • the binary code information generated from the integrated voltage signal can be used to generate frame-based images.
  • the second buffer 470 may be used to store information generated by the second encoder 460 encoding.
  • the dynamic vision sensing device shown in FIG. 11 or FIG. 12, or other dynamic vision sensing device including the dynamic vision sensor described in FIG. 3 to FIG. 10, can be used according to a period of time (such as 3 milliseconds) in actual use. Accumulated event signals to generate the output corresponding to the sensors that generate these event signals; it can also generate output based on the preset proportion reached by the sensors that generate the event signals (for example: more than 80% of sensors in a row or column have event signals Output, that is, the pulse output that produces the row or column). It should be noted that the event signal here can be either a first-order event signal or a second-order event signal.
  • the first-order event signal can be considered as an event signal that characterizes a change in light brightness
  • the second-order event signal can be considered as a characterization of a change in the rate of light brightness change.
  • the background information will be reduced, and the information that changes the light change speed (such as objects that suddenly enter the screen) is mainly retained in the image, thereby reducing Redundant background information from camera movement.
  • the camera is installed on a moving vehicle, stationary trees, parked vehicles, etc. on the roadside may be filtered out, and overtaking vehicles, falling rocks, etc. may be captured, thereby achieving better shooting effects. .
  • FIG. 13 is a schematic diagram of a possible process of a dynamic vision sensing method according to the present invention.
  • S510 Convert the optical signal into an electrical signal to generate a photoinductive voltage.
  • Optical signals can be converted into electrical signals by a photoelectric conversion device as an input for differential operation.
  • S520 Perform a first-order difference on the photoinductive voltage.
  • a first-order difference is performed on the photoinductive voltage to obtain a first-order differential voltage.
  • S530 Determine whether to perform first-order difference. Performing the first-order differential on the first-order differential voltage obtained in S520 again will obtain the second-order differential voltage of the photoinductive voltage.
  • the first-order differential voltage characterizes the change in light
  • the second-order differential characterizes the change in the speed of light change. You can determine whether to perform the second-order first-order difference according to the requirements of different usage scenarios. For example, the second-order first-order difference is performed when the sensor is moving, and the second-order first-order difference is not performed when the sensor is stationary.
  • S540 performs the second first-order difference.
  • the first-order differential voltage obtained in S520 is to be subjected to another first-order difference.
  • S540 may be performed directly after S520 without the judgment of S530, that is, the second-order first-order difference is directly performed on the first-order differential voltage generated by S520 without judgment, which is not limited here.
  • S550 compares the differential voltage with a threshold and outputs a comparison result.
  • the first-order differential voltage generated by S520 is received, the first-order differential voltage is compared with the first-order comparison threshold, and a first-order event signal is generated according to the comparison result.
  • the second-order differential voltage generated by S540 is received, the second-order differential voltage is received. The voltage is compared with a second-order comparison threshold to generate a second-order event signal.
  • FIG. 14 is another schematic flowchart of a dynamic visual sensing method according to the present invention.
  • S610 converts the optical signal into an electrical signal, and generates a photoinductive voltage.
  • S620 performs second-order difference on the photoinductive voltage to generate a second-order differential voltage.
  • S630 generates a second-order event signal according to the second-order differential voltage.
  • S640 integrates the photoinductive voltage.
  • S650 generates an integrated voltage signal according to the integration result over a period of time.
  • the integrated voltage signal is used to generate a frame-based image.
  • FIG. 15 is a schematic diagram of a logical structure of an embodiment of a dynamic vision camera to which the dynamic vision sensing device of the present invention is applied.
  • the dynamic vision camera 700 includes a dynamic vision sensing device 710 and a working circuit 720.
  • the dynamic visual sensing device 710 is configured to generate an address event representation according to the optical signal.
  • the working circuit 720 is configured to generate an image according to the address event indication.
  • the working circuit 720 may be implemented in the form of a processor, a chip, or the like, which is not limited herein.
  • FIG. 16 is a schematic diagram of a logical structure of an embodiment of a mobile terminal device to which the dynamic visual sensing device of the present invention is applied.
  • the mobile terminal 800 includes a dynamic vision camera 810 and a communication unit 820.
  • the dynamic vision camera 810 is configured to generate an image based on a light signal.
  • the communication unit 820 is configured to send an image to another device.
  • the communication unit may be a wireless signal transmitting device or a transmission port, which is not limited herein.
  • FIG. 17 is a schematic diagram of a logical structure of an embodiment of a real-time positioning and map construction system using the dynamic visual sensing device of the present invention.
  • the real-time positioning and map building system 900 includes a dynamic visual sensing device 910 and a working circuit 920.
  • the dynamic vision sensing device 910 is configured to generate an address event indication according to a light signal.
  • the working circuit 920 is configured to perform scene recognition according to the address event indication.
  • the working circuit in one or more embodiments may use an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), At least one of a field programmable gate array (field programmable gate array, FPGA), a processor, a controller, a microcontroller, and / or a microprocessor is used to implement the embodiments of the present application.
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • PLD programmable logic device
  • FPGA field programmable gate array
  • processor a controller, a microcontroller, and / or a microprocessor is used to implement the embodiments of the present application.
  • the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code, and executed by a hardware-based processing unit.
  • an embodiment means that a particular feature, structure, or characteristic related to the embodiment is included in at least one implementation of the invention. Example. Thus, the appearances of "in one embodiment”, “in an embodiment”, or “in some possible implementations” appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
  • B corresponding to A means that B is associated with A, and B can be determined according to A.
  • determining B based on A does not mean determining B based on A alone, but also determining B based on A and / or other information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

本申请提供一种动态视觉传感器。该传感器通过光电转换单元将光信号转换为电信号，生成光感电压，并通过二阶差分电路对光感电压进行二阶差分，根据二阶差分的结果生成二阶事件信号。包含该传感器的摄像机能够根据该二阶事件信号生成图像，该图像表征光的变化速度的变化，因此在摄像机运动时能减少冗余背景信息。

Description

一种动态视觉传感器
本申请要求于2018年09月29日提交中国国家知识产权局、申请号为201811151126.3、申请名称为“一种动态视觉传感器”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及一种图像采集的技术领域，特别是涉及一种动态视觉图像采集的装置和方法。
背景技术
随着社会的发展，图像采集领域对采集图像的质量有了越来越高的要求。一般摄像头每秒可采集25-30帧，其帧率不足以支撑采集高速运动物体的图像。目前在运动图像采集领域主要有两种摄像技术：基于“帧”采集数据的高速摄像头和基于“事件信号”采集数据的动态视觉传感器(dynamic vision sensor，DVS)。高速摄像头每秒可采集约1000帧，虽然能捕捉到高速运动的物体，但由于采集的帧数非常多，其数据采集量巨大，能耗偏高，效率不佳。相比之下，DVS成像设备由于其数据采集快，采集量不高，功耗偏低，因而在运动图像采集领域得到了广泛应用。
DVS中像素的采集电路结构模仿了视网膜的成像原理。DVS中的光电二极管将感受到的光信号转换为电信号，并通过一阶差分电路将亮度变化导致的电信号变化放大输出。当电信号变化大于某一阈值时，这一变化被定义为一个“事件信号”，只有当事件信号发生即像素感受到亮度变化时，传感器才会产生脉冲并进行数据通信，进而生成图像。DVS成像效果与普通相机成像效果的区别请参见图1。如图2所示，现有的DVS传感器结构通常包括图像接收器、差分电路和比较装置，在功能上分别对应了生物视觉系统中的光感受器、双极细胞和神经节细胞。在图像接收器中，光电二极管将感受到的光信号转换为电信号，形成对数电压，对数的表示方法便于衡量大范围的光线亮度；差分电路将对数电压作为输入，经过差分放大器，输出放大之后的电压差V diff，差分电路具有“对共模信号抑制，对差模信号放大”的特征，可以将亮度变化导致的电信号变化放大输出；最后比较装置将输入的V diff与设定的阈值电压作比较，输出脉冲信号，比如在V diff大于正阈值时输出正脉冲，小于负阈值时输出负脉冲，脉冲信号通过握手协议传入外部电路，进行数据通信，同时也反馈给差分电路中的复位开关，电压复位，以便在下一个“事件”到来时进行数据通信。综合这三个阶段，只有事件发生即亮度变化时，传感器才会产生脉冲，进行数据通信。传感器对时间衬比(TCON,Temporal CONtrast)十分敏感，TCON衡量了一定时间内电流对数值的变化量，定义为：
Figure PCTCN2019105187-appb-000001
差分电路的输出电压V diff与TCON的关系可以表示为：
Figure PCTCN2019105187-appb-000002
比较装置将V diff和阈值电压作比较后输出正脉冲或负脉冲。可以说，DVS成像设备本身记录的并不是实际场景画面，而是场景中亮度的变化。同时，异步的输出方式能够快速的将像素变化输出，防止了总线堵塞。因此，DVS成像设备很大程度上减小了数据采集量，优化了总线使用效率，同时降低了能耗。
然而，现有DVS利用一阶差分电路进行运动图像的捕捉，对亮度变化十分敏感。在成像设备本身也在运动的情况下，相当于整个画面中每一个像素都会感受到亮度变化，所捕获到的画面十分嘈杂，难以滤除背景干扰，影响识别场景中的关键信息；同时，异步输出的优势也无法体现，可能造成总线堵塞。比如，如果用现有DVS成像设备做自动驾驶系统的路况侦测，在行车途中持续记录路况及周边景物，若前方突然出现障碍物，摄像头捕捉到的图像将是障碍物混杂在周边场景里的全景图，机器很难分辨和识别障碍物，容易出现安全隐患。此外，在成像设备本身运动的情况下，由于每个像素点的亮度都会变化，导致所有像素点都会持续采集信息，使得不仅原本减少数据采集量的优势难以显现，而且还有可能造成信道堵塞，导致通信不畅，影响行车安全。因此，如何在DVS成像设备自身移动的情况下滤除背景干扰，有效识别场景中的关键信息，成为亟待本领域技术人员解决的重要课题。
发明内容
本申请提供了一种动态视觉成像传感器和传感装置，以及该传感器和传感装置应用的成像方法。通过对光感电压进行二阶差分，根据二阶差分电压来生成二阶事件信号，使得应用该传感器或传感装置的摄像机在运动时有效减少成像中的背景信息，有效捕获引起光改变速度变化的突发事件。
第一方面，本申请提供一种动态视觉成像传感器。该传感器可以包括光电转换单元，二阶差分单元和比较单元。其中，光电转换单元可以用于将接收到的光信号转换为电信号，生成光感电压；二阶差分单元可以用于对光感电压进行二阶差分，生成二阶差分电压；比较单元可以用于将二阶差分电压与二阶比较阈值进行比较，根据比较结果生成二阶事件信号。传感器通过对光感电压的二阶差分结果进行反应，使得输出的二阶事件信号表征了光线变化速度的变化，即传感器对光线变化的加速度进行响应，从而减少了传感器因自身运动导致的持续而无意义的响应输出，提高了成像质量，减少了总线堵塞。
可选的，二阶差分电压可以包括非零二阶差分电压和零二阶差分电压；前述对光感电压进行二阶差分，生成二阶差分电压可以包括：在当前时刻光感电压的变化速度与之前时刻光感电压的变化速度不同时，二阶差分单元用于生成非零二阶差分电压；在当前时刻光感电压的变化速度与之前时刻光感电压的变化速度相同时，二阶差分单元用于生成零二阶差分电压。这里的变化速度相同，即可以指当前时刻与之前时刻的电压都在变化，且变化的速度相等，也可以指当前时刻与之前时刻的电压都保持不变，即变化速度为0。二阶差分电路通过对当前时刻与之前时刻光感电压变化速度的关系做不同响应，实现了输出反映光变化速度变化的效果。
可选的，二阶比较阈值可以包含正二阶比较阈值和负二阶比较阈值，正二阶比较阈值和负二阶比较阈值可以对称设置，也可以不对称设置，此处不做限定。比较单元可以在二阶差分电压不低于正二阶比较阈值时生成正脉冲信号，可以在二阶差分电压不高于负二阶比较阈值时生成负脉冲信号，可以在二阶差分电压低于正二阶比较阈值高于负二阶比较阈值时生成零脉冲信号。设置正负阈值，并根据二阶差分电压与正负阈值的不同关系输出不同信号，能更准确的反映光感电压变化速率的变化情况，提高传感器的输出精度。
可选的，二阶差分单元可以包括采样保持电路和二阶差分电路。其中，采样保持电路可以用于将光感电压转换为差分输入电压；二阶差分电路可以用于根据差分输入电压对光感电压进行二阶差分，以生成二阶差分电压。当对光感电压进行多阶差分时，差分输入电压通常 需要包含多个电压，这些电压与多个不同时刻的光感电压一一对应。在进行二阶差分时，光感电压通常包括三个电压，分别对应了三个不同时刻的光感电压。二阶差分单元包括采样保持电路使得传感器不需要引入额外的装置来获取三个不同时刻的光感电压来同时输入给二阶差分电路，精简了传感器的结构。
可选的，二阶差分电路可以包括第一一阶差分电路和第二一阶差分电路，并且前述两个差分电路以串联的形式连接。其中，第一一阶差分电路可以用于根据差分输入电压生成第一一阶差分电压和第二一阶差分电压，第二一阶差分电路可以用于根据前述两个一阶差分电压生成二阶差分电压。采用一阶差分电路串联的形式，增加了传感器功能的灵活性，可以将第一一阶差分电路的输出作为最终输出结果的一部分，以获得光感电压的一阶差分数据，使传感器兼具获得一阶事件信号和二阶事件信号的功能。
可选的，二阶差分电路也可以不采用一阶差分电路串联的形式，而是采用整体设计的二阶差分电路。采用整体设计的二阶差分电路降低了电路的复杂度，增加了设备的可靠性。
可选的，第一一阶差分电压或第二一阶差分电压可以被择一发送给比较单元，比较单元还可以用于将接收到的一阶差分电压与一阶比较阈值进行比较，根据比较结果生成一阶事件信号。可选的，第一一阶差分电压和第二一阶差分电压也可以被同时发送给比较单元，比较单元将两个一阶差分电压分别与一阶比较阈值进行比较，根据第一一阶差分电压的比较结果生成第一一阶事件信号，根据第二一阶差分电压的比较结果生成第二一阶事件信号。一阶比较阈值的具体情况请参见前述二阶比较阈值的相关描述，此处不再赘述。将第一一阶差分电路的输出作为输出结果的一部分，以获得光感电压的一阶差分数据，使传感器兼具获得一阶事件信号和二阶事件信号的功能，增加了传感器功能的灵活性。
可选的，传感器还可以包括选通单元，用于在满足预设条件时控制第二一阶差分电路根据第一一阶差分电压和第二一阶差分电压生成二阶差分电压。预设条件可以是判定传感器是否在运动，当判定传感器在运动时将第一一阶差分电压和第二一阶差分电压输入给第二一阶差分电路，以生成二阶差分电压；当判定传感器静止时，不给第二一阶差分电路输入第一一阶差分电压和第二一阶差分电压。根据不同情境选择是否对光进行第二次一阶差分，使得传感器的输出更好的契合场景需求，在传感器静止时输出一阶事件信号保证了输出的内容准确性，在传感器移动时输出二阶事件信号减少了传感器自身运动带来的图像噪声。
可选的，传感器还可以包括读出电路，光电转换单元还用于将光感电压发送给读出电路，读出电路用于对光感电压进行积分，生成积分电压信号。积分电压信号用于生成基于帧的图像，即常规摄像机生成的图像。传感器在输出二阶事件信号或一阶事件信号的同时，还输出积分电压信号，使得传感器的输出能够同时用于生成动态视觉图像和常规图像，进而使将二者融合来提高画质成为了可能。
第二方面，本申请提供一种动态视觉成像传感装置，包括动态视觉传感器阵列和外围电路。其中，动态视觉传感器阵列包括多个传感器，这些传感器是第一方面任意一种可能实施方式所描述的动态视觉传感器。这些动态视觉传感器用于根据接收到的光信号生成二阶事件信号，并将二阶事件信号发送给外围电路。外围电路用于根据二阶事件信号生成地址事件表示。地址事件表示包含了二阶事件信号和生成该二阶事件信号的动态视觉传感器的位置信息。位置信息可以是传感器的坐标信息，也可以是其他任何能表示该传感器位置的信息，此处不做限定。传感装置通过对光感电压的二阶差分结果进行反应，使得输出的地址事件表示表征了光线变化速度的变化，即传感器对光线变化的加速度进行响应，从而减少了传感装置因自 身运动导致的持续而无意义的响应输出，提高了成像质量，防止总线堵塞。
可选的，当动态视觉传感器阵列中的传感器采用将一阶差分电路串联得到二阶差分电路的构造形式时，传感器还可以输出一阶事件信号到外围电路，由外围电路生成基于一阶事件信号的地址事件表示。这样，传感装置既能获得基于一阶事件信号的地址事件表示，也能获得基于二阶事件信号的地址事件表示，增加了传感装置功能的灵活性。此外，根据不同情境来选择输出基于一阶事件信号的地址事件表还是基于二阶事件信号的地址事件表，使得传感装置的输出更好的契合场景需求，在传感装置静止时输出基于一阶事件信号的地址事件表示，保证了输出的内容准确性，在传感器移动时输出基于二阶事件信号的地址事件表示，减少了传感器自身运动带来的图像噪声。
可选的，当动态视觉传感器阵列中的传感器包括读出电路时，传感器还可以输出积分电压信号到外围电路。传感装置在输出地址事件表示的同时，还可以输出用于生成基于帧的图像的积分电压信号，使得传感装置的输出能够同时用于生成动态视觉图像和常规图像，进而使将二者融合来提高画质成为了可能。
第三方面，本申请提供一种动态视觉摄像机，包括如第二方面任意一种实施方式所描述的动态视觉传感装置，还包括工作电路，可以用于根据动态视觉传感装置输出的基于二阶事件信号的地址事件表示生成图像。这样生成的图像，记录了光线变化速度发生变化的像素点的信息，即只记录使光线变化的速度发生变化的关键事件，从而减少了因摄像机自身运动导致的背景噪声，提高了成像质量。
第四方面，本申请提供一种移动终端设备，包括如第三方面所描述的动态视觉摄像机，还包括通信单元，可以用于将生成的图像发送给其他设备。该终端设备生成的图像，记录了光线变化速度发生变化的像素点的信息，即只记录使光线变化的速度发生变化的关键事件，从而减少了因终端设备自身运动导致的背景噪声，提高了成像质量。同时，还可以将图像发送给其他设备，实现了该图像在多设备之间的共享。
第五方面，本申请提供一种即时定位与地图构建系统，包括如第二方面任意一种实施方式所描述的动态视觉传感装置，还包括工作电路，用于根据动态视觉传感装置输出的基于二阶事件信号的地址事件表示进行场景识别。由于传感装置通过对光感电压的二阶差分结果进行反应，使得输出的地址事件表示表征了光线变化速度的变化，即传感器对光线变化的加速度进行响应，从而减少了因系统自身运动导致的背景噪声干扰，提高了物体识别的准确度。可选的，该即时定位与地图构建系统可以实现为头戴式显示设备、无人机、智能路况识别系统等形式。
第六方面，本申请提供一种动态视觉传感方法，包括将光信号转换为电信号生成光感电压；对光感电压进行二阶差分，生成二阶差分电压；将二阶差分电压与二阶比较阈值进行比较，根据比较结果生成二阶事件信号。通过对光感电压的二阶差分结果进行反应，使得输出的二阶事件信号表征了光线变化速度的变化，即输出的二阶事件信号对光线变化的加速度进行响应，从而减少了因获取光信号的设备自身运动而导致的持续而无意义的响应输出，减少噪音，减少了总线堵塞，提高了成像质量。
可选的，二阶比较阈值可以包含正二阶比较阈值和负二阶比较阈值，正二阶比较阈值和负二阶比较阈值可以对称设置，也可以不对称设置，此处不做限定。可以在二阶差分电压不低于正二阶比较阈值时生成正脉冲信号，可以在二阶差分电压不高于负二阶比较阈值时生成负脉冲信号，可以在二阶差分电压低于正二阶比较阈值高于负二阶比较阈值时生成零脉冲信 号。设置正负阈值，并根据二阶差分电压与正负阈值的不同关系输出不同信号，能更准确的反映光感电压变化速率的变化情况，提高输出精度。
可选的，二阶差分电压可以包括非零二阶差分电压和零二阶差分电压；前述对光感电压进行二阶差分，生成二阶差分电压可以包括：在当前时刻光感电压的变化速度与之前时刻光感电压的变化速度不同时，生成非零二阶差分电压；在当前时刻光感电压的变化速度与之前时刻光感电压的变化速度相同时，生成零二阶差分电压。这里的变化速度相同，即可以指当前时刻与之前时刻的电压都在变化，且变化的速度相等，也可以指当前时刻与之前时刻的电压都保持不变，即变化速度为0。二阶差分电路通过对当前时刻与之前时刻光感电压变化速度的关系做不同响应，实现了输出反映光变化速度变化的效果。
可选的，对光感电压进行二阶差分可以包括：首先对光感电压进行一阶差分，生成一阶差分电压，通过对前述一阶差分电压再进行一阶差分，生成二阶差分电压。采用两次一阶差分串联的形式，增加了输出功能的灵活性，可以将一阶差分电压作为输出结果的一部分，以获得光感电压的一阶差分数据，从而同时获得一阶事件信号和二阶事件信号。
可选的，可以将一阶差分电压与一阶比较阈值进行比较，根据比较结果生成一阶事件信号。一阶比较阈值的具体情况请参见前述二阶比较阈值的相关描述，此处不再赘述。将一阶事件信号作为最终输出结果的一部分，以获得光感电压的一阶差分数据，从而能够获得一阶事件信号和/或二阶事件信号，增加了方法的灵活性。
可选的，还可以对光感电压进行积分，生成积分电压信号。积分电压信号用于生成基于帧的图像，即常规摄像机生成的图像。在输出二阶事件信号或一阶事件信号的同时，还输出积分电压信号，使得最终的输出能够同时用于生成动态视觉图像和常规图像，进而使将二者融合来提高画质成为了可能。
附图说明
以下对本申请实施例用到的附图进行介绍。
图1为现有技术DVS相机成像效果对比图；
图2为现有技术DVS传感器结构示意图；
图3为一种动态视觉传感器的逻辑结构示意图；
图4为一种光电转换电路的结构示意图；
图5为一种采样保持电路的结构示意图；
图6为一种采样保持电路时序控制示意图；
图7为一种二阶差分电路的结构示意图；
图8为又一种二阶差分电路的逻辑结构示意图；
图9为又一种二阶差分电路的结构示意图；
图10为又一种动态视觉传感器的逻辑结构示意图；
图11为一种动态视觉传感装置的逻辑结构示意图；
图12为一种动态视觉传感装置的结构示意图；
图13为一种动态视觉传感方法的流程示意图；
图14为又一种动态视觉传感方法的流程示意图；
图15为一种动态视觉摄像机的逻辑结构示意图；
图16为一种移动终端的逻辑结构示意图；
图17为一种即时定位与地图构建系统的逻辑结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述。
请参阅图3，为本发明动态视觉传感器一可能实施例的逻辑结构示意图。动态视觉传感器100包括光电转换单元110、二阶差分单元120和比较单元130，其中，二阶差分单元包括采样保持电路121和二阶差分电路122。在一些可能的实施方式中，动态视觉传感器100还可能包括读出电路140。在一些可能的实施方式中，采样保持电路121也可能被认为是属于光电转换单元110的组成部分。为方便说明，本实施例仅以采样保持电路121属于二阶差分单元120为例进行描述，须知，无论将采样保持电路划归哪一功能单元，此类结构划分方式均属于本申请技术方案所保护的结构范围。
光电转换单元110用于将接收到的光信号转换为电信号，生成对应的光感电压，并发送给二阶差分单元120。当动态视觉传感器100包括读出电路140时，光电转换单元110还会将光感电压发送给读出电路140。光电转换单元110可以由传统的光电转换电路构成，用于将接收到的光信号转换成对应的电信号，即光感电压。光感电压的大小通常与光的强度成线性正比关系。光电转换电路的一种可能实现方式请参见图4。光信号的输入通过光电二极管，经如图所示的典型光电转换电路之后转换为光感电压。光电转换单元110还可以实现为任何其他形式的具有光电转换功能的装置或电路，此处不做限定。
二阶差分单元120用于对光感电压进行二阶差分，生成二阶差分电压。其中，采样保持电路121用于将光感电压转换为差分输入电压，二阶差分电路122用于根据差分输入电压对光感电压进行二阶差分。通常，光感电压是由光电转换单元110输出的一种连续的电压，而二阶差分电路122进行二阶差分则需要多个不同时刻对应的光感电压同时作为二阶差分电路122的输入，这里同时作为二阶差分电路122输入的多个不同时刻对应的光感电压叫做差分输入电压。本实施例中二阶差分电路的差分输入电压以包括三个不同时刻对应的光感电压为例。须知，在一些可能的实施方式中，随着对光感电压的差分阶数的需求不同(比如三阶差分或四阶差分)，差分输入电压可能包含的对应时刻光感电压的数量也会不同，在此不做限定。
采样保持电路121的一种可能实施方式请参见图5。采样保持电路121包括三个采样保持器1211，用于将各自得到的输入电压保持一定时间；三个选择器1212，用于在各自对应的A 1、A 2、A 3三路输入中选择一路进行输出。光感电压的持续输入为U x，差分输入电压包括的三个电压分别为三个选择器的输出电压U 1、U 2和U 3。s1、s2和s3分别为三个采样保持器的时序控制输入，M1和M2为选择器的时序控制输入。图6所示为采样保持电路121可以采用的一种时序控制方式。图6中的时序图展示了时序控制的两个循环周期，在实际应用中这一周期可以在电路的使用过程中持续循环下去。现举例说明采样保持电路的工作原理。当采用如图6所示的时序控制方式时，比如，在初始时刻采样保持器s3及所有开关A 1导通，U 1，U 2，U 3存在初始电压。在时刻1，采样保持器s1及所有开关A 2导通，使得原U 3处电压输出给U 2，原U 2处电压输出给U 1，时刻1的U x输出给U 3；在时刻2，采样保持器s2及所有开关A 3导通，原U 3处电压输出给U 2，原U 2处电压输出给U 1，时刻2的U x输出给U 3；在时刻3，采样保持器s3及所有开关A 1导通，原U 3处电压输出给U 2,原U 2处电压输出给U 1，时刻3的U x输出给U 3，以此类推。在一些可能的实施方式中，采样保持电路也可采取其他任何具有采样保持功能的电 路形式，比如电容、实现延迟相位功能的数字移相器等，此处不做限定。
二阶差分电路122的一种可能实施方式请参见图7。图中U 1、U 2和U 3为差分输入电压，GND为接地点，U 0为二阶差分电压。差分输入电压U 1、U 2、U 3输入给图示的二阶差分电路后，输出二阶差分电压U 0。U 0与U 1、U 2、U 3的关系满足如下等式：
Figure PCTCN2019105187-appb-000003
通常，图示电路中的所有电阻会采用相同阻值(比如10千欧姆)，则此时U 0＝U 3-2U 2+U 1。
二阶差分电路122的另一种可能实施方式请参见图8和图9。图8为该二阶差分电路实施方式的逻辑结构示意图，二阶差分电路122包括第一一阶差分电路1221和第二一阶差分电路1222，第一一阶差分电路和第二一阶差分电路采用串联的方式连接。第一一阶差分电路1221接收差分输入电压，并根据差分输入电压生成第一一阶差分电压和第二一阶差分电压，第二一阶差分电路根据第一一阶差分电压和第二一阶差分电压生成二阶差分电压。图9为该二阶差分电路实施方式的电路图。图中U 1、U 2和U 3为差分输入电压，GND为接地点，U 0为输出的二阶差分电压。第一一阶差分电路1221根据U 1、U 2、U 3生成第一一阶差分电压U A和第二一阶差分电压U B，第二一阶差分电路1222根据U A和U B生成二阶差分电压U 0。U A、U B与U 1、U 2、U 3的关系满足如下公式：
Figure PCTCN2019105187-appb-000004
U 0与U A、U B的关系满足如下公式：
Figure PCTCN2019105187-appb-000005
通常，图示电路中的所有电阻会采用相同阻值(比如10千欧姆)，则此时U 0＝U 3-2U 2+U 1。
由二阶差分电压与差分输入电压的关系可知，若U 3为当前时刻光感电压，则在当前时刻光感电压的变化速度与之前时刻光感电压的变化速度不同时，二阶差分电路122会生成正电压或负电压，即非零二阶差分电压；在当前时刻光感电压的变化速度与之前时刻光感电压的变化速度相同时，则二阶差分电路122的输出电压为0，即生成零二阶差分电压。
比较单元130用于将二阶差分电压与二阶比较阈值进行比较，根据比较结果生成二阶事件信号。比较单元130可以实现为任何能够对电压进行比较并根据比较结果输出信号的装置，比如包括但不限于窗口比较器、专用电压比较芯片等。二阶比较阈值可以包括正二阶比较阈值和负二阶比较阈值，正二阶比较阈值可以设定为正300毫伏，负二阶比较阈值可以设定为负300毫伏。正负阈值也可以不做对称设置，此处不做限定。比较单元130可以在二阶差分电压高于或等于正二阶比较阈值时输出正脉冲，可以在二阶差分电压低于或等于负二阶比较阈值时输出负脉冲，也可以在二阶差分电压低于正二阶比较阈值高于负二阶比较阈值时输出零脉冲。在一些可能的实施方式中，二阶比较阈值也可能只有正阈值，只在二阶差分电压大于正阈值时生成正脉冲；或者只有负阈值，只在二阶差分电压小于负阈值时生成负脉冲，此处不做限定。
读出电路140用于根据接收到的光感电压生成积分电压信号。读出电路会对收到的光感电压进行积分，并根据一段时间内(比如1毫秒)的积分结果输出积分电压信号。读出电路140可以由晶体管构成。读出电路140生成的积分电压信号用于生成基于帧(frame-based)的图像。
在一些可能的实施方式中，当二阶差分电路122采用如图9所示的结构时，第一一阶差分电路1221生成的第一一阶差分电压U A和第二一阶差分电压U B可以直接发送给比较单元130，比较单元130还可以用于将U A和/或U B与一阶差分阈值进行比较，根据比较结果生成一阶事件信号。具体生成一阶事件信号的方式请参见前文生成二阶事件信号方式的表述，以及发明内容中的相关表述，此处不再赘述。
请参阅图10，为本发明动态视觉传感器又一可能实施例的逻辑结构示意图。动态视觉传感器200包括光电转换单元210，二阶差分单元220，一阶比较单元231和二阶比较单元232。其中，二阶差分单元包括采样保持电路221，第一一阶差分电路222，第一选通单元223，第二一阶差分电路224和第二选通单元225。在一些可能的实施方式中，所述动态视觉传感器还可能包括读出电路240。在一些可能的实施方式中，采样保持电路221也可能被认为是属于光电转换单元210的组成部分。为方便说明，本实施例仅以采样保持电路221属于二阶差分单元220为例进行描述，须知，无论将采样保持电路划归哪一功能单元，此类结构划分方式均属于本申请技术方案所保护的结构范围。
光电转换单元210的具体情况请参见图3中对光电转换单元110的描述，此处不再赘述。
二阶差分单元220用于对光感电压进行二阶差分，生成二阶差分电压。其中，采样保持电路221用于将光感电压转换为差分输入电压，通常，光感电压是由光电转换单元210输出的一种连续的电压，而第一一阶差分电路222则需要多个不同时刻对应的光感电压同时作为输入。本实施例中第一一阶差分电路222的差分输入电压以包括三个不同时刻对应的光感电压为例，由采样保持电路221提供，具体提供方式请参见图5和图6中的对应描述，此处不再赘述。须知，在一些可能的实施方式中，随着对光感电压的差分阶数的需求不同，差分输入电压可能包含的对应时刻光感电压的数量也会不同，比如，在三阶差分的情况下，差分输入电压可能包含5个对应时刻的光感电压，在此不做限定。第一一阶差分电路222接收差分输入电压，并根据差分输入电压生成第一一阶差分电压和第二一阶差分电压，第一选通单元223判定是否进行第二次一阶差分，当判定结果为进行第二次一阶差分时，第二一阶差分电路根据第一一阶差分电压和第二一阶差分电压生成二阶差分电压，并且第一一阶差分电压和第二一阶差分电压也被发送至第二选通单元；当判定结果为不进行第二次一阶差分时，则第一一阶差分电压和第二一阶差分电压直接发送给第二选通单元。具体生成第一一阶差分电压、第二一阶差分电压和二阶差分电压的方式请参见图8和图9中的描述，此处不再赘述。第一选通单元223用于控制是否对光感电压进行第二次一阶差分，控制的条件可以是判断相机是否在运动，如果判断为在运动就做第二次一阶差分，判断为静止就不做第二次一阶差分。第二选通单元225用于在一阶差分电压和二阶差分电压都作为输入时，选通后输出一阶差分电压给一阶比较单元231和/或输出二阶差分电压给二阶比较单元232；当只有一阶差分电压或只有二阶差分电压作为输入时，则将收到的电压输出给对应的比较单元。第一选通单元和第二选通单元具体可以通过多路复用器或者矩阵开关等形式来实现，此处不做限定。
一阶比较单元231用于将接收到的一阶差分电压与一阶差分阈值进行比较，根据比较结果生成一阶事件信号。二阶比较单元232用于将接收到的二阶差分电压与二阶比较阈值进行比较，根据比较结果生成二阶事件信号。一阶比较单元231和二阶比较单元232的具体情况请参见图3中对比较单元130的相关描述，此处不再赘述。
读出电路240的具体情况请参阅图3中的相关描述，此处不再赘述。
在一些可能的实施方式中，二阶差分单元220可能包括更多的一阶差分电路和选通单元， 以实现对光感电压的进一步多阶差分，具体实现原理请参阅图10中的对应描述，此处不再赘述。
请参阅图11，为本发明一种动态视觉传感装置一可能实施例的逻辑结构示意图。动态视觉传感装置300包括传感器阵列310和外围电路320。其中，外围电路320包括地址编码器321和第一编码器322。在一些可能的实施方式中，外围电路320还包括第一缓存323。在一些可能的实施方式中，外围电路320还包括第二编码器324和第二缓存325。
传感器阵列310包含多个本申请实施例中记载的任意一种动态视觉传感器。传感器阵列310中的动态视觉传感器用于根据光信号生成二阶事件信号或一阶事件信号，具体生成方式请参见图3至图10中的相关描述，此处不再赘述。这里我们将二阶事件信号和一阶事件信号统称为事件信号。
外围电路320用于根据事件信号生成地址事件表示。具体的，传感器阵列310将事件信号发送给地址编码器321，地址编码器321将生成这一事件信号的传感器的坐标连同事件信号一起发送给第一编码器322。
第一编码器322用于根据收到的事件信号和坐标生成地址事件表示，并对地址事件表示进行编码，生成二进制码信息。在一些可能的实施方式中，外围电路320还包括第一缓存323，用于存储第一编码器322编码的地址事件表示，以供外部设备调用。在一些可能的实施方式中，第一缓存323也可以用于存储地址编码器321输出的信息，此处不做限定。
在一些可能的实施方式中，外围电路320还包括第二编码器324和第二缓存325。第二编码器用于接收和编码传感器阵列发送的积分电压信号，生成二进制码信息，以供外部设备调用。由积分电压信号生成的二进制码信息可以用来生成基于帧的图像。第二缓存325可以用于存储第二编码器324编码生成的信息。
请参阅图12，为本发明一种动态视觉传感装置又一可能实施例的装置结构示意图。动态视觉传感装置400包括传感器阵列410、行地址编码器420、列地址编码器430、第一编码器440和第一缓存450。在一些可能的实施方式中，还可能包括第二编码器460和第二缓存470。
传感器阵列410包含多个本申请实施例中记载的任意一种动态视觉传感器。传感器阵列410中的动态视觉传感器用于根据光信号生成事件信号，具体生成方式请参见图3至图10中的相关描述，此处不再赘述。事件信号包括二阶事件信号和一阶事件信号。传感器阵列410中的传感器会将事件信号分别发送给行地址编码器420和列地址编码器430。
行地址编码器420用于在接收到传感器发送的事件信号后将生成该事件信号的传感器的行坐标连同事件信号一起发送给第一编码器440；列地址编码器430用于在接收到传感器发送的事件信号后将生成该事件信号的传感器的列坐标连同事件信号一起发送给第一编码器440。
第一编码器440用于根据收到的事件信号和坐标生成地址事件表示，并对地址事件表示进行编码，生成二进制码信息。
第一缓存450用于存储编码器编码的地址事件表示，以供外部设备调用。第一缓存450还用于存储列地址编码器430输出的信息。在一些可能的实施方式中，第一缓存450还可以用于存储行地址编码器420输出的信息，此处不做限定。
第二编码器460用于接收和编码传感器阵列发送的积分电压信号，生成二进制码信息，以供外部设备调用。由积分电压信号生成的二进制码信息可以用来生成基于帧的图像。第二缓存470可以用于存储第二编码器460编码生成的信息。
如图11或图12所描述的动态视觉传感装置，或其他包含图3至图10所描述的动态视觉传感器的动态视觉传感装置，在实际使用中可以根据一段时间(比如3毫秒)内累计的事件信号，产生生成这些事件信号的传感器所对应的输出；也可以根据生成事件信号的传感器达到的预设占比时产生输出(例如：某一行或列有超过80％的传感器有事件信号输出，即产生该行或列的脉冲输出)。需要注意的是，这里的事件信号既可以是一阶事件信号，也可以是二阶事件信号。一阶事件信号可以被认为是表征了光线亮度变化的事件信号，二阶事件信号可以被认为是表征了光线亮度变化的速度发生的变化。当传感器阵列中的每一个传感器对应摄像头中的一个像素点时，如果传感器生成的事件信号是二阶事件信号，则摄像头拍摄出来的信息记录了光亮度变化相对于时间的二阶导数图像，即拍摄的画面记录了整个画面中各个像素的光变化的速度的变化量。这样，当摄像机运动时，由于背景的光相对于相机通常是匀速变化的，背景信息将被减少，图像中主要保留了光变化速度有改变的信息(比如突然进入画面的物体)，从而减少了摄像机运动带来的冗余背景信息。具体来说，如果在运动的车辆上安装该摄像机，那么路边静止的树木、停靠的车辆等将可能被滤除，而超车的车辆、落石等可能会被捕获，从而达到更好的拍摄效果。
请参阅图13，为本发明一种动态视觉传感方法的一可能流程示意图。
S510，将光信号转换为电信号，生成光感电压。可以通过光电转换装置将光信号转换为电信号，以作为差分操作的输入。具体方式请参见发明内容及实施例部分中的相关描述，此处不再赘述。
S520，对光感电压进行一阶差分。对光感电压进行一阶差分，得到一阶差分电压，具体方式请参见发明内容及实施例部分中的相关描述，此处不再赘述。
S530，判断是否再进行一阶差分。对S520中得到的一阶差分电压再进行一次一阶差分，就会得到光感电压的二阶差分电压。一阶差分电压表征了光的变化，而二阶差分表征了光变化的速度的变化。可以根据不同的使用场景需求，判断是否进行第二次一阶差分，比如当传感器在运动的时候进行第二次一阶差分，当传感器静止的时候不进行第二次一阶差分。
S540进行第二次一阶差分。当S530判定再进行一次一阶差分时，对S520中得到的一阶差分电压再进行一次一阶差分，具体方式请参见发明内容及实施例部分中的相关描述，此处不再赘述。在一些可能的实施方式中，在S520之后也可能不经过S530的判断而直接进行S540，即不做判断直接对S520生成的一阶差分电压进行第二次一阶差分，此处不做限定。
S550将差分电压与阈值进行比较，并输出比较结果。当接收到S520生成的一阶差分电压时，将一阶差分电压与一阶比较阈值进行比较，根据比较结果生成一阶事件信号；当接收到S540生成的二阶差分电压时，将二阶差分电压与二阶比较阈值进行比较，生成二阶事件信号。具体生成一阶事件信号和二阶事件信号的方法请参见图3中比较单元130的相关描述，此处不再赘述。
如图14为本发明一种动态视觉传感方法的又一可能流程示意图。
S610将光信号转换为电信号，生成光感电压。
S620对光感电压进行二阶差分，生成二阶差分电压。
S630根据二阶差分电压生成二阶事件信号。S610、S620和S630的具体过程请参见图13中的相关描述，此处不再赘述。
S640对光感电压进行积分。
S650根据一段时间的积分结果生成积分电压信号。积分电压信号用于生成基于帧 (frame-based)的图像。
如图15，为应用本发明动态视觉传感装置的动态视觉摄像机一实施例逻辑结构示意图。动态视觉摄像机700包括动态视觉传感装置710和工作电路720。动态视觉传感装置710用于根据光信号生成地址事件表示，具体生成方式请参见发明内容及实施例部分中的具体描述，此处不再赘述。工作电路720用于根据地址事件表示生成图像。工作电路720可以实现为处理器、芯片等形式，此处不做限定。
如图16，为应用本发明动态视觉传感装置的移动终端设备一实施例逻辑结构示意图。移动终端800包括动态视觉摄像机810和通信单元820。动态视觉摄像机810用于根据光信号生成图像，具体生成方式请参见图15，此处不再赘述。通信单元820用于将图像发送给其他设备。通信单元可以是无线信号发射装置，也可以是传输端口等，此处不做限定。
如图17，为应用本发明动态视觉传感装置的即时定位与地图构建系统一实施例逻辑结构示意图。即时定位与地图构建系统900包括动态视觉传感装置910和工作电路920。动态视觉传感装置910用于根据光信号生成地址事件表示，具体生成方式请参见前文具体描述，此处不再赘述。工作电路920用于根据该地址事件表示进行场景识别。
在一个或多个实施例中的工作电路，可以使用专用集成电路(application specific integrated circuit,ASIC)、数字信号处理器(digital signal processor,DSP)、可编程逻辑器件(programmable logic device,PLD)、现场可编程门阵列(field programmable gate array,FPGA)、处理器、控制器、微控制器和/或微处理器等电子单元中的至少一个来实现本申请的实施方式。
在一个或多个实例中，所描述的功能可以硬件、软件、固件或其任何组合来实施。如果以软件实施，则功能可作为一个或多个指令或代码而存储于计算机可读媒体上或经由计算机可读媒体而发送，且通过基于硬件的处理单元执行。
应理解，说明书通篇中提到的“一个实施例”、“一实施例”或“一些可能的实施方式”意味着与实施例有关的特定特征、结构或特性包括在本发明的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”、“在一实施例中”或“在一些可能的实施方式中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。
在本申请所提供的实施例中，应理解，“与A相应的B”表示B与A相关联，根据A可以确定B。但还应理解，根据A确定B并不意味着仅仅根据A确定B，还可以根据A和/或其它信息确定B。
本领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考方法实施例中的对应过程，在此不再赘述。

Claims (18)

  1. 一种动态视觉传感器，其特征在于，所述传感器包括：
    光电转换单元，用于将接收到的光信号转换为电信号，生成光感电压；
    二阶差分单元，用于对所述光感电压进行二阶差分，生成二阶差分电压；
    比较单元，用于将所述二阶差分电压与二阶比较阈值进行比较，根据比较结果生成二阶事件信号。
  2. 如权利要求1所述的动态视觉传感器，其特征在于，所述二阶差分电压包括非零二阶差分电压和零二阶差分电压；所述用于对所述光感电压进行二阶差分，生成二阶差分电压包括：当当前时刻光感电压的变化速度与之前时刻光感电压的变化速度不同时，所述二阶差分单元用于生成所述非零二阶差分电压；当所述当前时刻光感电压的变化速度与所述之前时刻光感电压的变化速度相同时，所述二阶差分单元用于生成所述零二阶差分电压。
  3. 如权利要求1或2所述的动态视觉传感器，其特征在于，所述二阶差分单元包括采样保持电路和二阶差分电路，所述采样保持电路用于将所述光感电压转换为差分输入电压；所述二阶差分电路用于根据所述差分输入电压对所述光感电压进行二阶差分，生成所述二阶差分电压。
  4. 如权利要求3所述的动态视觉传感器，其特征在于，所述差分输入电压包括多个电压，所述多个电压分别与多个不同时刻的所述光感电压一一对应。
  5. 如权利要求3或4所述的动态视觉传感器，其特征在于，所述二阶差分电路包括第一一阶差分电路和第二一阶差分电路，所述第一一阶差分电路与所述第二一阶差分电路串联；所述第一一阶差分电路用于根据所述差分输入电压生成第一一阶差分电压和第二一阶差分电压，所述第二一阶差分电路用于根据所述第一一阶差分电压和所述第二一阶差分电压生成所述二阶差分电压。
  6. 如权利要求5所述的动态视觉传感器，其特征在于，所述比较单元还用于将所述第一一阶差分电压或所述第二一阶差分电压与一阶比较阈值进行比较，根据比较结果生成一阶事件信号。
  7. 如权利要求5所述的动态视觉传感器，其特征在于，所述比较单元还用于将所述第一一阶差分电压和所述第二一阶差分电压分别与一阶比较阈值进行比较，根据所述第一一阶差分电压的比较结果生成第一一阶事件信号，根据所述第二一阶差分电压的比较结果生成第二一阶事件信号。
  8. 如权利要求5、6或7所述的动态视觉传感器，其特征在于，所述二阶差分单元还包括选通单元，所述选通单元用于在满足预设条件时控制所述第二一阶差分电路根据所述第一一阶差分电压和所述第二一阶差分电压生成所述二阶差分电压。
  9. 如权利要求1-8任一项所述的动态视觉传感器，其特征在于，所述传感器还包括读出电路，所述光电转换单元还用于将所述光感电压发送给所述读出电路，所述读出电路用于对所述光感电压进行积分，生成积分电压信号。
  10. 一种动态视觉传感装置，其特征在于，所述传感装置包括动态视觉传感器阵列和外围电路，其中：
    所述动态视觉传感器阵列包括多个如权利要求1-9任一项所述的动态视觉传感器，所述动态视觉传感器用于根据接收到的光信号生成二阶事件信号，并将所述二阶事件信号发送给所述外围电路；
    所述外围电路用于根据所述二阶事件信号生成地址事件表示，所述地址事件表示包含了 所述二阶事件信号和生成所述二阶事件信号的动态视觉传感器的位置信息。
  11. 一种动态视觉摄像机，其特征在于，包含如权利要求10所述的动态视觉传感装置，还包括工作电路，用于根据所述地址事件表示生成图像。
  12. 一种移动终端设备，其特征在于，包含如权利要求11所述的动态视觉摄像机，还包括通信单元，所述通信单元用于将所述图像发送给其他设备。
  13. 一种即时定位与地图构建系统，其特征在于，包含如权利要求10所述的动态视觉传感装置，还包括工作电路，所述工作电路用于根据所述地址事件表示进行场景识别。
  14. 一种动态视觉传感方法，其特征在于：
    将光信号转换为电信号，生成光感电压；
    对所述光感电压进行二阶差分，生成二阶差分电压；
    将所述二阶差分电压与二阶比较阈值进行比较，根据比较结果生成二阶事件信号。
  15. 如权利要求14所述的动态视觉传感方法，其特征在于，所述二阶差分电压包括非零二阶差分电压和零二阶差分电压；所述对所述光感电压进行二阶差分，生成二阶差分电压包括：当当前时刻光感电压的变化速度与之前时刻光感电压的变化速度不同时，生成所述非零二阶差分电压；当所述当前时刻光感电压的变化速度与所述之前时刻光感电压的变化速度相同时，生成所述零二阶差分电压。
  16. 如权利要求14或15所述的动态视觉传感方法，其特征在于，所述对所述光感电压进行二阶差分包括：对所述光感电压进行一阶差分，生成一阶差分电压，再对所述一阶差分电压进行一阶差分，生成所述二阶差分电压。
  17. 如权利要求16所述的动态视觉传感方法，其特征在于，所述方法还包括：
    将所述一阶差分电压与一阶比较阈值进行比较，根据比较结果生成一阶事件信号。
  18. 如权利要求14至17任一项所述的动态视觉传感方法，其特征在于，所述方法还包括：对所述光感电压进行积分，生成积分电压信号。
PCT/CN2019/105187 2018-09-29 2019-09-10 一种动态视觉传感器 WO2020063332A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19864611.9A EP3849168A4 (en) 2018-09-29 2019-09-10 DYNAMIC VISION SENSOR
US17/215,729 US11310445B2 (en) 2018-09-29 2021-03-29 Dynamic vision sensor

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811151126.3 2018-09-29
CN201811151126.3A CN110971792B (zh) 2018-09-29 2018-09-29 一种动态视觉传感器

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/215,729 Continuation US11310445B2 (en) 2018-09-29 2021-03-29 Dynamic vision sensor

Publications (1)

Publication Number Publication Date
WO2020063332A1 true WO2020063332A1 (zh) 2020-04-02

Family

ID=69951032

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/105187 WO2020063332A1 (zh) 2018-09-29 2019-09-10 一种动态视觉传感器

Country Status (4)

Country Link
US (1) US11310445B2 (zh)
EP (1) EP3849168A4 (zh)
CN (1) CN110971792B (zh)
WO (1) WO2020063332A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210158483A1 (en) * 2019-11-26 2021-05-27 Samsung Electronics Co., Ltd. Jointly learning visual motion and confidence from local patches in event cameras
CN114339089A (zh) * 2021-12-29 2022-04-12 深圳锐视智芯科技有限公司 一种事件图像输出方法、装置、设备及可读存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20210000985A (ko) * 2019-06-26 2021-01-06 삼성전자주식회사 비전 센서, 이를 포함하는 이미지 처리 장치 및 비전 센서의 동작 방법
CN116530092A (zh) * 2020-12-31 2023-08-01 华为技术有限公司 一种视觉传感器芯片、操作视觉传感器芯片的方法以及设备
US20240137663A1 (en) * 2021-03-29 2024-04-25 Harman International Industries, Incorporated Method and system for generating dvs frames
CN117677989A (zh) * 2021-07-01 2024-03-08 哈曼国际工业有限公司 用于停车位检测的方法和系统
CN114500868B (zh) * 2022-04-18 2022-07-12 深圳锐视智芯科技有限公司 一种evs像素工作方法及相关装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105163048A (zh) * 2015-09-11 2015-12-16 天津大学 基于放大器复用的动态视觉传感器
CN107071314A (zh) * 2017-05-12 2017-08-18 天津大学 一种时域灵敏度增强型的动态视觉传感器
CN107147856A (zh) * 2017-03-30 2017-09-08 深圳大学 一种像素单元及其去噪方法、动态视觉传感器、成像装置
CN108200362A (zh) * 2017-12-19 2018-06-22 清华大学 基于空间对比度的仿生视网膜摄像电路及子电路
US20180191972A1 (en) * 2016-12-30 2018-07-05 Insightness Ag Dynamic vision sensor architecture

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8625012B2 (en) * 2009-02-05 2014-01-07 The Hong Kong University Of Science And Technology Apparatus and method for improving dynamic range and linearity of CMOS image sensor
CN103458279B (zh) * 2013-08-27 2016-08-10 西北工业大学 一种基于广告视频图像数据的吸引力判定方法
CN104821060A (zh) * 2015-05-19 2015-08-05 深圳市保千里电子有限公司 一种抢劫预警方法及装置
US10368021B2 (en) * 2016-08-26 2019-07-30 Mems Start, Llc Systems and methods for derivative sensing using filtering pixels
CN108600579B (zh) * 2018-05-16 2020-03-20 上海集成电路研发中心有限公司 一种应用于cis的运动检测电路及运动检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105163048A (zh) * 2015-09-11 2015-12-16 天津大学 基于放大器复用的动态视觉传感器
US20180191972A1 (en) * 2016-12-30 2018-07-05 Insightness Ag Dynamic vision sensor architecture
CN107147856A (zh) * 2017-03-30 2017-09-08 深圳大学 一种像素单元及其去噪方法、动态视觉传感器、成像装置
CN107071314A (zh) * 2017-05-12 2017-08-18 天津大学 一种时域灵敏度增强型的动态视觉传感器
CN108200362A (zh) * 2017-12-19 2018-06-22 清华大学 基于空间对比度的仿生视网膜摄像电路及子电路

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3849168A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210158483A1 (en) * 2019-11-26 2021-05-27 Samsung Electronics Co., Ltd. Jointly learning visual motion and confidence from local patches in event cameras
US11694304B2 (en) * 2019-11-26 2023-07-04 Samsung Electronics Co., Ltd. Jointly learning visual motion and confidence from local patches in event cameras
CN114339089A (zh) * 2021-12-29 2022-04-12 深圳锐视智芯科技有限公司 一种事件图像输出方法、装置、设备及可读存储介质

Also Published As

Publication number Publication date
CN110971792B (zh) 2021-08-13
CN110971792A (zh) 2020-04-07
US20210218913A1 (en) 2021-07-15
US11310445B2 (en) 2022-04-19
EP3849168A1 (en) 2021-07-14
EP3849168A4 (en) 2021-11-10

Similar Documents

Publication Publication Date Title
WO2020063332A1 (zh) 一种动态视觉传感器
CN110192387B (zh) 用于基于事件的视觉传感器的数据速率控制
US11765480B2 (en) Pixel collection circuit, dynamic vision sensor, and image capture device
CN111770290A (zh) 一种动态视觉传感器输出事件流降噪方法
CN109005329B (zh) 一种像素单元、图像传感器以及相机
JP4973334B2 (ja) 撮像装置および撮像装置の切り替え方法
CN110198418B (zh) 图像处理方法、装置、存储介质及电子设备
EP3917134A1 (en) Image sensor with a plurality of super-pixels
US20210258525A1 (en) Event array readout control of event-based vision sensing
EP3261331A2 (en) Dual mode image sensor and method of using same
WO2023201907A1 (zh) 一种evs像素工作方法及相关装置
US20230008550A1 (en) Sensor with low power synchronous readout
EP3267678B1 (en) Pixel acquisition circuit, image sensor and image acquisition system
CN113873142B (zh) 多媒体处理芯片、电子设备和动态图像处理方法
TWI789196B (zh) 視訊採集方法和裝置
US11917290B2 (en) Photoelectric conversion device, image pickup apparatus, control method, and storage medium
CN219555081U (zh) 图像传感器中光信号的采集电路和电子设备
JP5383349B2 (ja) 画像データ転送装置
WO2024093240A1 (zh) 亮度变化检测装置及方法、电子设备
US20220394196A1 (en) Sensor data encoding with synchronous readout
WO2023093986A1 (en) A monolithic image sensor, a camera module, an electronic device and a method for operating a camera module
CN113923341A (zh) 电子设备
CN116939390A (zh) 图像传感器、图像信号处理方法、设备及存储介质
JP2022067623A (ja) 光電変換素子、光電変換素子の制御方法、および情報処理装置。
CN116506750A (zh) 一种低功耗图像传感器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19864611

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019864611

Country of ref document: EP

Effective date: 20210408