CN117848493A

CN117848493A - Signal processing device, sensor chip, signal processing method, device, and medium

Info

Publication number: CN117848493A
Application number: CN202410172436.2A
Authority: CN
Inventors: 杨哲宇; 祝夭龙
Original assignee: Beijing Lynxi Technology Co Ltd
Current assignee: Beijing Lynxi Technology Co Ltd
Priority date: 2024-02-07
Filing date: 2024-02-07
Publication date: 2024-04-09
Anticipated expiration: 2044-02-07
Also published as: CN117848493B

Abstract

The disclosure provides a signal processing device, a sensor chip, a signal processing method, equipment and a medium, and belongs to the technical field of computers. The signal processing device includes: the photosensitive module is used for generating an electric signal corresponding to the optical signal incident to the photosensitive module at a preset sampling time; the storage module is used for storing the electric signals of the photosensitive module, wherein the storage module comprises at least two storage sub-modules, and each storage sub-module is used for storing the electric signals of one sampling moment of the photosensitive module; the processing module is used for processing the electric signals stored in the storage module according to a preset signal processing mode to obtain a signal processing result, wherein the signal processing result comprises at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result. The embodiment of the disclosure can be suitable for various signal processing scenes and has higher flexibility.

Description

Signal processing device, sensor chip, signal processing method, device, and medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a signal processing apparatus, a sensor chip, a signal processing method, an electronic device, and a computer readable storage medium.

Background

Currently, most complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) image sensors (CMOS Image Sensor, CIS) are based on the frame sampling principle to capture video, mainly record the absolute value of the light intensity of all pixel points in a pixel array with respect to incident light, and each frame is equally spaced. An Event Camera (Event Camera) is a novel imaging system, which, unlike a CIS, records only a light intensity variation of incident light at a corresponding position per pixel of the Event Camera, and outputs a positive or negative pulse only when the light intensity variation exceeds a certain threshold.

Disclosure of Invention

The present disclosure provides a signal processing apparatus, a sensor chip, a signal processing method, an electronic device, and a computer-readable storage medium.

In a first aspect, the present disclosure provides a signal processing apparatus comprising: the device comprises a photosensitive module, a sampling module and a display module, wherein the photosensitive module is used for generating an electric signal corresponding to an optical signal incident to the photosensitive module at a preset sampling moment; the storage module is used for storing the electric signals of the photosensitive modules, wherein the storage module comprises at least two storage sub-modules, each storage sub-module is used for storing the electric signals of one sampling moment of the photosensitive modules, and the electric signals stored for any sampling moment of each photosensitive module comprise at least one of the following: the electric signal of the photosensitive module at the sampling moment, the electric signal of the photosensitive module at the sampling moment and at least one historical sampling moment; the processing module is used for processing the electric signals stored in the storage module according to a preset signal processing mode to obtain a signal processing result, wherein the signal processing result comprises at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result.

In a second aspect, the present disclosure provides a sensor chip comprising at least one signal processing device; wherein the signal processing device adopts the signal processing device according to any one of the embodiments of the present disclosure.

In a third aspect, the present disclosure provides a signal processing method, including: determining a signal processing mode according to the received mode setting instruction; processing the electric signals of the photosensitive modules stored in the storage module according to the signal processing mode to obtain a signal processing result; wherein the signal processing device adopts the signal processing device according to any one of the embodiments of the present disclosure.

In a fourth aspect, the present application provides an electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executable by the at least one processor to enable the at least one processor to perform the signal processing method of any one of the embodiments of the present disclosure.

In a fifth aspect, the present application provides a computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor/processing core implements the signal processing method of any of the embodiments of the present disclosure.

In the embodiment provided by the disclosure, in a signal processing device, a photosensitive module is used for generating an electric signal corresponding to an optical signal incident to the photosensitive module at a preset sampling time; the storage module is used for storing the electric signals of the photosensitive modules, wherein the storage module comprises at least two storage sub-modules, each storage sub-module is used for storing the electric signals of one sampling moment of the photosensitive modules, and the electric signals stored for any sampling moment of each photosensitive module comprise at least one of the following: the method comprises the steps of detecting an electric signal of a photosensitive module at a sampling moment, detecting the electric signal of the photosensitive module at the sampling moment and detecting at least one historical sampling moment; the processing module is used for processing the electric signals stored in the storage module according to a preset signal processing mode to obtain a signal processing result, wherein the signal processing result comprises at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result. Therefore, the electric signals generated by the photosensitive module can be stored in the storage module, and the electric signals at one sampling moment can be respectively stored in at least two storage sub-modules of the storage module, so that a data basis is provided for subsequent signal processing; further, the signal processing device supports a plurality of preset signal processing modes, so that when processing is performed based on the stored electrical signals, a proper signal processing mode can be selected from the stored electrical signals, and further, the stored electrical signals are processed based on the signal processing mode, so that at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result is obtained. That is, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing apparatus may multiplex the photosensitive module to implement multiple signal processing modes, thereby being capable of adapting to various signal processing scenarios, and having higher flexibility.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure, without limitation to the disclosure. The above and other features and advantages will become more readily apparent to those skilled in the art by describing the detailed exemplary embodiments with reference to the accompanying drawings, which are shown below.

Fig. 1 is a block diagram of a signal processing apparatus according to an embodiment of the present disclosure.

Fig. 2 is a schematic diagram of a signal processing apparatus according to an embodiment of the disclosure.

Fig. 3 is a schematic diagram of a signal processing apparatus according to an embodiment of the disclosure.

Fig. 4 is a schematic diagram of a processing procedure of a signal processing apparatus according to an embodiment of the disclosure.

Fig. 5 is a schematic diagram of a processing procedure of a signal processing apparatus according to an embodiment of the disclosure.

Fig. 6 is a schematic diagram of a sensor chip according to an embodiment of the disclosure.

Fig. 7 is a flowchart of a signal processing method according to an embodiment of the present disclosure.

Fig. 8 is a block diagram of an electronic device according to an embodiment of the present disclosure.

Fig. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

For a better understanding of the technical solutions of the present disclosure, exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Embodiments of the disclosure and features of embodiments may be combined with each other without conflict.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In the related art, the image sensor supports fewer processing modes, and the corresponding processing modes cannot be freely configured for the pixels according to requirements, so that the application scene of the image sensor is limited.

In view of this, embodiments of the present disclosure provide a signal processing apparatus, a sensor chip, a signal processing method, an electronic device, and a computer-readable storage medium.

According to the embodiment provided by the disclosure, the electric signals generated by the photosensitive module can be stored in the storage module, and the electric signals at one sampling moment can be respectively stored in at least two storage sub-modules of the storage module, so that a data basis is provided for subsequent signal processing; further, the signal processing device supports a plurality of preset signal processing modes, so that when processing is performed based on the stored electrical signals, a proper signal processing mode can be selected from the stored electrical signals, and further, the stored electrical signals are processed based on the signal processing mode, so that at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result is obtained. That is, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing apparatus may multiplex the photosensitive module to implement multiple signal processing modes, thereby being capable of adapting to various signal processing scenarios, and having higher flexibility.

A first aspect of an embodiment of the present disclosure provides a signal processing apparatus.

Fig. 1 is a block diagram of a signal processing apparatus according to an embodiment of the present disclosure. Referring to fig. 1, the signal processing apparatus 100 may include the following modules.

The photosensitive module 110 is configured to generate an electrical signal corresponding to an optical signal incident on the photosensitive module at a preset sampling time.

The storage module 120 is configured to store an electrical signal of the photosensitive module, where the storage module includes at least two storage sub-modules, each storage sub-module is configured to store an electrical signal of one sampling time of the photosensitive module, and the electrical signal stored for each photosensitive module about any sampling time includes at least one of the following: the light sensing module is used for sensing the electric signal at the sampling time, and sensing the electric signal at the sampling time and at least one historical sampling time.

The processing module 130 is configured to process the electrical signal stored in the storage module according to a preset signal processing mode, so as to obtain a signal processing result, where the signal processing result includes at least one of a direct transmission result, a time difference result, a space difference result, and a high-order difference result.

In the embodiment of the disclosure, the photosensitive module is used for generating an electric signal corresponding to an optical signal incident to the photosensitive module at a preset sampling time; the storage module is used for storing the electric signals of the photosensitive modules, wherein the storage module comprises at least two storage sub-modules, each storage sub-module is used for storing the electric signals of one sampling moment of the photosensitive modules, and the electric signals stored for any sampling moment of each photosensitive module comprise at least one of the following: the method comprises the steps of detecting an electric signal of a photosensitive module at a sampling moment, detecting the electric signal of the photosensitive module at the sampling moment and detecting at least one historical sampling moment; the processing module is used for processing the electric signals stored in the storage module according to a preset signal processing mode to obtain a signal processing result, wherein the signal processing result comprises at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result. Therefore, the electric signals generated by the photosensitive module can be stored in the storage module, and the electric signals at one sampling moment can be respectively stored in at least two storage sub-modules of the storage module, so that a data basis is provided for subsequent signal processing; further, the signal processing device supports a plurality of preset signal processing modes; therefore, when processing is performed based on the stored electrical signals, a proper signal processing mode can be selected from the stored electrical signals, and then the stored electrical signals are processed based on the signal processing mode, so that at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result is obtained. That is, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing apparatus may multiplex the photosensitive module to implement multiple signal processing modes, thereby being capable of adapting to various signal processing scenarios, and having higher flexibility.

In some alternative implementations, the photosensitive module is a functional module with photoelectric conversion capability, and can convert an optical signal incident on the photosensitive module into an electrical signal that is more easily processed by an electronic device.

In some alternative implementations, the photosensitive module may sample and convert the incident optical signal according to a preset sampling period, so as to generate an electrical signal corresponding to a preset sampling time.

For example, if the preset sampling period is T, the photosensitive module may generate an electrical signal at a sampling time T0, a sampling time T1, sampling times T2, … …, and a sampling time tn. Wherein n is an integer greater than or equal to 1, and the time difference between two adjacent sampling moments is T.

In some optional implementations, the electrical signal includes an induction signal, the induction signal is a signal generated by the photosensitive module through photoelectric induction of the optical signal, and the induction signal is a signal including a spatial dimension and a temporal dimension; or the electric signal comprises a sensing signal and a reset signal corresponding to the sensing signal, wherein the reset signal is used for resetting the sensing signal.

It follows that the composition of the electrical signal comprises two cases, the first case where the electrical signal comprises only the sensing signal and the second case where the electrical signal comprises the sensing signal and the corresponding reset signal. The spatial dimension of the sensing signal may be determined by the position information of the pixel module that generates the sensing signal, and the temporal dimension of the sensing signal may be determined by the sampling time of the sensing signal. For example, the sensing signal may be represented as S (x, y, t), where x and y represent a horizontal position and a vertical position of the photosensitive module generating the sensing signal, respectively, and t represents a sampling time corresponding to the sensing signal.

In some alternative implementations, in the signal processing apparatus, the number of photosensitive modules may be plural, and the plural photosensitive modules may be arranged in an array form, so as to form a corresponding photosensitive array.

In some alternative implementations, the plurality of photosensitive modules may correspond to the same type of photosensitive module, or may correspond to different types of photosensitive modules, which is not limited by embodiments of the present disclosure.

Illustratively, the plurality of photosensitive modules is at least one of: the plurality of photosensitive modules are all color photosensitive modules; the plurality of photosensitive modules are gray photosensitive modules; the partial photosensitive modules are color photosensitive modules, and the partial photosensitive modules are gray photosensitive modules; the color photosensitive module is a photosensitive module for collecting color information, and the gray photosensitive module is a photosensitive module for collecting brightness information.

Illustratively, the color sensing module includes an RGB sensing module and/or a YUV sensing module that generates an electrical signal that is a signal having color information. Where "RGB" represents Red, green, and Blue, respectively, "Y" represents brightness (Luminance or Luma), and "U" and "V" represent chromaticity (Chroma or Chroma).

In some alternative implementations, for the portion of the photosensitive modules that are color photosensitive modules and the portion of the photosensitive modules that are photosensitive arrays of grayscale photosensitive modules, the color photosensitive modules and grayscale photosensitive modules may be arranged at equal intervals, may be arranged at unequal intervals, or may be arranged in a region of one array in a centralized manner, while the grayscale photosensitive modules are arranged in a region of another array in a centralized manner.

In some alternative implementations, the grayscale photosensitive module may be regarded as a dynamic sensor (Dynamic Vision Sensor, DVS) photosensitive module, which has higher sensitivity to light intensity than the color photosensitive module, and may collect a light intensity signal with higher light intensity and higher accuracy.

In some alternative implementations, the storage module is a functional module for storing the electrical signals of the photosensitive module.

In some alternative implementations, the storage module includes at least two storage sub-modules, each of which may store an electrical signal at one sampling instant of the photosensitive module, the stored electrical signal for each photosensitive module for any sampling instant including at least one of: the light sensing module is used for sensing the electric signal at the sampling time, and sensing the electric signal at the sampling time and at least one historical sampling time. Wherein the historical sampling time is the sampling time before the current sampling time.

In some alternative implementations, the storage module includes two storage sub-modules; wherein, at the j-1 sampling moment, the two storage sub-modules respectively store the electrical signal at the j-1 sampling moment and the electrical signal at the j-2 sampling moment; at the j-th sampling moment, a storage sub-module for storing the electrical signals at the j-2 th sampling moment is used as a target storage sub-module, the electrical signals at the j-th sampling moment are stored in the target storage sub-module, the electrical signals at the j-1 th sampling moment stored in the other storage sub-module are reserved, j is an integer, and j is more than 2.

In the 1 st sampling time, only the electric signal at the 1 st sampling time may be stored, and the electric signals at the 1 st sampling time and the 2 nd sampling time may be obtained as long as the electric signal at the 2 nd sampling time is entered, so that the differential operation in the time dimension can be supported. In addition, the electrical signals of different photosensitive modules may be differentiated in the spatial dimension, and may be differentiated in the time dimension and the spatial dimension (i.e., a high-order difference), which is not limited by the embodiments of the present disclosure.

In some optional implementations, the number of photosensitive modules is plural, and there is a correspondence between the photosensitive modules and the storage module, where the storage module is configured to store the electrical signals of the photosensitive modules with the correspondence.

In some alternative implementations, the storage module may be a shared functional module that may provide storage support for at least two photosensitive modules. In other words, the correspondence between the photosensitive modules and the storage modules is a many-to-one correspondence, that is, a plurality of photosensitive modules corresponds to one storage module.

It should be noted that, for the shared storage module, the electrical signals of the plurality of photosensitive modules stored in the storage module may be distinguished by carrying the photosensitive module identifier in the electrical signals, or setting an independent storage space for each corresponding photosensitive module in the storage module, or the like.

For example, it may be determined which photosensitive modules share the same memory module according to their locations. For example, a plurality of photosensitive modules in the same row (column) share one memory module, and/or a plurality of photosensitive modules in adjacent L rows (column) share one memory module, and/or a plurality of photosensitive modules in a certain preset area range share one memory module, where L > 1, and the preset area range may be a regular shape or an irregular shape area range.

For example, which photosensitive modules share the same storage module may be determined according to the type of the photosensitive module. For example, the color sensing modules share the same memory module (or share multiple memory modules), and the grayscale sensing modules share the same memory module (or share multiple memory modules).

It is also possible to determine which photosensitive modules share the same memory module, for example, together based on the location and type of the photosensitive modules. For example, a plurality of color photosensitive modules within a certain preset area share one storage module, and a plurality of gray scale photosensitive modules within another preset area share one storage module.

In some alternative implementations, the photosensitive modules may be in a one-to-one correspondence with the storage modules, i.e., each photosensitive module has a dedicated storage module in which only the electrical signals of the corresponding photosensitive module are stored, but not the electrical signals of the other photosensitive modules.

In some alternative implementations, the memory module may include at least two memory sub-modules; correspondingly, the processing module is further used for selecting the target storage sub-module from the storage modules with corresponding relations with the photosensitive modules under the condition that the photosensitive modules generate the electric signals at the current sampling time, and storing the electric signals at the current sampling time into the target storage sub-module.

It should be noted that if the electrical signal at the current sampling time needs to be overlaid with the electrical signal at a certain (or several) historical sampling time when being stored, before overlaying, it needs to be ensured that the electrical signal at the historical sampling time to be overlaid has been read out, or has been restored, or has been used, or is not used any more, so that the situation that the corresponding signal processing cannot be performed due to the data overlaying is avoided.

In some alternative implementations, the storage sub-module may include at least one storage unit, and each storage unit is provided with a switch; accordingly, the processing module may store the electrical signal into a certain memory cell by gating the switch of that memory cell.

The switch may be implemented by a transistor, for example, by turning on or off the transistor by adjusting the voltage between the base and emitter of the transistor, etc.

Fig. 2 is a schematic diagram of a signal processing apparatus according to an embodiment of the disclosure. Referring to fig. 2, the signal processing apparatus 100 includes m photosensitive modules, each corresponding to one memory module, and each including a plurality of memory sub-modules (not shown in the drawing) corresponding to n memory cells, and a switch is provided for each memory cell, and it is possible to determine whether data can be written to or read from the memory cell by closing and opening the switch.

As shown in fig. 2, the photosensitive module 1101 corresponds to the storage module 1201, and the storage module 1201 includes a storage unit 12011, storage units 12012, … …, and a storage unit 1201n; the photosensitive module 1102 corresponds to the storage module 1202, and the storage module 1202 includes a storage unit 12021, storage units 12022, … …, and a storage unit 1202n; … …; the photosensitive module 110m corresponds to the storage module 120m, and the storage module 120m includes a storage unit 120m1, storage units 120m2, … …, and a storage unit 120mn. For any one of the photosensitive modules, for the electrical signal generated at the sampling time ti, the processing module 130 selects at least one target storage unit from the contents stored in each storage unit of the storage module corresponding to the photosensitive module, gates the switch of the target storage unit, and writes the electrical signal Si corresponding to ti into the target storage unit. In addition, the switches of other memory cells in the memory module are in the closed state except the target memory cell, so that the data stored in the memory cells are kept and cannot be lost. After determining what signal processing mode to use, the processing module may determine which electrical signals need to be used, so that the portion of the electrical signals may be read from the memory module and corresponding signal processing may be performed based on the read electrical signals.

In some alternative implementations, the memory cells may be fabricated by capacitive devices, and multiple memory cells may be stacked, thereby reducing the occupied chip area and improving the area utilization of the chip.

In some alternative implementations, the photosensitive module is composed of a photodiode (Pinned Photodiode Pixel, PPD) with a plurality of transistors, the memory module (or cell) is composed of a plurality of capacitors, and the switches of the cells are also made up of transistors. Among them, PPD is a semiconductor device built based on a photoelectric conversion function. An initial electric signal generated by the PPD through photoelectric conversion and passing through the transmission tube M _TG Gain adjusting tube M _DCG Reset tube M _RST Source follower M _SF After a series of processes of (a) a final electrical signal is obtained by the action of a Rolling Shutter (Rolling Shutter).

The memory module may include four memory cells corresponding to the capacitor C1, the capacitor C2, the capacitor C3 and the capacitor C4, respectively, and the four capacitors pass through the gate tube M _SEL1 Gate tube M _SEL2 Gate tube M _SEL3 And gate tube M _SEL4 The switch state is controlled.

After the electrical signal is obtained, the processing module stores the electrical signal into the gated capacitor by gating the gate of at least one capacitor. The capacitor is not gated or the gate tube is in a closed state, and the charge of the capacitor is not moved, so that the information in the capacitor is unchanged, and the stored historical information (such as an electric signal at the time of historical sampling) is reserved.

In some alternative implementations, multiple pixel module arrays may be arranged to form a single pixel array. Each pixel module comprises a photosensitive module and a corresponding storage module thereof, and each photosensitive module and the corresponding storage module thereof are integrated together, so that the storage of electric signals is facilitated. The pixel array comprises a pixel array, a pixel module, a storage module circuit and a control module, wherein the pixel array comprises a pixel module, a light sensing module circuit and a storage module circuit, the light sensing module circuit consists of PPD and a plurality of transistors, the storage module circuit consists of a plurality of capacitors, a plurality of gate tubes and other devices (for example, four capacitors and four gate tubes), and one capacitor and one gate tube can be regarded as one storage unit.

Fig. 3 is a schematic diagram of a signal processing apparatus according to an embodiment of the disclosure. Referring to fig. 3, the signal processing apparatus 100 includes m photosensitive modules (including photosensitive modules 1101, … …,110k, … …,110q, … …,110 m) and the m photosensitive modules are arranged in an array of q×k form to form an array, wherein q, k, and m are integers greater than 1, and q < m, k < m. The m photosensitive modules share the storage module 120, and the processing module 130 may process the electrical signal stored in the storage module 120 according to a preset signal processing mode, so as to obtain a signal processing result corresponding to the signal processing mode.

In some alternative implementations, the electrical signal generated by the photosensitive modules includes an induction signal, each photosensitive module corresponds to one storage module, and each storage module includes two storage units, namely a first storage unit and a second storage unit, and then, for each sampling time (for example, t1, t2, t3, t4, etc.), information stored by one storage module is shown in table 1.

Table 1 stored information schematic table of memory module

As can be seen from table 1, for any photosensitive module, the sensing signal S1 is generated at the sampling time t1, and since both memory cells of the memory module corresponding to the photosensitive module can store data, any switch may be turned on to store S1 in the turned-on memory cell (the memory cell is the target memory cell). As shown in table 1, if the switch of the first memory cell is selected to be turned on, S1 may be stored in the first memory cell, and at this time, the first memory cell stores S1 and the second memory cell does not store data.

For the sampling time t2, before the corresponding sensing signal S2 is not stored, since the first storage unit stores S1 and the second storage unit does not store data, the second storage unit is determined as a target storage unit, and the switch of the second storage unit is turned on to store S2 into the second storage unit, and at the same time, the switch of the first storage unit is turned off to retain S1 stored in the first storage unit.

For the sampling time t3, before the corresponding sensing signal S3 is not stored, the first storage unit stores S1, the second storage unit stores S2, and since the history sampling time of S1 is earlier than the history sampling time of S2, the first storage unit storing S1 is selected as the target storage unit, the switch of the first storage unit is turned on to store S3 into the first storage unit, and the switch of the second storage unit is turned off to retain S2 stored in the second storage unit.

For the sampling time t4, before the corresponding sensing signal S4 is not stored, the first storage unit stores S3, the second storage unit stores S2, and since the history sampling time of S2 is earlier than the history sampling time of S3, the second storage unit storing S2 is selected as the target storage unit, the switch of the second storage unit is turned on to store S4 into the second storage unit, and the switch of the first storage unit is turned off to retain S3 stored in the first storage unit.

Similarly, the sensing signals stored by the storage module with respect to each sampling time can be obtained for subsequent signal processing.

Table 2 shows information stored for each sampling timing (e.g., t1, t2, t3, t4, etc.) when the memory module includes three memory cells (a first memory cell, a second memory cell, and a third memory cell).

Table 2 stored information schematic table of memory module

As shown in table 2, since the sensing signal S1 is generated at the sampling time t1, all of the three memory cells can store data, any one of the switches may be turned on to store S1 in the turned-on memory cell (i.e., the target memory cell). As shown in table 2, if the switch of the first memory cell is selected to be turned on, S1 may be stored in the first memory cell, and at this time, the first memory cell stores S1, and the second memory cell and the third memory cell do not store data.

For the sampling time t2, before the corresponding sensing signal S2 is not stored, since the first storage unit stores S1 and the second storage unit and the third storage unit do not store data, the second storage unit may be determined as a target storage unit, and the switch of the second storage unit may be turned on to store S2 into the second storage unit, while the switch of the first storage unit is turned off to retain S1 stored in the first storage unit.

For the sampling time t3, before the corresponding sensing signal S3 is not stored, the first storage unit stores S1, the second storage unit stores S2, and the third storage unit does not store data, therefore, the third storage unit is selected as a target storage unit, the switch of the third storage unit is turned on to store S3 into the first storage unit, and the switches of the first storage unit and the second storage unit are turned off at the same time to retain S1 stored in the first storage unit and S2 stored in the second storage unit.

For the sampling time t4, before the corresponding sensing signal S4 is not stored, the first storage unit stores S1, the second storage unit stores S2, and the third storage unit stores S3, and since the history sampling time of S1 is earlier than the history sampling times of S2 and S3, the first storage unit storing S1 is selected as a target storage unit, the switch of the first storage unit is turned on, S4 is stored in the first storage unit, and the switches of the second storage unit and the third storage unit are turned off at the same time, so that S2 stored in the second storage unit and S3 stored in the third storage unit are retained.

In some optional implementations, each photosensitive module corresponds to one storage module, and the storage modules include two storage sub-modules, the first storage sub-module including a first storage unit and a second storage unit, the second storage sub-module including a third storage unit and a fourth storage unit; correspondingly, the processing module is further configured to: for the ith sampling moment, under the condition that the first storage unit and the second storage unit respectively store the induction signal and the reset signal of the ith-1 sampling moment, and the third storage unit and the fourth storage unit respectively store the induction signal and the reset signal of the ith-2 sampling moment, the switches of the first storage unit and the second storage unit are turned off, and the switches of the third storage unit and the fourth storage unit are turned on, so that the third storage unit stores the induction signal of the ith sampling moment, and the fourth storage unit stores the reset signal corresponding to the induction signal of the ith sampling moment, wherein i is an integer and i >2.

Table 3 shows information stored for each sampling timing (e.g., t1, t2, t3, t4, etc.) when the memory module includes four memory cells (first memory cell, second memory cell, third memory cell, and fourth memory cell).

Table 3 stored information schematic table of memory module

As shown in table 3, the sensing signal S1 and the reset signal R1 are generated at the sampling time t1, and since data can be stored in each of the four memory cells, the switches of any two of the memory cells are turned on to store S1 and R1 in each of the two memory cells (the memory cell is the target memory cell). As shown in table 3, if the switch of the first memory cell and the second memory cell is selected to be turned on, S1 may be stored in the first memory cell and R1 may be stored in the second memory cell, and at this time, the first memory cell stores S1, the second memory cell stores R1, and the third memory cell and the fourth memory cell do not store data.

For the sampling time t2, before the corresponding sensing signal S2 and reset signal R2 are not stored, since the first storage unit stores S1, the second storage unit stores R1, and the third storage unit and the fourth storage unit do not store data, it is possible to determine the third storage unit and the fourth storage unit as target storage units, and gate the switch of the third storage unit to store S2 into the third storage unit, gate the switch of the fourth storage unit to store R2 into the fourth storage unit, and simultaneously turn off the switches of the first storage unit and the second storage unit to preserve S1 stored in the first storage unit and R1 stored in the second storage unit.

For the sampling time t3, before the corresponding sensing signal S3 and the reset signal R3 are not stored, since the first storage unit stores S1, the second storage unit stores R1, the third storage unit stores S2, the fourth storage unit stores R2, and S3 and R3 need to occupy two storage units, the historical sampling times of S1 and R1 are earlier than the historical sampling times of S2 and R2, the first storage unit and the second storage unit may be determined as the target storage unit, and the switch of the first storage unit is turned on to store S3 into the first storage unit, the switch of the second storage unit is turned on to store R3 into the second storage unit, and the switch of the third storage unit and the switch of the fourth storage unit are turned off to retain S2 stored in the third storage unit and R2 stored in the fourth storage unit.

For the sampling time t4, before the corresponding sensing signal S4 and the reset signal R4 are not stored, since the first storage unit stores S3, the second storage unit stores R3, the third storage unit stores S2, the fourth storage unit stores R2, and S4 and R4 need to occupy two storage units, and the historical sampling times of S2 and R2 are earlier than the historical sampling times of S3 and R3, the third storage unit and the fourth storage unit may be determined as the target storage unit, and the switch of the third storage unit is turned on to store S4 into the third storage unit, the switch of the fourth storage unit is turned on to store R4 into the fourth storage unit, and the switch of the first storage unit and the switch of the second storage unit are turned off to retain S3 stored in the first storage unit and R3 stored in the second storage unit.

In some optional implementations, the pixel module further includes a gain adjustment unit, and the gain adjustment unit is configured to adjust the conversion gain according to the ambient brightness of the pixel module and generate the sensing signal corresponding to the conversion gain.

In some optional implementations, the gain adjustment unit is configured to generate the nth gain sense signal based on the nth conversion gain when the ambient brightness is greater than the nth-1 brightness threshold and less than or equal to the nth brightness threshold, where n is greater than or equal to 2, and generate the 1 st gain sense signal based on the 1 st conversion gain when the ambient brightness is less than the 1 st brightness threshold.

In some optional implementations, the conversion gain corresponding to the photosensitive module includes a first gain and a second gain; the sensing signal comprises a first gain sensing signal corresponding to the first gain and/or a second gain sensing signal corresponding to the second gain, and the reset signal comprises a first gain reset signal corresponding to the first gain and/or a second gain reset signal corresponding to the second gain.

It should be noted that, by setting different conversion gains, the readout noise of the signal can be reduced in a dark light scene, the signal to noise ratio can be improved, and the charge capacity can be increased and the saturation can be reduced in a bright light scene.

Table 4 shows information stored for each sampling timing (e.g., t1, t2, t3, t4, etc.) in the case where the memory module includes four memory units (first, second, third, and fourth memory units) and the gain adjustment unit corresponds to the first and second gains.

Table 4 stored information schematic table of memory module

For the sampling time t1, the photosensitive module obtains a corresponding sensing signal HS1 and a reset signal HR1 corresponding to HS1 based on the first gain, and the photosensitive module obtains a corresponding sensing signal LS1 and a reset signal LR1 corresponding to LS1 based on the second gain. And since the previous four memory cells can all store data, the switches of the four memory cells can be turned on to store HS1 in the first memory cell, HR1 in the second memory cell, LS1 in the third memory cell, and LR1 in the fourth memory cell.

For the sampling time t2, the gain adjustment unit determines a conversion gain to be used according to the ambient brightness, and generates a sense signal and a reset signal corresponding to the sampling time t2 based on the determined conversion gain. Further, if it is determined that the second gain is adopted, the sense signal LS2 and the reset signal LR2 corresponding to t2 may be generated. Since sampling timings of signals stored in the first to fourth memory cells are the same before LS2 and LR2 are not stored, the first and second memory cells are randomly selected as target memory cells, and the switch of the first memory cell is turned on to store LS2 in the first memory cell, the switch of the second memory cell is turned on to store LR2 in the second memory cell, and the switches of the third and fourth memory cells are turned off to retain LS1 stored in the third memory cell and LR1 stored in the fourth memory cell.

For the sampling time t3, the gain adjustment unit determines a conversion gain to be used according to the ambient brightness, and generates a sense signal and a reset signal corresponding to the sampling time t3 based on the determined conversion gain. Further, if it is determined that the second gain is adopted, the sense signal LS3 and the reset signal LR3 corresponding to t3 may be generated. Because the first storage unit and the second storage unit store LS2 and LR2 respectively before LS3 and LR3 are not stored, the third storage unit and the fourth storage unit store LS1 and LR1 respectively, and the history sampling time corresponding to LS1 and LR1 is earlier than the history sampling time corresponding to LS2 and LR2, the third storage unit and the fourth storage unit are selected as the target storage unit, and the switch of the third storage unit is turned on to store LS3 to the third storage unit, the switch of the fourth storage unit is turned on to store LR3 to the fourth storage unit, and the switch of the first storage unit and the switch of the second storage unit are turned off to retain LR2 stored in the LS2 and the second storage unit stored in the first storage unit.

For the sampling time t4, the gain adjustment unit determines a conversion gain to be used according to the ambient brightness, and generates a sense signal and a reset signal corresponding to the sampling time t4 based on the determined conversion gain. Further, if it is determined that the first gain is adopted, the sense signal HS4 and the reset signal HR4 corresponding to t4 may be generated. Because the first storage unit and the second storage unit store LS2 and LR2 respectively before HS4 and HR4 are not stored, the third storage unit and the fourth storage unit store LS3 and LR3 respectively, and the history sampling time corresponding to LS2 and LR2 is earlier than the history sampling time corresponding to LS3 and LR3, the first storage unit and the second storage unit are selected as target storage units, and the switch of the first storage unit is turned on to store HS4 to the first storage unit, the switch of the second storage unit is turned on to store HR4 to the second storage unit, and the switch of the third storage unit and the switch of the fourth storage unit are turned off to retain LR3 stored in the LS3 and the fourth storage unit stored in the third storage unit.

In some alternative implementations, the conversion gain may be adjusted once every interval period, instead of before every sampling instant, taking into account that there is some continuity of the ambient light.

For example, the reset operation of the conversion gain may be performed after every preset reset period, and signal sampling may be performed at a new sampling timing based on the reset conversion gain. For example, the sampling period corresponding to the sampling time is T1, the reset period is T2, and t2=n×t1, N > 1.

Table 5 shows information stored for each sampling timing (e.g., t1, t2, t3, t4, etc.) in the case where the memory module includes four memory cells (first, second, third, and fourth memory cells) and the gain adjustment unit performs conversion gain reset based on a preset reset period.

Table 5 stored information schematic table of memory module

As shown in table 5, the reset of the conversion gain is performed every 30 sampling moments. At time t1, the photosensitive module obtains a corresponding sensing signal HS1 and a reset signal HR1 corresponding to HS1 based on the first gain, and the photosensitive module obtains a corresponding sensing signal LS1 and a reset signal LR1 corresponding to LS1 based on the second gain. And since the previous four memory cells can all store data, the switches of the four memory cells can be turned on to store HS1 in the first memory cell, HR1 in the second memory cell, LS1 in the third memory cell, and LR1 in the fourth memory cell.

Further, the processing module determines which conversion gain is suitable by comparing the sensing signals with different conversion gains, and selects one of the conversion gains as the conversion gain at the subsequent 29 sampling moments. For example, if it is determined that the second gain is more suitable than the first gain, at the sampling time t2, signal acquisition and conversion are performed based on the second gain, and stored in the corresponding target storage unit. The selection of the target storage unit may be referred to in the related content of the embodiments of the present disclosure, and will not be described herein.

When the sampling time t31 is reached, a new reset period is entered, and the conversion gain needs to be reset, similarly, when the sampling time t61 is reached, a new reset period is entered, and the conversion gain needs to be reset again, and so on. The reset of the conversion gain can be referred to from the sampling time t1 to the sampling time t30, and will not be described here.

It should be noted that, for the case of three or more conversion gains, the processing procedure is similar to the procedure shown in table 5, and when each reset period is reached, the conversion gain to be selected in the current reset period is redetermined, and the acquisition of the signal is performed based on the redetermined conversion gain.

In some optional implementation manners, the signal processing device supports a plurality of preset signal processing modes, the processing module can select one signal processing mode to process the electric signals stored in the storage module according to instructions, experience, statistical data, simulation results and the like, and can change the current signal processing mode into another signal processing mode according to requirements.

In some alternative implementations, the signal processing mode includes at least one of a direct-current mode, a time-differential (Temporal Difference) mode, a spatial-differential (Spatial Difference) mode, and a high-order differential mode, and the signal processing result includes at least one of a direct-current result, a time-differential result, a spatial-differential result, and a high-order differential result; corresponding to:

the processing module is used for reading the electric signals of the plurality of photosensitive modules at the same sampling time from the storage module under the condition that the signal processing mode is a direct transmission mode, and obtaining a direct transmission result corresponding to the sampling time according to the electric signals of the plurality of photosensitive modules at the same sampling time;

the processing module is used for reading the electric signal of the photosensitive module at the current sampling moment and the electric signal of the photosensitive module at least one historical sampling moment from the storage module under the condition that the signal processing mode is a time difference mode, and performing time difference according to the electric signal of the current sampling moment and the electric signal of the at least one historical sampling moment to obtain a time difference result corresponding to the photosensitive module;

The processing module is used for reading the electric signals of the photosensitive modules and the electric signals of the adjacent photosensitive modules at the same sampling time from the storage module under the condition that the signal processing mode is a space difference mode, and performing space difference according to the electric signals of the photosensitive modules and the electric signals of the adjacent photosensitive modules to obtain a space difference result corresponding to the photosensitive modules;

the processing module is used for reading the electric signal of the photosensitive module at the current sampling moment and the electric signal of the adjacent photosensitive module at least one historical sampling moment from the storage module under the condition that the signal processing mode is a high-order differential mode, and carrying out time difference according to the electric signal of the photosensitive module at the current sampling moment and the electric signal of the adjacent photosensitive module at the at least one historical sampling moment to obtain a high-order differential result corresponding to the photosensitive module.

It should be noted that the adjacent photosensitive module is a broad concept, and includes both a photosensitive module adjacent to the current photosensitive module and a photosensitive module not adjacent to the current photosensitive module but closer to the current photosensitive module. For example, if a plurality of photosensitive modules form a pixel array, the photosensitive module adjacent to the current photosensitive module belongs to an adjacent photosensitive module, and the photosensitive module with a distance smaller than the preset distance threshold value from the current photosensitive module may also be an adjacent photosensitive module.

In some alternative implementations, the first implementation of the time differential mode includes at least one of: globally executing synchronously and adopting a fixed time difference step length, globally executing synchronously and adopting an adjustable time difference step length and executing asynchronously; the second implementation of the spatial differential mode includes at least one of: globally synchronous execution and with a fixed spatial differential step size, globally synchronous execution and with an adjustable spatial differential step size, asynchronously execution.

It can be known that, for a plurality of photosensitive modules, the time difference may be performed globally and synchronously, or may be performed asynchronously and discretely, the time difference step may be a fixed value, or may be an adjustable non-fixed value, and the time difference step may be globally or not completely the same, which is not limited by the embodiments of the present disclosure. Similarly, when the spatial differential is executed for the plurality of photosensitive modules, the spatial differential may be executed globally and synchronously, or may be executed asynchronously and dispersedly, the spatial differential step may be a fixed value, or may be an adjustable non-fixed value, and the spatial differential step may be globally the same, or may not be completely the same, which is not limited by the embodiment of the present disclosure.

In some alternative implementations, the signal processing apparatus further includes a processing path corresponding to the signal processing mode; correspondingly, the processing module is used for selecting a processing path corresponding to the signal processing mode, and processing the electric signals stored in the storage module based on the selected processing path to obtain a signal processing result.

As can be seen, in the embodiments of the present disclosure, corresponding processing paths are configured for different signal processing modes, and after determining a signal processing mode, corresponding signal processing can be performed through a processing path corresponding to the signal processing mode.

In some alternative implementations, the signal processing modes include at least one of a direct mode, a time differential mode, a spatial differential mode, and a high order differential mode; the processing path includes at least one of a direct transmission path corresponding to a direct transmission mode, a time differential path corresponding to a time differential mode, a space differential path corresponding to a space differential mode, and a high-order differential path corresponding to a high-order differential mode. The time differential path corresponds to a Dynamic Visual Sensor (DVS) processing path, and signal output based on Event (Event-based) can be realized.

Illustratively, the direct-transmission path may include a color path, which is mainly used for signal processing of the color sensing module. Wherein the color channel outputs a sampling time t _n The absolute value of the electrical signal of the photosensitive module corresponding to the position (x, y) can be expressed as:

RGB(x,y,t _n )=I(x,y,t _n )

wherein I (x, y, t) _n ) Representing electrical signals (also known as visual signals), RGB (x, y, t _n ) Representing the output color signal.

The time difference path is mainly used for outputting time difference values of electric signals of the photosensitive modules corresponding to the positions (x, y) at different sampling moments, and can be expressed as follows:

TD(x,y,t _n )=Q _TD (I(x,y,t _n )-I(x,y,t _n-1 ))

wherein t is _n And t _n-1 Representing two sampling instants, TD (x, y, t _n ) Representing time differential signals, Q _TD Representing a method for quantizing a time difference signal, which may be multi-valued #>1bit, i.e., multi-valued time differential mode), or single valued (e.g., positive and negative pulses). Wherein the sampling time t of the electric signal participating in the time difference _n 、t _n-1 Etc. may be in a full array synchronous and same time interval, or full array synchronous but variable time interval, or full array asynchronous manner.

The space differential path is mainly used for outputting the sampling time t _n And the spatial difference value of the electric signals of the photosensitive module corresponding to the position (x, y) and the adjacent photosensitive module (which can be inclined, horizontal or vertical). For spatial differentiation in the horizontal direction, this can be expressed as:

SD _x (x,y,t _n )=Q _SD (I(x,y,t _n )-I(x-1,y,t _n ))

Wherein SD is _x (x,y,t _n ) Representing the spatially differential signal in the horizontal direction.

For spatial differentiation in the vertical direction, this can be expressed as:

SD _y (x,y,t _n )=Q _SD (I(x,y,t _n )-I(x,y-1,t _n ))

wherein SD is _y (x,y,t _n ) Representing the spatially differential signal in the vertical direction.

For oblique spatial differentiation, this can be expressed as:

SD _↙ (x,y,t _n )=Q _SD (I(x,y,t _n )-I(x-1,y-1,t _n ))

SD _↘ (x,y,t _n )=Q _SD (I(x,y,t _n )-I(x+1,y-1,t _n ))

wherein SD is _↙ (x,y,t _n ) Representing the spatially differential signal in 135 direction, SD _↘ (x,y,t _n ) A spatially differential signal in the 45 degree direction is represented (with the three o' clock direction as the starting direction and the clockwise direction as the positive direction of rotation).

In addition, Q _SD Quantization method for representing space differential signal, which can be multi-valued #, the method can be used for the quantization of space differential signal>1bit, i.e., multi-valued spatial differential mode), or single valued (e.g., positive and negative pulses). The sampling time of the participation space differential signal can adopt a mode of full array synchronization and same time interval, or full array synchronization and variable time interval, or full array asynchronization.

It should be noted that the above is only an example for the signal processing mode and the processing path, and the embodiments of the present disclosure are not limited thereto.

It can be seen that the signal processing apparatus according to the embodiments of the present disclosure supports multiple signal processing modes, which may directly output an electrical signal, or may output a differential processing result after performing differential processing in a time dimension and/or a space dimension based on a stored electrical signal.

The processing module is configured to read, for each photosensitive module, an electrical signal at a first sampling time from a storage module corresponding to the photosensitive module, and obtain, according to the electrical signals of the plurality of photosensitive modules at the first sampling time, a direct-transmission result corresponding to the first sampling time, when the signal processing mode is a direct-transmission mode.

For example, the photosensitive module P1 directly outputs a signal sequence in the direct transmission mode with S1 for the electrical signal at the sampling time t1, S2, … … for the electrical signal at the sampling time t2, and Sn for the electrical signal at the sampling time tn: s1, S2, … …, sn. If a plurality of photosensitive modules form a pixel array, and each pixel corresponds to a direct transmission mode, the signal sequences output by each photosensitive module are arranged according to the time dimension and are combined according to the positions of the photosensitive modules, n video frames can be obtained, and each video frame comprises the electric signals of the photosensitive modules at the corresponding sampling moments. And if the photosensitive module is a color photosensitive module, the electric signal is a color signal, and the corresponding video frame is a color video frame, and if the photosensitive module is a gray photosensitive module, the electric signal is a gray signal (corresponding to brightness information), and the corresponding video frame is a gray video frame.

The processing module is configured to read, when the signal processing mode is a time difference mode, the electrical signal at the second sampling time and the electrical signal at the third sampling time from the storage module corresponding to the photosensitive module, and perform a time difference according to the electrical signal at the second sampling time and the electrical signal at the third sampling time, so as to obtain a time difference result corresponding to the photosensitive module, where the second sampling time and the third sampling time are different.

For example, taking table 1 as an example, if the time difference step is one sampling period, a time difference result may be obtained by performing time difference once according to S2 and S1, a time difference result may be obtained by performing time difference once according to S3 and S2, and so on.

For example, still taking table 1 as an example, if the time difference step is two sampling periods, a time difference result may be obtained by performing a time difference according to S3 and S1, a time difference result may be obtained by performing a time difference according to S4 and S2, and so on.

In some alternative implementations, the time differential mode includes a polar time differential mode and/or a multi-valued time differential mode. The polarity time difference mode refers to a time difference result which is a result with positive and negative signs and does not represent a specific numerical value. For example, the electric signals S2 and S1 are time-differentiated based on the polarity time-differentiation mode, and if S2 is greater than S1, a positive signal P1 is output, and if S2 is equal to or less than S1, a negative signal P2 is output. The multi-value time difference mode is to preset a plurality of value intervals, assign a uniform value to each value interval, and determine a time difference result according to the uniform value corresponding to the value interval as long as the signal difference value of the electric signals at different sampling moments falls into a certain value interval. When the value interval is smaller, the accuracy of the multi-value time difference result is higher, and the value interval can be determined according to actual requirements, which is not limited by the embodiment of the disclosure.

The processing module is configured to read, when the signal processing mode is a spatial differential mode, an electrical signal at a fourth sampling time from a storage module corresponding to the first photosensitive module, read, from a storage module of a second photosensitive module adjacent to the first photosensitive module, the electrical signal at the fourth sampling time, and perform spatial differential according to the electrical signal at the fourth sampling time of the first photosensitive module and the electrical signal at the fourth sampling time of the second photosensitive module, so as to obtain a spatial differential result corresponding to the first photosensitive module. In other words, in the spatial differential mode, differential operation can be performed on the electrical signals corresponding to the same sampling time by different light sensing modules in the spatial dimension, so as to obtain a corresponding spatial differential result.

In some alternative implementations, the spatial differential mode includes a polar spatial differential mode and/or a multi-valued spatial differential mode. The polar spatial differential mode refers to that the spatial differential result is a result with positive and negative signs, and does not represent specific numerical values. The multi-value space differential mode is to preset a plurality of value intervals, assign a uniform value to each value interval, and determine a space differential result according to the uniform value corresponding to the value interval as long as signal differences of electric signals of different light sensing modules at the same sampling time fall into a certain value interval. When the value interval is smaller, the accuracy of the multi-value space difference result is higher, and the value interval can be determined according to actual requirements, which is not limited by the embodiment of the disclosure.

In some alternative implementations, the differential direction of the spatial differential mode includes at least one of a horizontal direction, a vertical direction, and a tilt direction. In other words, for any one of the photosensitive modules, the spatial difference operation may be performed with the photosensitive module located above or below itself, the spatial difference operation may be performed with the photosensitive module located on the left and right sides of itself, or the spatial difference operation may be performed with the photosensitive module located on the diagonal corner of itself (for example, the upper left corner, the lower left corner, the upper right corner, and the lower right corner).

The processing module is configured to read, when the signal processing mode is a high-order differential mode, an electrical signal at a fifth sampling time from a storage module corresponding to the third photosensitive module, read an electrical signal at a sixth sampling time from a storage module of a fourth photosensitive module adjacent to the third photosensitive module, and perform a time difference according to the electrical signal at the fifth sampling time of the third photosensitive module and the electrical signal at the sixth sampling time of the fourth photosensitive module, so as to obtain a high-order differential result corresponding to the third photosensitive module. It can be seen that the high-order differential mode essentially combines the time differential and the space differential, so that the time differential can be performed for the photosensitive modules at different positions.

It should be noted that, before the processing module performs signal processing based on the stored electrical signal, the processing module generally needs to read the required electrical signal from the storage module. For a memory cell prepared by using a capacitor or the like, if a new electrical signal is to be stored in a certain capacitor, a charge-discharge behavior may occur, thereby covering the electrical signal originally stored in the capacitor. Based on this, if the electric signal Si at the sampling time ti is stored in the capacitor Cj and the electric signal Si is used for signal processing, it is necessary to read the electric signal Si from the capacitor Cj before charging and discharging the capacitor Cj.

For example, after the switch of the capacitor Cj is turned on, the electric signal Si stored in the capacitor Cj is read out, and e is equal to or greater than 1, before the capacitor Cj is charged and discharged based on the electric signal si+e at the sampling time ti+e.

Fig. 4 is a schematic diagram of a processing procedure of a signal processing apparatus according to an embodiment of the disclosure. Referring to fig. 4, if the first memory cell shown in table 5 corresponds to the capacitor C1, the electrical signal writing process and the electrical signal reading process of the capacitor C1 are schematically shown in fig. 4.

As shown in fig. 4, since the sense signal HS1 at the sampling time t1 is written into C1, the switch of C1 should be set to the on state (the on state is indicated by "on" in the figure, and the off state is indicated by "off") before t1, and based on this, HS1 can be written into C1 after being generated.

Further, since the sense signal LS2 at the sampling time t2 is to be written into C1, after t1, before t2, the switch of C1 should be set to the on state, and HS1 should be read out of C1 before writing into LS 2. Illustratively, the read operation of HS1 may be performed at an intermediate time between t1 and t 2.

After LS2 is written in C1, since the sense signal at time t3 does not need to be written in C1, C1 can be turned off. However, since the sense signal LS4 at the sampling time t4 is written into C1, after t3, before t4, the switch of C1 should be set to the on state, and LS2 should be read out of C1 before writing into LS 4. Illustratively, the read operation of LS2 may be performed at an intermediate time between t3 and t 4.

Similarly, after LS4 is written to C1, C1 can be placed in the OFF state because the sense signal at time t5 need not be written to C1. However, since the sense signal LS6 at the sampling time t6 is written into C1, after t5, before t6, the switch of C1 should be set to the on state, and LS4 should be read out of C1 before writing into LS 6. Illustratively, the read operation of LS4 may be performed at an intermediate time between t5 and t 6.

Further, after writing LS6 to C1, C1 may be placed in an off state to preserve LS6 stored therein since the sense signal at time t7 need not be written to C1. Similarly, the writing and reading of the electrical signals can be sequentially performed, providing a data basis for the subsequent processing module to perform signal processing based on the signal processing mode.

In some alternative implementations, the spatial differential processing with a differential direction being horizontal and a spatial differential step size of 1 can be characterized asThe difference operation is performed on the electric signals of the photosensitive modules at two left and right adjacent positions corresponding to the first row.

In some alternative implementationsWherein the differential direction is horizontal, and the spatial differential process with the spatial differential step length of 2 can be characterized asThe three photosensitive modules corresponding to the first row are represented, and the electric signals of the leftmost photosensitive module and the rightmost photosensitive module are subjected to differential operation. By analogy, for the case where the differential direction is horizontal, and when the spatial differential step size is greater than 2, the spatial differential processing mode may be characterized in a similar manner.

In some alternative implementations, the differential direction is vertical and the spatial differential processing with a spatial differential step size of 1 can be characterized as The difference operation is performed on the electric signals of the photosensitive modules at two upper and lower adjacent positions corresponding to the first column.

In some alternative implementations, the differential direction is vertical and the spatial differential processing with a spatial differential step size of 2 can be characterized asThe three corresponding photosensitive modules in the first column are represented, and the electric signals of the uppermost photosensitive module and the lowermost photosensitive module are subjected to differential operation. By analogy, for the case where the differential direction is the vertical direction, and when the spatial differential step size is greater than 2, the spatial differential processing manner may be characterized in a similar manner.

In some alternative implementations, the spatial differential processing with a differential direction of 45 degrees and a spatial differential step of 1 may be characterized asThe difference operation is performed on the electric signals of the photosensitive modules at the upper left corner of the first row and the photosensitive modules at the lower right corner of the second row.

In some alternative implementations, the spatial differential processing with a differential direction of 45 degrees and a spatial differential step of 2 may be characterized asThe difference operation is performed on the electric signals of the photosensitive modules at the upper left corner of the first row and the photosensitive modules at the lower right corner of the third row. And so on, for the case that the differential direction is 45 degrees, and the space differential step length is greater than 2, the space differential processing mode can be represented in a similar way.

In some alternative implementations, the spatial differential processing with a differential direction of 135 degrees and a spatial differential step of 1 may be characterized asThe difference operation is performed on the electric signals of the photosensitive modules at the upper right corner of the first row and the photosensitive modules at the lower left corner of the second row.

In some alternative implementations, the spatial differential processing with a differential direction of 135 degrees and a spatial differential step of 2 may be characterized asThe difference operation is performed on the electric signals of the photosensitive modules at the upper right corner of the first row and the photosensitive modules at the lower left corner of the third row. By analogy, for the case where the differential direction is 135 degrees and the spatial differential step size is greater than 2, the spatial differential processing mode may be characterized in a similar manner.

It should be noted that the above is merely an example for the spatial differential mode, and the embodiments of the present disclosure are not limited thereto.

Fig. 5 is a schematic diagram of a processing procedure of a signal processing apparatus according to an embodiment of the present disclosure, which includes (a), (b), (c), and (d), and exemplarily illustrates a signal processing procedure of a 4×4 array distributed pixel array, each pixel including a photosensitive module and a storage module. In the pixel array shown in the (a), the photosensitive modules of all pixels are gray photosensitive modules; (b) In the pixel array, the photosensitive modules of all pixels are color photosensitive modules, including a red photosensitive module, a green photosensitive module and a blue photosensitive module; (c) And (d) the pixel array comprises a color photosensitive module and a gray photosensitive module, and the arrangement modes of the two photosensitive modules are different.

As shown in (a), first, taking the gray scale pixel module 0 as an example, it can spatially differentiate with the gray scale pixel module 1, where the spatial differentiation step is 1 and the differentiation direction is the horizontal direction (or 0 degree direction); the gray pixel module 0 can also perform space difference with the gray pixel module 2, the space difference step length is 2, and the difference direction is the horizontal direction; the gray scale pixel module 0 may also perform spatial difference with the gray scale pixel module 3, where the spatial difference step size is 3 and the difference direction is the horizontal direction.

Secondly, the gray pixel module 0 and the gray pixel module 4 can be spatially differentiated, the spatial differentiating step length is 1, and the differentiating direction is the vertical direction (or 90 degrees direction); the gray pixel module 0 can also perform space difference with the gray pixel module 8, the space difference step length is 2, and the difference direction is the vertical direction; the gray pixel module 0 may be spatially differentiated from the gray pixel module 12, where the spatial differentiating step is 3 and the differentiating direction is vertical.

Again, the gray scale pixel module 0 may spatially differ from the gray scale pixel module 5 by a spatial difference step of 1, the difference direction being the lower right direction (or 45 degree direction); the gray pixel module 0 can also perform spatial difference with the gray pixel module 10, wherein the spatial difference step length is 2, and the difference direction is the lower right corner direction; the gray scale pixel module 0 may be spatially differentiated from the gray scale pixel module 15, and the spatial differentiating step is 3, and the differentiating direction is the lower right corner direction. The other gray scale pixel blocks are similar to gray scale pixel block 0 and will not be described again.

As shown in (b), if the (q, v) th row is denoted as the (q, v) th pixel block, taking the (1, 1) position red pixel block as an example, it may be spatially differentiated from the (1, 3) position red pixel block, where the spatial differentiating step is 2, and the differentiating direction is the horizontal direction (or 0 degree direction); the red pixel module at the (1, 1) position can be spatially differentiated with the red pixel module at the (3, 1) position, the spatial differentiating step length is 2, and the differentiating direction is the vertical direction (or 90 degrees direction); the red pixel module at the (1, 1) position can be spatially differentiated from the red pixel module at the (3, 3) position, and the spatial differentiating step is 2, and the differentiating direction is the lower right corner direction (or 45-degree direction). Other pixel modules are similar and will not be described here.

In (c), the spatial differential manner of the red pixel module, the green pixel module, and the blue pixel module may be referred to as (b), and for the gray pixel module in (c), it may be spatially differential from the adjacent gray pixel module. For example, the (2, 1) position gray scale pixel module may be spatially differentiated from the (2, 3) position gray scale pixel module, where the spatial difference step size is 2, the difference direction is the horizontal direction (or 0 degree direction), the spatial difference step size is 2, the difference direction is the vertical direction (or 90 degree direction), the spatial difference step size is 2, and the difference direction is the lower right angle direction (or 45 degree direction).

In (d), the spatial differential manner of the red pixel module, the green pixel module and the blue pixel module may refer to (b) or (c), and for the gray pixel module in (d), it may be spatially differential from the adjacent gray pixel module. Taking the gray pixel module 16 as an example, it can perform spatial difference with the gray pixel module 17, and the spatial difference step length at this time is 1, and the difference direction is the horizontal direction (or 0 degree direction); the gray pixel module 16 may also perform spatial difference with the gray pixel module 18, where the spatial difference step is 1, and the difference direction is the vertical direction (or 90 degrees direction); the gray pixel module 16 may also perform spatial difference with the gray pixel module 19, where the spatial difference step is 1, and the difference direction is the lower right corner direction (or 45 degree direction); the gray pixel module 16 may also perform spatial difference with the gray pixel module 20, where the spatial difference step is 2, and the difference direction is the lower right corner direction (or 45 degree direction); the gray scale pixel module 16 may also be spatially differentiated from the gray scale pixel module 23 by a spatial differentiation step of 3, and the differentiation direction is the lower right corner direction (or 45 degree direction). Of course, the gray scale pixel module 16 may also be spatially differentiated from the gray scale pixel modules 21 and 22, which is not limited by the embodiments of the present disclosure.

A second aspect of the disclosed embodiments provides a sensor chip.

Fig. 6 is a schematic diagram of a sensor chip according to an embodiment of the disclosure. Referring to fig. 6, the sensor chip 600 includes at least one signal processing device 610. The signal processing device 610 may be any one of the signal processing devices according to the embodiments of the present disclosure.

In some alternative implementations, the sensor chip 600 may be applied to an image imaging field, a video photographing field, etc., and may be capable of adjusting a signal processing mode according to a need, thereby outputting at least one of a color signal, a light intensity signal, a time differential signal, a space differential signal, and a high-order differential signal, which is suitable for diversified imaging scenes.

The sensor chip comprises at least one signal processing device, wherein the signal processing device comprises a photosensitive module, a storage module and a processing module, and the photosensitive module is used for generating an electric signal corresponding to an optical signal incident to the photosensitive module at a preset sampling moment; the storage module is used for storing the electric signals of the photosensitive modules, wherein the storage module comprises at least two storage sub-modules, each storage sub-module is used for storing the electric signals of one sampling moment of the photosensitive modules, and the electric signals stored for any sampling moment of each photosensitive module comprise at least one of the following: the method comprises the steps of detecting an electric signal of a photosensitive module at a sampling moment, detecting the electric signal of the photosensitive module at the sampling moment and detecting at least one historical sampling moment; the processing module is used for processing the electric signals stored in the storage module according to the signal processing mode to obtain signal processing results, wherein the signal processing results comprise at least one of direct transmission results, time difference results, space difference results and high-order difference results. Therefore, the electric signals generated by the photosensitive module can be stored in the storage module, and the electric signals at one sampling moment can be respectively stored in at least two storage sub-modules of the storage module, so that a data basis is provided for subsequent signal processing; further, the signal processing device supports a plurality of preset signal processing modes, so that when processing is performed based on the stored electrical signals, a proper signal processing mode can be selected from the stored electrical signals, and further, the stored electrical signals are processed based on the signal processing mode, so that at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result is obtained. That is, in the embodiment of the present disclosure, different signal processing modes are configured for the photosensitive module, so that the signal processing apparatus may multiplex the photosensitive module to implement multiple signal processing modes, thereby being capable of adapting to various signal processing scenarios, and having higher flexibility.

A third aspect of an embodiment of the present disclosure provides a signal processing method.

Fig. 7 is a flowchart of a signal processing method according to an embodiment of the present disclosure. Referring to fig. 7, the method may include the following steps.

Step S701, determining a signal processing mode according to the received mode setting instruction.

Step S702, processing the electrical signal of the photosensitive module stored in the storage module according to the signal processing mode, to obtain a signal processing result.

The signal processing device may adopt any one of the embodiments of the present disclosure.

In some alternative implementations, the mode setting instruction may be an instruction issued according to a processing requirement or the like for indicating a signal processing mode for the electrical signal.

In some alternative implementations, the signal processing mode includes at least one of a direct mode, a time differential mode, a spatial differential mode, and a high order differential mode.

For example, for a shooting scene of a high-speed moving object, in order to avoid imaging blur, a time difference mode may be selected and a corresponding mode setting instruction may be issued so that, when signal processing is performed, a signal is output only for a pixel region where a light intensity change is large, and no signal is output for a pixel region where a light intensity change is small.

For example, in some landscape shooting scenes, in order to improve shooting effects, a direct-transmission mode may be selected and a corresponding mode setting instruction may be issued to accurately convert an optical signal into a corresponding color signal and output it to the outside when signal processing is performed.

In some alternative implementations, the signal processing results include at least one of a direct-input result, a time-difference result, a space-difference result, and a high-order-difference result; correspondingly, the processing of the electrical signals of the photosensitive modules stored in the storage module according to the signal processing mode to obtain signal processing results comprises the following steps:

under the condition that the signal processing mode is a direct transmission mode, reading electric signals of a plurality of photosensitive modules at the same sampling time from a storage module, and obtaining a direct transmission result corresponding to the sampling time according to the electric signals of the plurality of photosensitive modules at the same sampling time;

under the condition that the signal processing mode is a time difference mode, reading an electric signal of the photosensitive module at the current sampling time and an electric signal of the photosensitive module at least one historical sampling time from the storage module, and performing time difference according to the electric signal of the current sampling time and the electric signal of the at least one historical sampling time to obtain a time difference result corresponding to the photosensitive module;

Under the condition that the signal processing mode is a space difference mode, reading the electric signals of the photosensitive modules and the electric signals of the adjacent photosensitive modules at the same sampling time from the storage module, and performing space difference according to the electric signals of the photosensitive modules and the electric signals of the adjacent photosensitive modules to obtain a space difference result corresponding to the photosensitive modules;

under the condition that the signal processing mode is a high-order difference mode, reading an electric signal of the photosensitive module at the current sampling time and an electric signal of the adjacent photosensitive module at least one historical sampling time from the storage module, and carrying out time difference according to the electric signal of the photosensitive module at the current sampling time and the electric signal of the adjacent photosensitive module at the at least one historical sampling time to obtain a high-order difference result corresponding to the photosensitive module.

In the embodiment of the disclosure, the signal processing mode of the electric signal can be determined based on the mode setting instruction, and the required electric signal is read from the storage module, so that the read electric signal is processed in the determined signal processing mode to obtain a corresponding signal processing result, thereby being applicable to various signal processing scenes and having higher flexibility.

It will be appreciated that the above embodiments mentioned in the present disclosure may be combined with each other to form a combined embodiment without departing from the principle logic, and are limited to the descriptions of the embodiments are omitted. It will be appreciated by those skilled in the art that in the above-described methods of the embodiments, the particular order of execution of the steps and arrangement of functional blocks should be determined by their function and possible inherent logic.

Furthermore, the disclosure also provides an electronic device and a computer readable storage medium.

Referring to fig. 8, an embodiment of the present disclosure provides an electronic device including: at least one processor 801; at least one memory 802, and one or more I/O interfaces 803, coupled between the processor 801 and the memory 802; wherein the memory 802 stores one or more computer programs executable by the at least one processor 801, the one or more computer programs being executed by the at least one processor 801 to enable the at least one processor 801 to perform the signal processing methods of embodiments of the present disclosure.

Referring to fig. 9, an embodiment of the present disclosure provides an electronic device including a plurality of processing cores 901 and a network-on-chip 902, wherein the plurality of processing cores 901 are each connected to the network-on-chip 902, and the network-on-chip 902 is configured to interact data between the plurality of processing cores and external data.

Wherein one or more processing cores 901 have one or more instructions stored therein that are executed by the one or more processing cores 901 to enable the one or more processing cores 901 to perform the signal processing methods of the embodiments of the present disclosure.

In some embodiments, the electronic device may be a brain-like chip, and since the brain-like chip may employ a vectorization computing manner, parameters such as weight information of a neural network model need to be called into through an external memory, for example, double Data Rate (DDR) synchronous dynamic random access memory. Therefore, the operation efficiency of batch processing is high in the embodiment of the disclosure.

The disclosed embodiments also provide a computer readable storage medium having a computer program stored thereon, wherein the computer program when executed by a processor/processing core implements the various tasks to be processed. The computer readable storage medium may be a volatile or nonvolatile computer readable storage medium.

Embodiments of the present disclosure also provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when executed in a processor of an electronic device, performs various tasks to be processed.

Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer-readable storage media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).

The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, random Access Memory (RAM), read Only Memory (ROM), erasable Programmable Read Only Memory (EPROM), static Random Access Memory (SRAM), flash memory or other memory technology, portable compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and may include any information delivery media.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.

The computer program product described herein may be embodied in hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, it will be apparent to one skilled in the art that features, characteristics, and/or elements described in connection with a particular embodiment may be used alone or in combination with other embodiments unless explicitly stated otherwise. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. A signal processing apparatus, comprising:

the device comprises a photosensitive module, a sampling module and a display module, wherein the photosensitive module is used for generating an electric signal corresponding to an optical signal incident to the photosensitive module at a preset sampling moment;

the storage module is used for storing the electric signals of the photosensitive modules, wherein the storage module comprises at least two storage sub-modules, each storage sub-module is used for storing the electric signals of one sampling moment of the photosensitive modules, and the electric signals stored for any sampling moment of each photosensitive module comprise at least one of the following: the electric signal of the photosensitive module at the sampling moment, the electric signal of the photosensitive module at the sampling moment and at least one historical sampling moment;

The processing module is used for processing the electric signals stored in the storage module according to a preset signal processing mode to obtain a signal processing result, wherein the signal processing result comprises at least one of a direct transmission result, a time difference result, a space difference result and a high-order difference result.

2. The apparatus of claim 1, wherein the processing module is further configured to, in a case where the photosensitive module generates an electrical signal at a current sampling time, select a target storage sub-module from storage modules having a correspondence with the photosensitive module, and store the electrical signal at the current sampling time to the target storage sub-module.

3. The apparatus of claim 1, wherein the storage module comprises two of the storage sub-modules;

the storage submodules respectively store the electric signal of the j-1 sampling moment and the electric signal of the j-2 sampling moment at the j-1 sampling moment; and at the j-th sampling moment, taking a storage sub-module for storing the electrical signals at the j-2 th sampling moment as a target storage sub-module, storing the electrical signals at the j-th sampling moment in the target storage sub-module, and reserving the electrical signals at the j-1 th sampling moment stored in the other storage sub-module, wherein j is an integer and j is more than 2.

4. The apparatus of claim 1, wherein the electrical signal comprises a sense signal, the sense signal being a signal generated by the light sensing module by photo-electric sensing of the light signal, and the sense signal being a signal having a spatial dimension and a temporal dimension;

or the electric signal comprises the sensing signal and a reset signal corresponding to the sensing signal, wherein the reset signal is used for resetting the sensing signal.

5. The apparatus of claim 4, wherein each of said photosensitive modules corresponds to one of said storage modules, and said storage modules comprises two of said storage sub-modules, a first of said storage sub-modules comprising a first storage unit and a second storage unit, a second of said storage sub-modules comprising a third storage unit and a fourth storage unit;

the processing module is further configured to: for the ith sampling moment, when the first storage unit and the second storage unit respectively store the induction signal and the reset signal of the ith-1 sampling moment, and the third storage unit and the fourth storage unit respectively store the induction signal and the reset signal of the ith-2 sampling moment, the switches of the first storage unit and the second storage unit are turned off, and the switches of the third storage unit and the fourth storage unit are turned on, so that the third storage unit stores the induction signal of the ith sampling moment, and the fourth storage unit stores the reset signal corresponding to the induction signal of the ith sampling moment, wherein i is an integer and i >2.

6. The apparatus of claim 5, wherein the conversion gain corresponding to the photosensitive module comprises a first gain and a second gain;

the sensing signal comprises a first gain sensing signal corresponding to the first gain and/or a second gain sensing signal corresponding to the second gain, and the reset signal comprises a first gain reset signal corresponding to the first gain and/or a second gain reset signal corresponding to the second gain.

7. The apparatus of claim 6, wherein the photosensitive module further comprises a gain adjustment unit, and the gain adjustment unit is configured to adjust the conversion gain according to an ambient brightness of the photosensitive module, so that the photosensitive module generates an electrical signal corresponding to the conversion gain.

8. The apparatus of claim 1, wherein the signal processing mode comprises at least one of a direct mode, a time differential mode, a spatial differential mode, and a high order differential mode;

the processing module is used for reading the electric signals of the plurality of photosensitive modules at the same sampling moment from the storage module under the condition that the signal processing mode is the direct transmission mode, and obtaining a direct transmission result corresponding to the sampling moment according to the electric signals of the plurality of photosensitive modules at the same sampling moment;

The processing module is used for reading the electric signal of the photosensitive module at the current sampling moment and the electric signal of the photosensitive module at least one historical sampling moment from the storage module under the condition that the signal processing mode is the time difference mode, and performing time difference according to the electric signal of the current sampling moment and the electric signal of the at least one historical sampling moment to obtain a time difference result corresponding to the photosensitive module;

the processing module is used for reading the electric signals of the photosensitive module and the electric signals of the adjacent photosensitive modules at the same sampling time from the storage module under the condition that the signal processing mode is the space difference mode, and performing space difference according to the electric signals of the photosensitive modules and the electric signals of the adjacent photosensitive modules to obtain a space difference result corresponding to the photosensitive modules;

the processing module is used for reading the electric signal of the photosensitive module at the current sampling time and the electric signal of the adjacent photosensitive module at least one historical sampling time from the storage module under the condition that the signal processing mode is the high-order differential mode, and carrying out time difference according to the electric signal of the photosensitive module at the current sampling time and the electric signal of the adjacent photosensitive module at the at least one historical sampling time to obtain a high-order differential result corresponding to the photosensitive module.

9. The device of claim 8, wherein the photosensitive module and the storage module have a correspondence therebetween;

the processing module is used for reading the electric signals at the first sampling moment from the storage module corresponding to each photosensitive module under the condition that the signal processing mode is the direct transmission mode, and obtaining the direct transmission result corresponding to the first sampling moment according to the electric signals of the plurality of photosensitive modules at the first sampling moment;

the processing module is configured to read, when the signal processing mode is the time difference mode, an electrical signal at a second sampling time and an electrical signal at a third sampling time from the storage module corresponding to the photosensitive module, and perform time difference according to the electrical signal at the second sampling time and the electrical signal at the third sampling time, to obtain a time difference result corresponding to the photosensitive module, where the second sampling time and the third sampling time are different sampling times;

the processing module is configured to read an electrical signal at a fourth sampling time from a storage module corresponding to a first photosensitive module when the signal processing mode is the spatial differential mode, read the electrical signal at the fourth sampling time from a storage module of a second photosensitive module adjacent to the first photosensitive module, and perform spatial differential according to the electrical signal at the fourth sampling time of the first photosensitive module and the electrical signal at the fourth sampling time of the second photosensitive module, so as to obtain a spatial differential result corresponding to the first photosensitive module;

And the processing module is used for reading the electric signal at the fifth sampling moment from the storage module corresponding to the third photosensitive module under the condition that the signal processing mode is the high-order differential mode, reading the electric signal at the sixth sampling moment from the storage module of the fourth photosensitive module adjacent to the third photosensitive module, and carrying out time difference according to the electric signal at the fifth sampling moment of the third photosensitive module and the electric signal at the sixth sampling moment of the fourth photosensitive module to obtain a high-order differential result corresponding to the third photosensitive module.

10. The apparatus of claim 8, wherein the first implementation of the time differential mode comprises at least one of: globally executing synchronously and adopting a fixed time difference step length, globally executing synchronously and adopting an adjustable time difference step length and executing asynchronously;

the second implementation of the spatial differential mode includes at least one of: globally synchronous execution and with a fixed spatial differential step size, globally synchronous execution and with an adjustable spatial differential step size, asynchronously execution.

11. The apparatus of claim 8, wherein a plurality of said photosensitive modules are present in at least one of: the plurality of photosensitive modules are all color photosensitive modules; the plurality of photosensitive modules are gray photosensitive modules; part of the photosensitive modules are color photosensitive modules, and part of the photosensitive modules are gray photosensitive modules;

The color photosensitive module is a photosensitive module for collecting color information, and the gray photosensitive module is a photosensitive module for collecting brightness information.

12. The apparatus of claim 1, wherein the signal processing mode comprises at least one of a direct mode, a time differential mode, a spatial differential mode, and a high order differential mode;

the processing path of the signal processing device includes at least one of a direct transmission path corresponding to the direct transmission mode, a time differential path corresponding to the time differential mode, a space differential path corresponding to the space differential mode, and a high-order differential path corresponding to the high-order differential mode.

13. A sensor chip, characterized in that the sensor chip comprises at least one signal processing means;

wherein the signal processing device employs a signal processing device according to any one of claims 1-12.

14. A signal processing method, characterized by being applied to a signal processing apparatus, the method comprising:

determining a signal processing mode according to the received mode setting instruction;

processing the electric signals of the photosensitive modules stored in the storage module according to the signal processing mode to obtain a signal processing result;

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executable by the at least one processor to enable the at least one processor to perform the signal processing method of claim 14.

16. A computer readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the signal processing method as claimed in claim 14.