CN112188092A - Dual-mode signal processing system and method - Google Patents
Dual-mode signal processing system and method Download PDFInfo
- Publication number
- CN112188092A CN112188092A CN202011019239.5A CN202011019239A CN112188092A CN 112188092 A CN112188092 A CN 112188092A CN 202011019239 A CN202011019239 A CN 202011019239A CN 112188092 A CN112188092 A CN 112188092A
- Authority
- CN
- China
- Prior art keywords
- image
- data processor
- signal
- bimodal
- asynchronous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control
Abstract
The invention discloses a bimodal signal processing system and a bimodal signal processing method. The bimodal signal processing system comprises an image sensor unit, a first image processing unit and a second image processing unit, wherein the image sensor unit is used for acquiring bimodal image signals; and the image processing unit is connected with the image sensor unit and comprises a synchronous data processor and an asynchronous data processor, the asynchronous data processor is used for processing asynchronous signals in the bimodal image signals, and the synchronous processor is used for processing synchronous signals in the bimodal image signals. The scheme enables the image processing unit to simultaneously process the synchronous signal and the asynchronous signal in the bimodal image signal, thereby improving the efficiency of the image processing unit for processing the bimodal image signal, further improving the efficiency and the real-time performance of the bimodal signal processing system and improving the application range of the bimodal signal processing system.
Description
Technical Field
The embodiment of the invention relates to the technical field of image sensing, in particular to a bimodal signal processing system and a bimodal signal processing method.
Background
The vision sensor refers to an apparatus for acquiring image information of an external environment by using an optical element and an imaging device, and the vision sensor in the prior art generally includes: active Pixel Sensors (APS) and Dynamic Vision Sensors (DVS). Among them, the active pixel sensor is usually an image sensor based on a voltage signal or a current signal, and is widely applied to an image pickup unit of a mobile phone or a camera, and such an image sensor has the advantages of high color reproduction and high image quality, but the dynamic range of the acquired image signal is small, and the shooting speed is slow. The dynamic vision sensor is commonly used in the field of industrial control, and is characterized by being capable of sensing a dynamic scene, and because the shooting speed is high and the dynamic range of the obtained image signal is large, the quality of the image acquired by the sensor is poor.
In the prior art, a visual image processor for processing an image signal provided by an active pixel sensor is separated from storage according to a von neumann architecture, has a simple structure, and is easy to realize high-speed numerical calculation. However, when processing an image signal provided by a dynamic vision sensor, the image signal of the dynamic vision sensor includes unstructured, spatiotemporal correlation information, and a processor according to the "von neumann" architecture exhibits problems of low efficiency, high power consumption, poor real-time performance, and the like. When the image signal to be processed by the image signal processing system is a bimodal signal, the processor in the prior art cannot process the bimodal signal in real time and high efficiency, so that the application range of the image signal processing system is affected.
Disclosure of Invention
The invention provides a bimodal signal processing system and a bimodal signal processing method, which are used for improving the efficiency and the real-time performance of the bimodal signal processing system, so that the application range of the bimodal signal processing system is enlarged.
In a first aspect, an embodiment of the present invention provides a dual-mode signal processing system, including:
an image sensor unit for acquiring a bimodal image signal;
an image processing unit connected with the image sensor unit, the image processing unit includes a synchronous data processor and an asynchronous data processor, the asynchronous data processor is used for processing asynchronous signals in the bimodal image signals, and simultaneously the synchronous processor is used for processing synchronous signals in the bimodal image signals.
Furthermore, the image sensor unit and the image processing unit are connected through a high-speed data transmission interface and used for transmitting the bimodal image signal.
Further, the image sensor unit comprises an image sensor, and the asynchronous data processor is in communication connection with the image sensor and is further used for configuring parameters of the image sensor.
Further, the asynchronous data processor is in communication connection with the image sensor through a two-wire serial bus or a universal asynchronous receiver transmitter.
Further, the image sensor includes a cone cell circuit and a rod cell circuit; the cone cell circuit is used for collecting image color signals to form synchronous signals in the bimodal image signals, and the rod cell circuit is used for collecting gray gradient image signals to form asynchronous signals in the bimodal image signals.
Further, the parameters of the image sensor include an AD scanning range, a reset time length, a shutter speed, and a sampling speed in the cone cell circuit, and a DA current output range and speed in the rod cell circuit.
Further, the rod cell circuit comprises an excitatory rod cell circuit and an inhibitory rod cell circuit; the difference between the gray scale gradient image signal collected by the excitatory rod cell circuit and the gray scale gradient image signal collected by the inhibitory rod cell circuit is used as the gray scale gradient image signal of the rod cell circuit.
Further, the image sensor unit further comprises a front-end data processing unit;
the early data processing unit is connected with the image sensor and used for packing and compressing the bimodal image signals.
Further, the synchronous data processor is connected with the asynchronous data processor; the synchronous data processor is further configured to configure parameters of the asynchronous data processor according to the synchronous signal.
Further, the synchronous data processor is at least one of an ARM processor, a central processing unit, an application specific integrated circuit, a system on chip and an image signal processor.
Further, the asynchronous data processor comprises a field programmable gate array.
Further, the asynchronous data processor comprises a rod data processor;
the rod data processor is used for compressing the asynchronous signals in the bimodal image signals, encoding the signals into an asynchronous mode and forming address form event signals.
Further, the asynchronous data processor further comprises a viewing cone data processor;
the viewing cone data processor is used for carrying out automatic focusing, automatic exposure and automatic white balance on the synchronous signals in the bimodal image signals.
Further, the image processing unit further comprises a storage module; the storage module is used for caching the bimodal image signal.
In a second aspect, an embodiment of the present invention further provides a bimodal signal processing method, including:
acquiring a bimodal image signal through an image sensor unit;
and processing the asynchronous signal in the bimodal image signal through an asynchronous data processor in an image processing unit, and simultaneously processing the synchronous signal in the bimodal image signal through a synchronous data processor in the image processing unit.
Further, the image sensor unit includes an image sensor, and before acquiring the bimodal image signal by the image sensor unit, the image sensor unit further includes:
configuring, by the asynchronous data processor, parameters of the image sensor.
Further, the bimodal signal processing method further comprises:
and the synchronous data processor configures the parameters of the asynchronous data processor according to the synchronous signal.
According to the technical scheme of the embodiment of the invention, the image processing unit comprises the synchronous data processor and the asynchronous data processor, the asynchronous data processor processes asynchronous signals in the bimodal image signals, and the synchronous data processor simultaneously processes synchronous signals in the bimodal image signals, so that the image processing unit can simultaneously process synchronous signals and asynchronous signals in the bimodal image signals, the efficiency of the image processing unit for processing the bimodal image signals can be improved, the efficiency and the real-time performance of the bimodal signal processing system are improved, and the application range of the bimodal signal processing system is enlarged.
Drawings
FIG. 1 is a schematic structural diagram of a dual-mode signal processing system according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention;
fig. 7 is a flowchart of a bimodal signal processing method according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Fig. 1 is a schematic structural diagram of a dual-mode signal processing system according to an embodiment of the present invention. As shown in fig. 1, the dual-modality signal processing system includes:
an image sensor unit 110, the image sensor unit 110 being configured to acquire a bimodal image signal;
and an image processing unit 120 connected to the image sensor unit 110, the image processing unit 120 including a synchronous data processor 121 and an asynchronous data processor 122, the asynchronous data processor 122 processing asynchronous signals in the bimodal image signal, and the synchronous processor 121 processing synchronous signals in the bimodal image signal.
Wherein the bimodal image signal includes a synchronous signal and an asynchronous signal. The synchronization signal may be an image color signal which has a very high degree of color reproduction of an image and has a high image quality. The asynchronous signal may be a gray-scale gradient image signal, which may represent a gray-scale variation amount of an image at a high speed. By acquiring the bimodal image signal by the image sensor unit 110, it is possible to realize high signal fidelity in capturing an image at high speed while realizing a high dynamic range and high time resolution. The image processing unit 120 includes a synchronous data processor 121 and an asynchronous data processor 122, the asynchronous data processor 122 processes an asynchronous signal in the dual-modality image signal, and the asynchronous data processor 122 may, for example, packetize and compress the asynchronous signal in the dual-modality image signal, and the like. Meanwhile, the synchronous data processor 121 processes the synchronous signal in the bimodal image signal, and exemplarily, the synchronous data processor 121 performs correction, computational processing of a brain-like perception algorithm, and the like on the synchronous signal in the bimodal image signal. The image processing unit 120 can process the synchronous signal and the asynchronous signal in the bimodal image signal at the same time, so that the efficiency of the image processing unit 120 for processing the bimodal image signal can be improved, the efficiency and the real-time performance of the bimodal signal processing system are improved, and the application range of the bimodal signal processing system is widened.
Fig. 2 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention. As shown in fig. 2, the image sensor unit 110 and the image processing unit 120 are connected through a high-speed data transmission interface 130 for transmitting a bimodal image signal.
The high-speed data transmission interface 130 is a data transmission interface with a relatively large data transmission capability, for example, the high-speed data transmission interface 130 may be an interface with a data transmission rate greater than 1 Gbit/s. The image sensor unit 110 is connected to the image processing unit 120 through the high-speed data transmission interface 130, so that the bimodal image signal acquired by the image sensor unit 110 can be transmitted to the image processing unit 120 through the high-speed data transmission interface 130, and the bimodal image signal transmission is realized. Illustratively, the high-speed data transmission interface 130 may be a Low-Voltage Differential Signaling (LVDS) interface, the data transmission rate of which is at most 2Gbit/s, and the image sensor unit 110 may transmit the bimodal image signal to the image processing unit 120 at a high speed, and the image processing unit 120 receives and processes the bimodal image signal in real time, so as to improve the real-time performance of the bimodal signal processing system.
In addition, the image sensor unit 110 may be connected to the asynchronous data processor 122 through the high-speed data transmission interface 130, the asynchronous data processor 122 is connected to the synchronous data processor 121, and the synchronous signal in the dual-modality image signal is transmitted to the synchronous data processor 121 through the asynchronous data processor 122, so that it is possible to avoid a situation where the format of the synchronous signal in the dual-modality image signal is incompatible with the format of the signal that the synchronous data processor 121 can process.
Fig. 3 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention. As shown in fig. 3, the image sensor unit 110 includes an image sensor 111, and an asynchronous data processor 122 is communicatively connected to the image sensor 111, and the asynchronous data processor 122 is further configured to configure parameters of the image sensor 111.
Among them, the image sensor 111 may be a dual-modality image sensor, that is, the image sensor 111 includes a cone cell circuit and a rod cell circuit; the cone cell circuit is used for collecting image color signals to form synchronous signals in the bimodal image signals, and the rod cell circuit is used for collecting gray gradient image signals to form asynchronous signals in the bimodal image signals. The asynchronous data processor 122 is communicatively connected to the image sensor 111, and before the image sensor 111 acquires the bimodal image signal, the asynchronous data processor 122 transmits a parameter configuration instruction of the image sensor 111 to the image sensor 111 for configuring parameters of the image sensor 111, so that the image sensor 111 can normally operate when acquiring the bimodal image signal. Illustratively, the parameters of the image sensor 111 include an AD scanning range, a reset time length, a shutter speed and a sampling speed in the cone cell circuit, and a DA current output range and speed in the rod cell circuit. Through setting the AD scanning range, the reset time length, the shutter speed and the sampling speed in the cone cell circuit, the acquisition precision of image color signals can be set, so that the cone cell circuit can reasonably acquire the image color signals, and the image signal distortion is avoided. By setting the DA current output range and speed in the rod cell circuit, the frequency and the dynamic range of the rod cell circuit for acquiring the gray gradient image signals can be set, so that the gray gradient image signals acquired by the rod cell circuit meet the shooting requirements in different application scenes.
After the asynchronous data processor 122 configures parameters of the image sensor 111, the image sensor 111 starts to work, acquires an image to acquire a bimodal image signal, the acquired bimodal image signal is transmitted to the asynchronous data processor 122 through the high-speed data transmission interface 130, the asynchronous data processor 122 receives the bimodal image signal in real time, processes an asynchronous signal in the bimodal image signal, transmits a synchronous signal in the bimodal image signal to the synchronous data processor 121, and the synchronous data processor 121 processes the synchronous signal, so that the image processing unit 120 can simultaneously process the synchronous signal and the asynchronous signal in the bimodal image signal, the efficiency of the image processing unit 120 for processing the bimodal image signal is improved, and the efficiency and the real-time performance of the bimodal signal processing system are improved.
It should be noted that the outputs of the cone cell circuit and the rod cell circuit in the image sensor 111 may be in a synchronous mode, that is, the image color signals collected by the cone cell circuit are output in the synchronous mode, that is, the output signals are synchronous signals. The gray scale gradient image signals collected by the rod cell circuit are output in a synchronous mode, at this time, the synchronous data processor 121 may encode the gray scale gradient image signals output by the rod cell circuit into asynchronous signals and output the asynchronous signals to the asynchronous data processor 122, and the asynchronous data processor 122 processes the asynchronous signals. In other embodiments, the output of the rod cell circuit may also be in an asynchronous mode, that is, the gray scale gradient image signal collected by the rod cell circuit is output in an asynchronous mode, that is, an asynchronous signal.
With continued reference to FIG. 3, the asynchronous data processor 122 is communicatively coupled to the image sensor 111 via a two-wire serial bus or a universal asynchronous receiver/transmitter.
The two-wire serial bus (Inter-Integrated Circuit, IIC) has the characteristics of simplicity and effectiveness, and the asynchronous data processor 122 and the image sensor 111 are connected through IIC communication, so that the complexity of connection can be simplified and the cost can be reduced on the basis of meeting the data transmission requirement. Similarly, a Universal Asynchronous Receiver/Transmitter (UART) may implement Asynchronous communication between the Asynchronous data processor 122 and the image sensor 111.
On the basis of the technical schemes, the rod cell circuit comprises an excitatory rod cell circuit and an inhibitory rod cell circuit; the difference between the gradation gradient image signal collected by the excitatory rod cell circuit and the gradation gradient image signal collected by the inhibitory rod cell circuit is used as the gradation gradient image signal of the rod cell circuit.
Wherein one rod circuit may comprise one excitatory rod circuit and a plurality of inhibitory rod circuits. The inhibitory rod cell circuit may be disposed around the excitatory rod cell circuit, and when the rod cell circuit operates, a photocurrent generated by the excitatory rod cell circuit and a photocurrent generated by the inhibitory rod cell circuit are differentiated and quantized by a current DA, thereby forming a gray scale gradient image signal of the rod cell circuit.
In forming the gradation gradient image signal of the rod cell circuit, the difference between the photocurrent generated by the excitatory rod cell circuit and the photocurrent generated by any inhibitory rod cell circuit can be adaptively adjusted according to the intensity of the external light. For example, when the external light intensity is relatively strong, such as the case of direct sunlight, the photocurrent generated by the excitatory rod cell circuit may be different from the photocurrent generated by all the inhibitory rod cell circuits, so that the rod cell circuit may adapt to a scene with relatively strong light intensity. When the external light intensity is weak, for example, at night, the photocurrent generated by the excitatory rod cell circuit is different from the photocurrent generated by the zero inhibitory rod cell circuits, that is, the gray gradient image signal of the rod cell circuit is determined by the photocurrent generated by the excitatory rod cell circuit, so that the rod cell circuit can adapt to the scene with weak light intensity. When the external light intensity is moderate, such as in cloudy or cloudy days, the photocurrent generated by the excitatory rod cell circuit is different from the photocurrent generated by any inhibitory rod cell circuit, and the photocurrent generated by the excitatory rod cell circuit can make the rod cell circuit adapt to the scene with moderate light intensity.
Fig. 4 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention. As shown in fig. 4, the image sensor unit 110 further includes a previous data processing unit 112; the pre-stage data processing unit 112 is connected to the image sensor 111 for packetizing and compressing the bimodal image signal.
The front-end data processing unit 112 may receive the bimodal image signal output by the image sensor 111, perform front-end data processing on the bimodal image signal, for example, may perform preliminary packing and compression on the bimodal image signal, and then transmit the bimodal image signal to the asynchronous data processor 122, where the asynchronous data processor 122 further performs data processing on the bimodal image signal after the front-end data processing, so that the asynchronous signal output by the asynchronous data processor 122 satisfies an asynchronous signal format of image display.
In addition, the image sensor unit 110 may further include a power supply module that supplies power to other units on the image sensor unit 110.
Fig. 5 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention. As shown in fig. 5, the asynchronous data processor 122 comprises a field programmable gate array.
The Field-Programmable Gate Array (FPGA) has a flexible logic unit, and has the characteristics of high integration level and high programming flexibility, so that the time for designing an image processing program can be shortened.
With continued reference to FIG. 5, the asynchronous data processor 122 includes a rod data processor 1221; the rod data processor 1221 is used to compress the asynchronous signal in the bimodal image signal and encode it into an asynchronous mode, forming an address form event signal.
The gray scale gradient image signal collected by the rod cell circuit in the image sensor 111 is an asynchronous signal in the bimodal image signal. When the bimodal image signal is transmitted to the asynchronous data processor 122, the rod data processor 1221 greatly compresses the gray scale gradient image signal collected by the rod cell circuit, and then encodes it into an asynchronous mode, forming an address form event signal. The address form event signal may be (X, Y, P, T), where "X, Y" is an event address, "P" is a 4-valued event output, and "T" is a time when an event is generated. Therefore, the information in the address form event signal comprises the event address, the event output, the event generation time and other characteristic information, in the subsequent process, the characteristic information in the address form event signal is added into the characteristics of the synchronous signal in the bimodal image signal in a combined mode, and then the image is displayed according to the characteristics of the combined bimodal image signal, so that the image can utilize the discrete type and the sparsity of the asynchronous signal, the efficiency of processing the bimodal image signal is improved, and meanwhile, the fidelity of the image can be ensured in high-speed shooting.
With continued reference to FIG. 5, asynchronous data processor 122 also includes a view frustum data processor 1222; the viewing cone data processor 1222 is used for auto-focus, auto-exposure, and auto-white balance of the synchronization signal in the bimodal image signal.
The automatic focusing utilizes the principle of object light reflection, and the reflected light is received by the image sensor 111, processed by the viewing cone data processor 1222, and drives the electric focusing device to focus. The automatic exposure is such that the image sensor 111 automatically sets the shutter speed and the aperture value in accordance with the shutter and aperture exposure combination set at the time of production, based on the exposure value of the image measured by the photometry system. The automatic white balance is that the image sensor 111 automatically detects the color temperature value of the object according to the light conditions of the lens and the white balance sensor, so as to judge the shooting condition, and selects the closest tone setting, and the color temperature setting is corrected by the color temperature correction circuit, and the white balance automatic control circuit automatically adjusts the white balance to a proper position.
The image color signal collected by the cone cell circuit in the image sensor 111 is a synchronization signal in the bimodal image signal. When the asynchronous data processor 122 includes the viewing cone data processor 1222, the asynchronous data processor 122 may also be used to process a synchronous signal in the bimodal image signal. When the bimodal image signal is transmitted to the asynchronous data processor 122, the cone data processor 1222 performs auto-focus, auto-exposure, and auto-white balance on the image color signal collected by the cone cell circuit, improving the fidelity of the bimodal image signal.
In addition, the cone data processor 1222 feeds back the image color signals collected by the cone cell circuit to the image sensor 111 after auto-focusing, auto-exposure and auto-white balancing, so as to adjust the exposure time and the lens focal length of the image sensor 111.
On the basis of the technical schemes, the synchronous data processor is connected with the asynchronous data processor; the synchronous data processor is further configured to configure parameters of the asynchronous data processor based on the synchronization signal.
The synchronous signal in the bimodal image signal can be transmitted to the synchronous data processor through the asynchronous data processor, and the synchronous data processor processes the synchronous signal. After the synchronous data processor receives the synchronous signal, the synchronous data processor can also determine a prediction output range of the bimodal image signal according to the synchronous signal, and then feed back and adjust parameters of the asynchronous data processor according to the prediction output range, so that the asynchronous data processor adjusts the exposure time and the focal length of the lens of the image sensor 111 according to the set parameters.
Illustratively, the synchronous data processor is at least one of an ARM processor, a central processing unit, an application specific integrated circuit, a system on a chip, and an image signal processor.
The synchronous data processor mainly performs calculation processing of a brain-like perception algorithm, and various processors can be used. Illustratively, the synchronous data processor may be an ARM processor, which has features of small size, low power consumption, low cost, and high performance. The CPU may also be a Central Processing Unit (CPU) including an arithmetic logic Unit, a register Unit, a control Unit, and the like, and having functions of Processing instructions, executing operations, controlling time, Processing data, and the like, and may be complete in function. The isochronous data processor may also be an Application Specific Integrated Circuit (ASIC), which is an integrated circuit fabricated for a particular user or a particular electronic system, and thus may be customized according to the functionality of the isochronous data processor. The synchronous data processor may also be a System On Chip (SOC), which is an integrated circuit with a dedicated target that contains the complete System and has the entire contents of embedded software, and may be formed according to the functions of the synchronous data processor. The synchronous data Processor may also be an Image Signal Processor (ISP), and the ISP may process signals output by the Image sensor to match Image sensors of different manufacturers.
It should be noted that, in other embodiments, the synchronous data processor may further receive an address form event signal formed by the asynchronous data processor, extract feature information in the address form event signal, add the feature information to the feature of the synchronous signal in a merging manner, and implement fusion of the synchronous signal and the asynchronous signal, so that the dual-mode signal processing system can utilize the discrete type and sparsity of the asynchronous signal, improve the efficiency of processing the dual-mode image signal, and simultaneously can implement ensuring the fidelity of the image in high-speed shooting.
Fig. 6 is a schematic structural diagram of another dual-mode signal processing system according to an embodiment of the present invention. As shown in fig. 6, the image processing unit 120 further includes a storage module 123; the storage module 123 is used for buffering the bimodal image signal.
The storage module 123 is configured to provide a storage space for storing the bimodal image signal. The storage module 123 may be connected to the image sensor 111, the image sensor 111 transmits the bimodal image signals to the storage module 123 through the high-speed data transmission interface 130, and the storage module 123 buffers the bimodal image signals. The storage modality 123 is further connected to the asynchronous data processor 122, and the cached bimodal image signal is transmitted to the asynchronous data processor 122, so as to avoid data loss and pause or data overflow phenomena caused by data asynchronism between the image sensor 111 and the asynchronous data processor 122.
In addition, in other embodiments, the image processing unit 120 may further include an external interface, which is used to connect with an external device, and may implement data transmission between the image processing unit 120 and the external device. Illustratively, the external interface may be a Universal Serial Bus (USB) interface, or may be an Ethernet interface.
The embodiment of the invention also provides a bimodal signal processing method. Fig. 7 is a flowchart of a bimodal signal processing method according to an embodiment of the present invention. The method can be executed by the dual-mode signal processing system provided by the embodiment of the invention and is used for processing the dual-mode image signal. As shown in fig. 7, the method includes:
s10, acquiring a bimodal image signal through an image sensor unit;
s11, processing the asynchronous signal in the bimodal image signal by the asynchronous data processor in the image processing unit, and simultaneously processing the synchronous signal in the bimodal image signal by the synchronous data processor in the image processing unit.
The bimodal image signal comprises a synchronous signal and an asynchronous signal, the asynchronous signal is processed by the asynchronous data processor, and the synchronous signal is processed by the synchronous data processor, so that the synchronous signal and the asynchronous signal in the bimodal image signal can be processed by the image processing unit at the same time, the efficiency of the image processing unit for processing the bimodal image signal can be improved, the efficiency and the real-time performance of the bimodal signal processing system are improved, and the application range of the bimodal signal processing system is enlarged.
On the basis of the above technical solution, the image sensor unit includes an image sensor, and before acquiring the bimodal image signal by the image sensor unit, the method further includes:
parameters of the image sensor are configured by the asynchronous data processor.
The asynchronous data processor is connected with the image sensor, and before the image sensor works, the asynchronous data processor can configure parameters of the image sensor, so that the image sensor can normally acquire bimodal image signals.
On the basis of the above technical solutions, the bimodal signal processing method further includes:
the synchronous data processor configures parameters of the asynchronous data processor according to the synchronous signal.
After the synchronous data processor receives the synchronous signal, the synchronous data processor can determine a prediction output range of the bimodal image signal according to the synchronous signal, and then the prediction output range feeds back and adjusts parameters of the asynchronous data processor, so that the asynchronous data processor adjusts the exposure time and the focal length of the lens of the image sensor according to the set parameters.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (17)
1. A dual-modality signal processing system, comprising:
an image sensor unit for acquiring a bimodal image signal;
an image processing unit connected with the image sensor unit, the image processing unit includes a synchronous data processor and an asynchronous data processor, the asynchronous data processor is used for processing asynchronous signals in the bimodal image signals, and simultaneously the synchronous processor is used for processing synchronous signals in the bimodal image signals.
2. The dual-modality signal processing system of claim 1, wherein the image sensor unit and the image processing unit are connected by a high-speed data transmission interface for transmitting the dual-modality image signal.
3. The dual-modality signal processing system of claim 1, wherein the image sensor unit includes an image sensor, the asynchronous data processor is communicatively coupled to the image sensor, the asynchronous data processor is further configured to configure parameters of the image sensor.
4. The dual-modality signal processing system of claim 3, wherein the asynchronous data processor is communicatively coupled to the image sensor via a two-wire serial bus or a universal asynchronous receiver transmitter.
5. The dual-modality signal processing system of claim 3, wherein the image sensor includes a cone circuit and a rod circuit; the cone cell circuit is used for collecting image color signals to form synchronous signals in the bimodal image signals, and the rod cell circuit is used for collecting gray gradient image signals to form asynchronous signals in the bimodal image signals.
6. The dual-modality signal processing system of claim 5, wherein the parameters of the image sensor include AD scan range, reset time length, shutter speed and sampling speed in the cone-cell circuit, and DA current output range and speed in the rod-cell circuit.
7. The dual-modality signal processing system of claim 5, wherein the rod circuit includes an excitatory rod circuit and an inhibitory rod circuit; the difference between the gray scale gradient image signal collected by the excitatory rod cell circuit and the gray scale gradient image signal collected by the inhibitory rod cell circuit is used as the gray scale gradient image signal of the rod cell circuit.
8. The dual-modality signal processing system of claim 3, wherein the image sensor unit further comprises a pre-stage data processing unit;
the early data processing unit is connected with the image sensor and used for packing and compressing the bimodal image signals.
9. The dual-modality signal processing system of claim 1, wherein the synchronous data processor is connected to the asynchronous data processor; the synchronous data processor is further configured to configure parameters of the asynchronous data processor according to the synchronous signal.
10. The dual-modality signal processing system of any of claims 1-9, wherein the synchronous data processor is at least one of an ARM processor, a central processing unit, an application specific integrated circuit, a system-on-a-chip, and an image signal processor.
11. The dual-modality signal processing system of any of claims 1-9, wherein the asynchronous data processor comprises a field programmable gate array.
12. The dual-modality signal processing system of claim 11, wherein the asynchronous data processor comprises a rod data processor;
the rod data processor is used for compressing the asynchronous signals in the bimodal image signals, encoding the signals into an asynchronous mode and forming address form event signals.
13. The dual-modality signal processing system of claim 12, wherein the asynchronous data processor further comprises a cone data processor;
the viewing cone data processor is used for carrying out automatic focusing, automatic exposure and automatic white balance on the synchronous signals in the bimodal image signals.
14. The dual-modality signal processing system of claim 1, wherein the image processing unit further comprises a storage module; the storage module is used for caching the bimodal image signal.
15. A bimodal signal processing method, comprising:
acquiring a bimodal image signal through an image sensor unit;
and processing the asynchronous signal in the bimodal image signal through an asynchronous data processor in an image processing unit, and simultaneously processing the synchronous signal in the bimodal image signal through a synchronous data processor in the image processing unit.
16. The bimodal signal processing method according to claim 15, wherein said image sensor unit comprises an image sensor, further comprising, before acquiring the bimodal image signal by the image sensor unit:
configuring, by the asynchronous data processor, parameters of the image sensor.
17. The bimodal signal processing method of claim 15, further comprising:
and the synchronous data processor configures the parameters of the asynchronous data processor according to the synchronous signal.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011019239.5A CN112188092B (en) | 2020-09-24 | 2020-09-24 | Bimodal signal processing system and method |
| TW110135000A TWI802015B (en) | 2020-09-24 | 2021-09-17 | Dual-mode signal processing system and method |
| PCT/CN2021/119162 WO2022063060A1 (en) | 2020-09-24 | 2021-09-17 | System and method for processing dual-modal signal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011019239.5A CN112188092B (en) | 2020-09-24 | 2020-09-24 | Bimodal signal processing system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112188092A true CN112188092A (en) | 2021-01-05 |
| CN112188092B CN112188092B (en) | 2022-09-30 |
Family
ID=73943662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011019239.5A Active CN112188092B (en) | 2020-09-24 | 2020-09-24 | Bimodal signal processing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112188092B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022063060A1 (en) * | 2020-09-24 | 2022-03-31 | 北京灵汐科技有限公司 | System and method for processing dual-modal signal |
| WO2022135359A1 (en) * | 2020-12-21 | 2022-06-30 | 北京灵汐科技有限公司 | Dual-mode image signal processor and dual-mode image signal processing system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190081114A1 (en) * | 2017-09-13 | 2019-03-14 | Essential Products, Inc. | Display and a light sensor operable as an infrared emitter and infrared receiver |
| CN110365912A (en) * | 2015-04-20 | 2019-10-22 | 三星电子株式会社 | Imaging unit, system and image sensor cell |
| CN111083402A (en) * | 2019-12-24 | 2020-04-28 | 清华大学 | Bimodal bionic vision sensor |
| CN111683234A (en) * | 2020-06-04 | 2020-09-18 | 深圳开立生物医疗科技股份有限公司 | Endoscope imaging method and device and related equipment |
-
2020
- 2020-09-24 CN CN202011019239.5A patent/CN112188092B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110365912A (en) * | 2015-04-20 | 2019-10-22 | 三星电子株式会社 | Imaging unit, system and image sensor cell |
| US20190081114A1 (en) * | 2017-09-13 | 2019-03-14 | Essential Products, Inc. | Display and a light sensor operable as an infrared emitter and infrared receiver |
| CN111083402A (en) * | 2019-12-24 | 2020-04-28 | 清华大学 | Bimodal bionic vision sensor |
| CN111683234A (en) * | 2020-06-04 | 2020-09-18 | 深圳开立生物医疗科技股份有限公司 | Endoscope imaging method and device and related equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022063060A1 (en) * | 2020-09-24 | 2022-03-31 | 北京灵汐科技有限公司 | System and method for processing dual-modal signal |
| WO2022135359A1 (en) * | 2020-12-21 | 2022-06-30 | 北京灵汐科技有限公司 | Dual-mode image signal processor and dual-mode image signal processing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112188092B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112188093B (en) | Bimodal signal fusion system and method | |
| US11461910B2 (en) | Electronic device for blurring image obtained by combining plural images based on depth information and method for driving the electronic device | |
| CN107566747B (en) | Image brightness enhancement method and device | |
| US20220224825A1 (en) | Electronic device for recording image as per multiple frame rates using camera and method for operating same | |
| EP3435655B1 (en) | Electronic device for acquiring image using plurality of cameras and method for processing image using the same | |
| CN112188092B (en) | Bimodal signal processing system and method | |
| WO2016192627A1 (en) | Imaging device and method of zoom lens | |
| CN109688382B (en) | Underwater image processing system of underwater robot | |
| KR20190010322A (en) | Electronic device and method for compressing image thereof | |
| CN111670573A (en) | Electronic device for photographing moving picture by adjusting threshold associated with movement of object in region of interest according to movement of electronic device and operating method thereof | |
| KR20190025373A (en) | Method for controlling synchronization of multiple image sensors and electronic device implementing the same | |
| US9389678B2 (en) | Virtual image signal processor | |
| US11503223B2 (en) | Method for image-processing and electronic device | |
| KR20140080815A (en) | Photographing apparatus, method for controlling the same, and computer-readable storage medium | |
| CN112822371A (en) | Image processing chip, application processing chip, data statistical system and method | |
| KR20190030946A (en) | Electronic device and method for controlling a plurality of image sensors | |
| CN112087569B (en) | Camera and camera starting method and device | |
| US20220141374A1 (en) | Intelligent flash intensity control systems and methods | |
| CN109309784B (en) | Mobile terminal | |
| CN112702588B (en) | Dual-mode image signal processor and dual-mode image signal processing system | |
| KR102301940B1 (en) | Method and apparatus for image fusion | |
| TWI802015B (en) | Dual-mode signal processing system and method | |
| CN113873142A (en) | Multimedia processing chip, electronic device and dynamic image processing method | |
| CN113766142B (en) | Image processing apparatus, image signal preprocessing module, device, and processing method | |
| WO2022135359A1 (en) | Dual-mode image signal processor and dual-mode image signal processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |