CN114339089A - Event image output method, device and equipment and readable storage medium - Google Patents
Event image output method, device and equipment and readable storage medium Download PDFInfo
- Publication number
- CN114339089A CN114339089A CN202111635124.3A CN202111635124A CN114339089A CN 114339089 A CN114339089 A CN 114339089A CN 202111635124 A CN202111635124 A CN 202111635124A CN 114339089 A CN114339089 A CN 114339089A
- Authority
- CN
- China
- Prior art keywords
- event
- voltage
- time
- binary vector
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 239000013598 vector Substances 0.000 claims abstract description 84
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 abstract description 8
- 230000008859 change Effects 0.000 abstract description 7
- 238000013500 data storage Methods 0.000 abstract description 6
- 230000004044 response Effects 0.000 abstract description 5
- 230000003287 optical effect Effects 0.000 description 8
- 230000001960 triggered effect Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 1
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003990 capacitor Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
Images
Landscapes
- Studio Devices (AREA)
Abstract
The application provides an event image output method, an event image output device and a readable storage medium, wherein the method comprises the following steps: generating corresponding real-time voltage according to the incident light intensity of each pixel in a pixel array of the image sensor; after each real-time voltage is differed from a preset reference voltage, comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value respectively; correspondingly generating a binary vector of each pixel according to the comparison result; and outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair. Through the implementation of the scheme, the event of a specific time point and a specific pixel position is recorded in response to the scene brightness change, an event stream is formed instead of a frame stream, the data redundancy is greatly reduced compared with the traditional camera, the hardware requirements for data storage and processing are correspondingly reduced, and effective data can be effectively output under an extreme light environment.
Description
Technical Field
The present application relates to the field of computer vision technologies, and in particular, to an event image output method, apparatus, device, and readable storage medium.
Background
With the continuous development of scientific technology, the computer vision technology is more and more mature. The current mainstream image acquisition technology is realized based on a traditional camera provided with a CMOS image sensor, the CMOS image sensor has the advantages that the acquired real original image of a scene captured by the camera is high in data volume and redundancy, the data storage and processing of the image sensor have higher requirements on hardware, in addition, the traditional camera cannot accurately acquire image information in a fast moving scene and environment light, and the problems need to be solved urgently.
Disclosure of Invention
The embodiment of the application provides an event image output method, an event image output device, event image output equipment and a readable storage medium, which can at least solve the problems that the image processing technology based on a CMOS image sensor provided in the related technology has high requirements on hardware and the applicable scene is limited.
A first aspect of an embodiment of the present application provides an event image output method, applied to an event camera, including:
generating corresponding real-time voltage according to the incident light intensity of each pixel in a pixel array of the image sensor;
after each real-time voltage is different from a preset reference voltage, comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value respectively; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0;
correspondingly generating a binary vector of each pixel according to the comparison result; when the voltage difference is smaller than the first voltage threshold, a value of a first numerical value in the binary vector is 0, and when the voltage difference is larger than the second voltage threshold, a value of a second numerical value in the binary vector is 0;
and outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair.
A second aspect of the embodiments of the present application provides an event image output apparatus, applied to an event camera, including:
the first generation module is used for generating corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor;
the comparison module is used for comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value after each real-time voltage is different from a preset reference voltage; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0;
the second generating module is used for correspondingly generating a binary vector of each pixel according to the comparison result; when the voltage difference is smaller than the first voltage threshold, a value of a first numerical value in the binary vector is 0, and when the voltage difference is larger than the second voltage threshold, a value of a second numerical value in the binary vector is 0;
and the output module is used for outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair.
A third aspect of the embodiments of the present application provides a terminal device, including: the event image output method includes a memory and a processor, where the processor is configured to execute a computer program stored on the memory, and when the processor executes the computer program, the processor implements the steps of the event image output method provided by the first aspect of the embodiment of the present application.
A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the event image output method provided in the first aspect of the embodiments of the present application.
As can be seen from the above, according to the event image output method, apparatus, device and readable storage medium provided in the present application, the corresponding real-time voltage is generated according to the incident light intensity of each pixel in the pixel array of the image sensor; after each real-time voltage is differed from a preset reference voltage, comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value respectively; correspondingly generating a binary vector of each pixel according to the comparison result; and outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair. Through the implementation of the scheme, the event of a specific time point and a specific pixel position is recorded in response to the scene brightness change, an event stream is formed instead of a frame stream, the data redundancy is greatly reduced compared with the traditional camera, the hardware requirements for data storage and processing are correspondingly reduced, and effective data can be effectively output under an extreme light environment.
Drawings
Fig. 1 is a schematic basic flowchart of an event image output method according to a first embodiment of the present application;
fig. 2 is a schematic diagram of a pixel array composed of a plurality of pixels according to a first embodiment of the present application;
fig. 3 is a schematic flowchart illustrating a detailed process of an event image output method according to a second embodiment of the present application;
fig. 4 is a schematic diagram of program modules of an event image output device according to a third embodiment of the present application;
fig. 5 is a schematic structural diagram of a terminal device according to a fourth embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the embodiments of the present application, it is to be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.
In the embodiments of the present application, unless otherwise specifically stated or limited, the terms "mounted," "connected," and "fixed" are to be construed broadly, e.g., as meaning fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the embodiments of the present application can be understood by those of ordinary skill in the art according to specific situations.
The above description is only exemplary of the present application and should not be taken as limiting the invention, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
In order to solve the problems that the image processing technology based on the CMOS image sensor provided in the related art has a high requirement on hardware and the applicable scene is limited, a first embodiment of the present application provides an Event image output method, which is applied to an Event Camera (Event-based Camera), where each pixel of the Event Camera works independently and outputs an Event only when the luminance change of a certain pixel reaches a certain threshold.
Fig. 1 is a schematic basic flow chart of an event image output method provided in this embodiment, where the event image output method includes the following steps:
Specifically, the event camera is provided with an image sensor including a pixel array composed of a plurality of pixels, each pixel sensor in the pixel array being an integrated circuit in which a photodiode may be integrated with a capacitor accumulating charges, which generates a photocurrent by the photodiode in response to an incident light intensity, and then generates a real-time voltage according to the photocurrent, accordingly.
Specifically, in this embodiment, the reference voltage is subtracted from each real-time voltage to obtain a voltage difference, and then whether the intensity of the incident light changes (becomes stronger or weaker) is determined based on the voltage difference.
It should be noted that the reference voltage may be a preset fixed voltage, or may be a voltage that is updated flexibly. In addition, in a preferred embodiment of the present invention, the first voltage threshold and the second voltage threshold may be opposite numbers, for example, the value of the first voltage threshold is 0.1V, and the value of the second voltage threshold is-0.1V.
In an implementation manner of this embodiment, the step of subtracting each real-time voltage from the preset reference voltage includes: and according to the generation time sequence of each real-time voltage, carrying out time-sharing difference on each real-time voltage and a preset reference voltage.
Specifically, in this embodiment, the real-time voltage of each pixel may be sent to the first comparison circuit and the second comparison circuit in a time-sharing manner, so as to obtain the relationship between the real-time voltage signal of each pixel and the reference voltage signal, and then compare the relationship with the preset voltage threshold.
And 103, correspondingly generating a binary vector of each pixel according to the comparison result.
Specifically, in practical applications, each pixel generates a corresponding 2-bit vector, and the 2-bit vector is used to characterize whether the optical signal is strong or weak. When the voltage difference is smaller than the first voltage threshold, the value of the first numerical value in the binary vector is 0, and when the voltage difference is larger than the second voltage threshold, the value of the second numerical value in the binary vector is 0. In this embodiment, the binary vector may be represented in a form of a pair of [ a, b ], where a is determined by a comparison relationship between a voltage difference and a first voltage threshold, and b is determined by a comparison relationship between a voltage difference and a second voltage threshold, when the voltage difference is greater than the first voltage threshold, a takes a non-zero value, and when the voltage difference is smaller than the second voltage threshold, b takes a non-zero value, in a preferred embodiment of this embodiment, the non-zero values of a and b may be both 1, and of course, in other embodiments, may also be 1 and-1, respectively.
Next, the present embodiment exemplifies a generation manner of the binary vector, and it is assumed that the reference voltage Vm is 0.5V, and the first voltage threshold a1 and the second voltage threshold a2 are 0.1V and-0.1V, respectively. When the real-time voltage Vin is 0.7V, the voltage difference value Vin-Vm is greater than the first voltage threshold A1, at this time, a UP event is triggered, a vector value 1 is output, and the voltage difference value Vin-Vm is greater than the second voltage threshold A2, a DN event cannot be triggered, a vector value 0 is output, and then a 2-bit vector finally output according to a comparison result is [1, 0 ]; when the real-time voltage Vin is 0.5V, the voltage difference value Vin-Vm is smaller than the first voltage threshold A1, at this time, the UP event cannot be triggered, the vector value 0 is output, and the voltage difference value Vin-Vm is larger than the second voltage threshold A2, the DN event cannot be triggered, the vector value 0 is output, and then the finally output 2-bit vector is [0, 0] according to the comparison result; and when the real-time voltage Vin is 0.3V, the voltage difference value Vin-Vm is smaller than the first voltage threshold A1, at this time, the UP event is not triggered, the vector value 0 is output, and the voltage difference value Vin-Vm is smaller than the second voltage threshold A2, the DN event is triggered, the vector value 1 is output, and then the finally output 2-bit vector is [0, 1] according to the comparison result. It should be understood that the optical signal is characterized as unchanged when the 2-bit vector is all-zero pairs [0, 0], and is characterized as being either stronger or weaker when the 2-bit vector is a non-all-zero pair [1, 0] or [0, 1 ]. Of course, in practical applications it is not excluded that the system may be in error, outputting a 2-bit vector [1, 1], such information being generally considered as invalid events.
In an embodiment of this embodiment, after the step of generating the binary vector of each pixel according to the comparison result, the method further includes: and when the binary vector is a non-all-zero pair, updating the reference voltage in real time.
Specifically, in this embodiment, the real-time voltages of the pixels are sent to the comparison circuit in a time-sharing manner for voltage comparison, and voltage attenuation may occur due to leakage or other conditions in the circuit that stores the reference voltage after a period of time, so that when an event is detected because the 2-bit vector is non-all-zero data, global reset is performed, the reference voltage is reset, and the false UP or DN event generated by the abnormal condition is conveniently removed.
Further, in an implementation manner of this embodiment, before the step of updating the reference voltage, the method further includes: and counting the event generation rate in unit time length before the corresponding voltage generation time of the binary vector. Correspondingly, when the event generation rate is greater than the preset event generation rate threshold value, the step of updating the reference voltage in real time is executed.
Still further, preferably, the step of updating the reference voltage in real time includes: and updating the reference voltage to a corresponding real-time voltage of the target pixel in real time.
Specifically, in practical application, the data generated by the event camera is an asynchronous event stream, rather than a frame stream of a conventional image, and although the event stream has the advantage of high time resolution, the event stream may also have slow or even erroneous reading due to errors or data overload in event stream data transmission, so that in one solution, when the event generation rate is high in the historical duration and data overload may occur, the reference voltage may be globally reset, the reference voltage is adjusted to the real-time voltage of the current pixel triggering an event, and the sensitivity of the pixel to the change of the optical signal is dynamically reduced, so as to filter the low-frequency signal, limit the generation rate of the event in the subsequent unit time, and achieve the effect of controlling the bandwidth.
For example, assuming that the preset reference voltage is 0.5V, when the real-time voltage is 0.7V, the first voltage threshold is 0.1V, the second voltage threshold is-0.1V, at this time, the first comparison circuit generates an UP event and outputs 1, the second comparison circuit does not generate a DN event and outputs 0, then a 2-bit vector [1, 0] is output according to the comparison result, and at this time, the reference voltage is updated to 0.7V by the real-time voltage; similarly, if the real-time voltage is 0.3V, the first comparison circuit does not generate an UP event and outputs 0, the second comparison circuit generates a DN event and outputs 1, then a 2-bit vector [0, 1] is output according to the comparison result, and the reference voltage setting is updated to 0.3V by the real-time voltage signal; in addition, when the 2-bit vector of the pixel is not [0, 1] or not [1, 0], the reference voltage can be selected not to be updated.
And 104, outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair.
Specifically, when the binary vector output by the pixel in the pixel array is a non-all-zero number pair [1, 0], an UP event is generated, and when the binary vector output by the pixel in the pixel array is a non-all-zero number pair [0, 1], a DN event is generated. Fig. 2 is a schematic diagram of a pixel array provided in this embodiment, where the size of the pixel array represents the resolution of an event camera, and this embodiment only shows a 9 × 10 matrix for convenience of description, but in practical applications, the pixel array is actually much higher than 9 × 10, such as 480 × 640, and the like, where a value "+ 1" represents an UP event corresponding to a binary vector [1, 0], a value "-1" represents a DN event corresponding to a binary vector [0, 1], and a value "0" represents no event. This embodiment scans the entire pixel array, reads out pixels whose binary vectors are [0, 1] or [1, 0] in the entire pixel array, embeds pixel coordinate position information and time stamp information into these pixels, generates event data, and then outputs an event image based on the acquired event data.
In an embodiment of the present invention, the step of outputting an event image based on event data of a target pixel whose binary vector is a non-all-zero number pair includes: storing the event data of the target pixel of which each binary vector is a non-all-zero number pair in a time-sharing manner; when all event data are stored, a corresponding event data reading strategy is formulated according to the system reading performance; correspondingly reading the event data according to the event data reading strategy; and outputting an event image according to the read event data.
Specifically, the present embodiment performs data storage in a time-sharing manner for different binary vectors, and then adaptively reads event data according to system performance after all event data are stored, where the event data reading policy of the present embodiment includes an overall reading policy, a batch reading policy, and a partial reading policy, where the overall reading policy is to uniformly read all event data at one time, the batch reading policy is to read all data in steps, and the partial reading policy is to select only a part of data for reading, so that bandwidth can be effectively reduced and event data reading errors can be reduced under the condition of data overload.
In another embodiment of this embodiment, the step of outputting the event image based on the event data of the target pixel whose binary vector is a non-all-zero number pair includes: respectively generating first event images based on event data of target pixels of which the binary vectors are non-all-zero pairs; and after all the first event images are fused, outputting a second event image.
Specifically, in the present embodiment, an a image may be output according to the event data of the a pixel at time t1, a B image may be output according to the event data of the B pixel at time t2, a C image may be output according to the event data of the C pixel at time t3, and a D image may be output according to the event data of the D pixel at time t 4. In one image output solution, one or more of the a image, the B image, the C image and the D image can be selected and output, and in a more preferable solution, the a image, the B image, the C image and the D image can be fused to output a complete event image with higher resolution.
Based on the technical scheme of the embodiment of the application, the corresponding real-time voltage is generated according to the incident light intensity of each pixel in the pixel array of the image sensor; after each real-time voltage is differed from a preset reference voltage, comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value respectively; correspondingly generating a binary vector of each pixel according to the comparison result; and outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair. Through the implementation of the scheme, the event of a specific time point and a specific pixel position is recorded in response to the scene brightness change, an event stream is formed instead of a frame stream, the data redundancy is greatly reduced compared with the traditional camera, the hardware requirements for data storage and processing are correspondingly reduced, and effective data can be effectively output under an extreme light environment.
The method in fig. 3 is a refined event image output method provided in a second embodiment of the present application, and the event image output method includes:
And 302, according to the generation time sequence of each real-time voltage, performing time-sharing difference on each real-time voltage and a preset reference voltage.
In this embodiment, the first voltage threshold is greater than 0, the second voltage threshold is less than 0, the first voltage threshold and the second voltage threshold may be opposite numbers, when the voltage difference is smaller than the first voltage threshold, the value of the first value in the binary vector is 0, and when the voltage difference is greater than the second voltage threshold, the value of the second value in the binary vector is 0.
And step 304, counting the event generation rate in unit historical time length when the binary vector is a non-all-zero number pair.
Specifically, when the 2-bit vector is a non-all-zero number pair [1, 0], the representation optical signal becomes strong, and an UP event is triggered, and when the 2-bit vector is a non-all-zero number pair [0, 1], the representation optical signal becomes weak, and a DN event is triggered.
In this embodiment, when the event generation rate is large in the historical duration and data overload may occur, the reference voltage may be reset globally, the reference voltage is adjusted to the real-time voltage of the pixel currently triggering the event, and the sensitivity of the pixel to the optical signal change is dynamically adjusted down to filter the low-frequency signal, limit the generation rate of the event in the subsequent unit time, and achieve the effect of controlling the bandwidth.
And 307, outputting a second event image after all the first event images are fused.
Specifically, the present embodiment scans the entire pixel array, reads out pixels whose binary vectors are [0, 1] or [1, 0] in the entire pixel array, embeds pixel coordinate position information and time stamp information in these pixels, generates event data, and outputs an event image based on the acquired event data. In practical application, the embodiment performs image fusion on the event images corresponding to all event data, and finally outputs the event images with higher quality.
It should be understood that, the size of the serial number of each step in this embodiment does not mean the execution sequence of the step, and the execution sequence of each step should be determined by its function and inherent logic, and should not be limited uniquely to the implementation process of the embodiment of the present application.
Fig. 4 is an event image output device according to a third embodiment of the present application. The event image output apparatus can be used to implement the event image output method in the foregoing embodiments. As shown in fig. 4, the event image output apparatus mainly includes:
a first generating module 401, configured to generate a corresponding real-time voltage according to an incident light intensity of each pixel in a pixel array of the image sensor;
a comparing module 402, configured to compare each voltage difference with a preset first voltage threshold and a preset second voltage threshold after each real-time voltage is differentiated from a preset reference voltage; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0;
a second generating module 403, configured to generate a binary vector for each pixel according to the comparison result; when the voltage difference value is smaller than a first voltage threshold value, the value of a first numerical value in the binary vector is 0, and when the voltage difference value is larger than a second voltage threshold value, the value of a second numerical value in the binary vector is 0;
an output module 404, configured to output an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair.
In some embodiments of the present embodiment, when the comparing module 402 performs the above-mentioned function of making the difference between each real-time voltage and the preset reference voltage, it is specifically configured to: and according to the generation time sequence of each real-time voltage, carrying out time-sharing difference on each real-time voltage and a preset reference voltage.
Further, in some embodiments of the present embodiment, the event image output apparatus further includes: and the updating module is used for updating the reference voltage in real time when the binary vector is a non-all-zero time pair.
Further, in some embodiments of the present embodiment, the event image output apparatus further includes: and the counting module is used for counting the event generation rate in unit time before the corresponding voltage generation time of the binary vector. Correspondingly, the update module is specifically configured to: and when the event generation rate is greater than a preset event generation rate threshold value, updating the reference voltage in real time.
In other embodiments of this embodiment, the update module is specifically configured to: and updating the reference voltage to a corresponding real-time voltage of the target pixel in real time.
In some embodiments of this embodiment, the output module is specifically configured to: storing the event data of the target pixel of which each binary vector is a non-all-zero number pair in a time-sharing manner; when all event data are stored, a corresponding event data reading strategy is formulated according to the system reading performance; correspondingly reading the event data according to the event data reading strategy; and outputting an event image according to the read event data.
In other embodiments of this embodiment, the output module is specifically configured to: respectively generating first event images based on event data of target pixels of which the binary vectors are non-all-zero pairs; and after all the first event images are fused, outputting a second event image.
It should be noted that, the event image output methods in the first and second embodiments can be implemented based on the event image output device provided in this embodiment, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the event image output device described in this embodiment may refer to the corresponding process in the foregoing method embodiment, and details are not described here again.
According to the event image output device provided by the embodiment, the corresponding real-time voltage is generated according to the incident light intensity of each pixel in the pixel array of the image sensor; after each real-time voltage is differed from a preset reference voltage, comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value respectively; correspondingly generating a binary vector of each pixel according to the comparison result; and outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair. Through the implementation of the scheme, the event of a specific time point and a specific pixel position is recorded in response to the scene brightness change, an event stream is formed instead of a frame stream, the data redundancy is greatly reduced compared with the traditional camera, the hardware requirements for data storage and processing are correspondingly reduced, and effective data can be effectively output under an extreme light environment.
Fig. 5 is a terminal device according to a fourth embodiment of the present application. The terminal device may be configured to implement the event image output method in the foregoing embodiment, and mainly includes:
a memory 501, a processor 502 and a computer program 503 stored on the memory 501 and executable on the processor 502, the memory 501 and the processor 502 being communicatively connected. The processor 502, when executing the computer program 503, implements the method of one or both of the previous embodiments. Wherein the number of processors may be one or more.
The Memory 501 may be a high-speed Random Access Memory (RAM) Memory or a non-volatile Memory (non-volatile Memory), such as a disk Memory. The memory 501 is used for storing executable program code, and the processor 502 is coupled to the memory 501.
Further, an embodiment of the present application also provides a computer-readable storage medium, where the computer-readable storage medium may be provided in an electronic device in the foregoing embodiments, and the computer-readable storage medium may be the memory in the foregoing embodiment shown in fig. 5.
The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the event image output method in the foregoing embodiments. Further, the computer-readable storage medium may be various media that can store program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a RAM, a magnetic disk, or an optical disk.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules is merely a division of logical functions, and an actual implementation may have another division, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a readable storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method of the embodiments of the present application. And the aforementioned readable storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In view of the above description of the event image output method, apparatus, device and readable storage medium provided by the present application, those skilled in the art will recognize that there may be variations in the embodiments and applications of the event image output method, apparatus, device and readable storage medium according to the teachings of the present application.
Claims (10)
1. An event image output method applied to an event camera, the event image output method comprising:
generating corresponding real-time voltage according to the incident light intensity of each pixel in a pixel array of the image sensor;
after each real-time voltage is different from a preset reference voltage, comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value respectively; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0;
correspondingly generating a binary vector of each pixel according to the comparison result; when the voltage difference is smaller than the first voltage threshold, a value of a first numerical value in the binary vector is 0, and when the voltage difference is larger than the second voltage threshold, a value of a second numerical value in the binary vector is 0;
and outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair.
2. The event image output method according to claim 1, wherein the step of subtracting each of the real-time voltages from a preset reference voltage comprises:
and according to the generation time sequence of each real-time voltage, carrying out time-sharing difference on each real-time voltage and a preset reference voltage.
3. The event image output method according to claim 2, wherein, after the step of generating the binary vector for each of the pixels in accordance with the comparison result, further comprising:
and when the binary vector is a non-all-zero pair, updating the reference voltage in real time.
4. The event image output method according to claim 3, characterized in that the step of updating the reference voltage is preceded by:
counting the event generation rate in unit time length before the corresponding voltage generation time of the binary vector;
and when the event generation rate is greater than a preset event generation rate threshold value, executing the step of updating the reference voltage in real time.
5. The event image output method according to claim 3, wherein the step of updating the reference voltage in real time includes:
and updating the reference voltage to a corresponding real-time voltage of the target pixel in real time.
6. The event image output method according to claim 2, wherein the step of outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair includes:
storing the event data of the target pixel of which each binary vector is a non-all-zero number pair in a time-sharing manner;
when all the event data are stored, establishing a corresponding event data reading strategy according to the system reading performance;
correspondingly reading the event data according to the event data reading strategy;
outputting an event image according to the read event data.
7. The event image output method according to any one of claims 1 to 6, wherein the step of outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair includes:
respectively generating first event images based on the event data of the target pixels of which the binary vectors are non-all-zero pairs;
and outputting a second event image after fusing all the first event images.
8. An event image output apparatus applied to an event camera, the apparatus comprising:
the first generation module is used for generating corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor;
the comparison module is used for comparing each voltage difference value with a preset first voltage threshold value and a preset second voltage threshold value after each real-time voltage is different from a preset reference voltage; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0;
the second generating module is used for correspondingly generating a binary vector of each pixel according to the comparison result; when the voltage difference is smaller than the first voltage threshold, a value of a first numerical value in the binary vector is 0, and when the voltage difference is larger than the second voltage threshold, a value of a second numerical value in the binary vector is 0;
and the output module is used for outputting an event image based on the event data of the target pixel of which the binary vector is a non-all-zero number pair.
9. A terminal device, comprising a memory and a processor, wherein:
the processor is configured to execute a computer program stored on the memory;
the processor, when executing the computer program, performs the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111635124.3A CN114339089A (en) | 2021-12-29 | 2021-12-29 | Event image output method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111635124.3A CN114339089A (en) | 2021-12-29 | 2021-12-29 | Event image output method, device and equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114339089A true CN114339089A (en) | 2022-04-12 |
Family
ID=81017118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111635124.3A Pending CN114339089A (en) | 2021-12-29 | 2021-12-29 | Event image output method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114339089A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114885099A (en) * | 2022-05-10 | 2022-08-09 | 深圳锐视智芯科技有限公司 | Event image output method and related device |
| CN115022621A (en) * | 2022-06-27 | 2022-09-06 | 深圳锐视智芯科技有限公司 | Event camera testing method, device and equipment and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190364230A1 (en) * | 2018-05-23 | 2019-11-28 | Samsung Electronics Co., Ltd. | Method of processing data for dynamic vision sensor, dynamic vision sensor performing the same and electronic device including the same |
| WO2020063332A1 (en) * | 2018-09-29 | 2020-04-02 | 华为技术有限公司 | Dynamic vision sensor |
| US20210174092A1 (en) * | 2018-08-14 | 2021-06-10 | Huawei Technologies Co., Ltd. | Image processing apparatus and method for feature extraction |
| CN113647095A (en) * | 2019-03-27 | 2021-11-12 | 苹果公司 | Sensor system architecture with feedback loop and multiple power states |
-
2021
- 2021-12-29 CN CN202111635124.3A patent/CN114339089A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190364230A1 (en) * | 2018-05-23 | 2019-11-28 | Samsung Electronics Co., Ltd. | Method of processing data for dynamic vision sensor, dynamic vision sensor performing the same and electronic device including the same |
| US20210174092A1 (en) * | 2018-08-14 | 2021-06-10 | Huawei Technologies Co., Ltd. | Image processing apparatus and method for feature extraction |
| WO2020063332A1 (en) * | 2018-09-29 | 2020-04-02 | 华为技术有限公司 | Dynamic vision sensor |
| CN113647095A (en) * | 2019-03-27 | 2021-11-12 | 苹果公司 | Sensor system architecture with feedback loop and multiple power states |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114885099A (en) * | 2022-05-10 | 2022-08-09 | 深圳锐视智芯科技有限公司 | Event image output method and related device |
| CN115022621A (en) * | 2022-06-27 | 2022-09-06 | 深圳锐视智芯科技有限公司 | Event camera testing method, device and equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109005366B (en) | Night scene shooting processing method and device for camera module, electronic equipment and storage medium | |
| CN114339089A (en) | Event image output method, device and equipment and readable storage medium | |
| WO2019105154A1 (en) | Image processing method, apparatus and device | |
| KR101916355B1 (en) | Photographing method of dual-lens device, and dual-lens device | |
| CN109348088B (en) | Image noise reduction method and device, electronic equipment and computer readable storage medium | |
| CN109089046B (en) | Image noise reduction method and device, computer readable storage medium and electronic equipment | |
| EP3200446A1 (en) | Method and apparatus for generating high dynamic range image | |
| CN111652829B (en) | Image fusion method and device, electronic equipment and storage medium | |
| CN105187728B (en) | Photographic method and device | |
| CN109993722A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN110198418A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN110708463B (en) | Focusing method, focusing device, storage medium and electronic equipment | |
| CN109559352B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| JP2020150377A5 (en) | ||
| CN114500868B (en) | EVS pixel working method and related device | |
| CN110717871A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN110740266B (en) | Image frame selection method and device, storage medium and electronic equipment | |
| WO2024104439A1 (en) | Image frame interpolation method and apparatus, device, and computer readable storage medium | |
| CN114257744A (en) | Image processing method, device and equipment and readable storage medium | |
| US11503223B2 (en) | Method for image-processing and electronic device | |
| CN114244970A (en) | Event signal processing method, device and equipment and readable storage medium | |
| CN115250349A (en) | Sensor testing method and related device | |
| CN109120846B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN113055601B (en) | Binocular shooting correction method, binocular shooting correction device and electronic equipment | |
| CN114827585A (en) | Sensor correction method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |