CN116757968B - Noise reduction method and device, chip, event imaging device and electronic equipment - Google Patents
Noise reduction method and device, chip, event imaging device and electronic equipment Download PDFInfo
- Publication number
- CN116757968B CN116757968B CN202311041178.6A CN202311041178A CN116757968B CN 116757968 B CN116757968 B CN 116757968B CN 202311041178 A CN202311041178 A CN 202311041178A CN 116757968 B CN116757968 B CN 116757968B
- Authority
- CN
- China
- Prior art keywords
- pixel
- event
- time interval
- noise reduction
- statistics
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000003384 imaging method Methods 0.000 title claims abstract description 27
- 239000011159 matrix material Substances 0.000 claims description 60
- 238000004891 communication Methods 0.000 claims description 10
- 230000000903 blocking effect Effects 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 9
- 238000001914 filtration Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 abstract description 7
- 230000007547 defect Effects 0.000 abstract 1
- 238000003672 processing method Methods 0.000 abstract 1
- 238000012216 screening Methods 0.000 abstract 1
- 238000012545 processing Methods 0.000 description 16
- 238000003860 storage Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000006467 substitution reaction Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 208000012260 Accidental injury Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003306 harvesting Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a noise reduction method and device, a chip, an event imaging device and electronic equipment. In order to solve the defect that the prior scheme for filtering background noise in the event imaging process is easy to filter the effective events together by mistake, the invention preliminarily screens the effective events based on a time domain processing method for dividing time intervals. The invention further judges whether the event to be judged is likely to belong to noise or not according to the fluctuation of the event quantity generated by the pixel generating the event to be judged and the neighborhood pixel thereof between the first time interval and the time interval before the first time interval, thereby realizing the technical effect of noise event preliminary screening. The invention combines time domain and space domain to process noise, and obtains the technical effects of accurately, effectively and rapidly filtering background noise. The invention is suitable for the fields of event cameras and neuromorphic chips.
Description
Technical Field
The present invention relates to a noise reduction method and apparatus, a chip, an event imaging apparatus, and an electronic device, and in particular, to a noise reduction method and apparatus for a background noise event, a chip, an event imaging apparatus, and an electronic device.
Background
The background noise event of the nerve morphology sensor is randomly generated, the calculation resource consumption for filtering background noise by some existing noise reduction technologies is large, the position information of the pixel points is not considered in some existing technologies, some effective events are filtered as background noise together, and the noise reduction accuracy is low.
One common type of noise reduction scheme is correlation filtering, and is to judge whether other events occur within a period of time and a range when the current event occurs, and if so, consider the event as an effective event. In some early noise reduction schemes, the time stamp of a pixel is stored in time stamp storage units corresponding to 8 neighboring pixels around, and then whether the difference between the current time stamp and the previous time stamp corresponding to the pixel is greater than a threshold is determined to obtain time domain support. However, although the calculation cost of the method is low, the filtering effect is not ideal, and only low-frequency background noise can be filtered, and in addition, the method has incomplete utilization of the spatial information and can lose the spatial information.
Prior art 1: CN116170702a. In the event imaging device in the prior art 1, through setting a threshold value to sense an event, whether the light intensity change of the pixel unit reaches the set threshold value is judged, the event is generated only when the light intensity change reaches the set threshold value, and otherwise, the event is filtered as background noise. The threshold is usually set small to ensure high sensitivity, but at the same time, the external tiny interference is easily captured into noise events, so that the noise is extremely sensitive, and the threshold is increased, so that the number of captured effective events in the process of moving an object is reduced, and the imaging effect is affected. The noise reduction accuracy of the scheme is low, and effective events are easy to be misjudged as background noise.
Based on this, there is a need in the art for a noise reduction method: the method can accurately and efficiently filter background noise and reduce the probability of false injury effective events.
Disclosure of Invention
In order to solve or alleviate some or all of the above technical problems, the present invention is implemented by the following technical solutions:
a noise reduction method for reducing noise of an event stream output from an event imaging device, the event stream including events to be discriminated generated by a first pixel, each event in the event stream including at least coordinate information and timestamp information, the noise reduction method comprising the steps of: judging whether the event belongs to a first time interval or not according to the timestamp information of the event to be judged; and if the event to be judged does not belong to the first time interval, judging whether the event to be judged is blocked or not according to fluctuation between statistics of the event generated by the first pixel in the first time interval and statistics of the event generated by the first pixel in the previous time interval of the first time interval.
In some embodiments, for an unblocked event to be distinguished, it is determined whether the unblocked event to be distinguished should be cleared based on whether the time stamps of the spatial neighbor pixels surrounding the first pixel provide sufficient support.
In some embodiments, if the event to be discriminated belongs to the first time interval, the statistics of the events generated by the first pixel in the first time interval is incremented by one.
In some embodiments, if the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the statistics of the events generated by the first pixel in the previous time interval of the first time interval is greater than or equal to a first threshold, the event to be discriminated is released to enter the next stage system.
In some class of embodiments, if the fluctuation between the statistics of events generated by the first pixel during the first time interval and the statistics of events generated by the first pixel during a time interval preceding the first time interval is less than a first threshold value: judging whether the fluctuation between statistics of events generated by more than a preset number of pixels in airspace neighbor pixels of the first pixel in the first time interval and the time interval before the first time interval is larger than or equal to a first threshold value, if so, releasing the event to be judged to enter a next-stage system, otherwise, blocking the event to be judged.
In some embodiments, if the statistics of events generated by the first pixel during the first time interval is zero, updating the state value of the corresponding first pixel in the state matrix to be the first state value; if the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the statistics of the events generated by the first pixel in the previous time interval of the first time interval is smaller than a first threshold value, updating the state value corresponding to the first pixel in the state matrix into a second state value; if the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the statistics of the events generated by the first pixel in the previous time interval of the first time interval is greater than or equal to a first threshold value, the state value corresponding to the first pixel in the state matrix is updated to be a third state value.
In some embodiments, if a fluctuation between statistics of events generated by more than a predetermined number of pixels in spatial neighboring pixels of the first pixel in the first time interval and a time interval preceding the first time interval is greater than or equal to a first threshold, updating a state value of the corresponding first pixel in the state matrix to a third state value.
In some embodiments, if the state value corresponding to the first pixel in the state matrix is the third state value, updating the flag value corresponding to the first pixel in the flag matrix to be the first flag value;
if the state value corresponding to the first pixel in the state matrix is the second state value, updating the mark value corresponding to the first pixel in the mark matrix to be the second mark value.
In some embodiments, if the first pixel has a first mark value corresponding to the first pixel in the mark matrix, the event to be distinguished is released to enter the next stage system, and if the first pixel has a second mark value corresponding to the first pixel in the mark matrix, the event to be distinguished is blocked.
In some embodiments, the next-stage system is a noise reduction system, and the noise reduction system counts the number of the airspace neighbor pixels with the time difference smaller than or equal to a predetermined threshold based on the time difference between the time stamp of the airspace neighbor pixel corresponding to the first pixel in the time stamp matrix and the time stamp of the airspace neighbor pixel corresponding to the first pixel in the time stamp matrix, if the number exceeds a second threshold, the event to be discriminated is released, otherwise, the event to be discriminated is blocked.
In some embodiments, if the state value corresponding to the first pixel in the state matrix is the third state value, the event to be distinguished is released to enter the next stage system, and if the state value corresponding to the first pixel in the state matrix is the second state value, the event to be distinguished is blocked.
In some embodiments, after the statistics of the events generated by the first pixel in the first time interval is increased by one, a determination is made as to whether to release or block the event to be determined according to a corresponding flag value of the first pixel in the flag matrix or a corresponding state value of the first pixel in the state matrix.
In some embodiments, the length of the first time interval is the same as the length of the previous time interval of the first time interval, and the length of the first time interval is the same as the length of the next time interval of the first time interval.
A noise reduction device using any one of the noise reduction methods as described above.
An event imaging device comprising a noise reduction device as described above.
A chip having an event imaging device as hereinbefore described disposed.
An electronic device having a chip as described above, or an event imaging device as described above, or a noise reduction device as described above disposed thereon.
In certain classes of embodiments, the electronic device further comprises a computing module, and one or more of the following: the system comprises an energy collection module, a control module and a communication module.
Some or all embodiments of the present invention have the following beneficial technical effects:
1) According to the invention, the random noise distribution characteristic is found through the test, the novel noise reduction method is provided, the background noise can be accurately filtered, the probability of accidental injury effective events is greatly reduced, and the accuracy of the event imaging device such as DVS is improved.
2) The noise reduction method can efficiently filter the background noise, the effect of filtering the background noise by the common spatial filtering is only about 10 times, and the time domain and spatial filtering effect of the noise reduction method can reach 80 times or more and the noise reduction effect is outstanding through tests.
Further advantageous effects will be further described in the preferred embodiments.
The above-described technical solutions/features are intended to summarize the technical solutions and technical features described in the detailed description section, and thus the ranges described may not be exactly the same. However, these new solutions disclosed in this section are also part of the numerous solutions disclosed in this document, and the technical features disclosed in this section and the technical features disclosed in the following detailed description section, and some contents in the drawings not explicitly described in the specification disclose more solutions in a reasonable combination with each other.
The technical scheme combined by all the technical features disclosed in any position of the invention is used for supporting the generalization of the technical scheme, the modification of the patent document and the disclosure of the technical scheme.
Drawings
Fig. 1 is a schematic diagram of a time domain processing portion of a noise reduction method of the present invention.
Fig. 2 is a schematic diagram of a specific embodiment of a time domain processing portion of the noise reduction method of the present invention.
FIG. 3 is a schematic diagram of a specific class of embodiments of a next-level system.
Fig. 4 is a schematic diagram of an embodiment of the noise reduction method of the present invention applied to a device or chip.
Detailed Description
Since various alternatives are not exhaustive, the gist of the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention. Other technical solutions and details not disclosed in detail below, which generally belong to technical objects or technical features that can be achieved by conventional means in the art, are limited in space and the present invention is not described in detail.
Except where division is used, any position "/" in this disclosure means a logical "or". The ordinal numbers "first", "second", etc., in any position of the present invention are used merely for distinguishing between the labels in the description and do not imply an absolute order in time or space, nor do they imply that the terms preceded by such ordinal numbers are necessarily different from the same terms preceded by other ordinal terms.
The present invention will be described in terms of various elements for use in various combinations of embodiments, which elements are to be combined in various methods, products. In the present invention, even if only the gist described in introducing a method/product scheme means that the corresponding product/method scheme explicitly includes the technical feature.
The description of a step, module, or feature in any location in the disclosure does not imply that the step, module, or feature is the only step or feature present, but that other embodiments may be implemented by those skilled in the art with the aid of other technical means according to the disclosed technical solutions. The embodiments of the present invention are generally disclosed for the purpose of disclosing preferred embodiments, but it is not meant to imply that the contrary embodiments of the preferred embodiments are not intended to cover all embodiments of the invention as long as such contrary embodiments are at least one technical problem addressed by the present invention. Based on the gist of the specific embodiments of the present invention, a person skilled in the art can apply means of substitution, deletion, addition, combination, exchange of sequences, etc. to certain technical features, so as to obtain a technical solution still following the inventive concept. Such solutions without departing from the technical idea of the invention are also within the scope of protection of the invention.
Part of the important term interpretation:
an event imaging device is a sensor that responds to changes in light in the form of events by sensing such changes. Any form of sensor including pixels capable of generating pulsed events may be considered a vision sensor as defined herein, including but not limited to event camera/DVS, DAVIS sensors, as well as fusion sensors that fuse event imaging with RGB imaging, event imaging with infrared fusion, and the like.
The inventors have found through testing that the time intervals of random noise approximate poisson distribution, i.e. the same pixels have substantially similar event numbers over time intervals of the same length. If the pixel has more events in a certain time interval, the events are likely to be generated by actions, rather than background noise. Based on the test result, analysis and experiment of the inventor, the invention provides a method for efficiently and accurately filtering background noise, and whether the event is noise is judged by timestamp information of the event to be judged and position information of pixels generating the event to be judged.
In the invention, without losing generality, a certain event to be distinguished contained in an event stream generated by an event imaging device is marked as being generated by a first pixel (also called as an A pixel), and the position coordinate (x, y) and timestamp t information are marked as A #x, y, t), time interval [ t ] 0 ，t 1 ]The previous time interval is the previous time interval (also called the second time interval) of the first time interval.
The core of the invention is the time domain processing part. Preferably, the present invention includes a spatial domain processing part (which is still in essence a scheme relying on spatial-temporal support) in addition to the temporal processing part.
And storing statistics of events generated by a corresponding pixel in the pixel array in a current time interval in a storage space-storage unit, and calling the statistics set corresponding to the whole pixel array as an event number matrix. Similarly, the aforementioned type of matrix corresponding to the second time interval is referred to as a second event number matrix.
And storing the state value of a corresponding pixel in the pixel array in the current time interval in a storage unit in another storage space, and calling the state value set corresponding to the whole pixel array as an event state matrix.
After the time domain processing of the invention, a storage unit in a storage space stores the mark of the pulse event generated by a corresponding pixel in the pixel array, and the mark set corresponding to the whole pixel array is called as a mark matrix.
Preferably, the length of each time interval is equal, such as the first time interval and the second time interval are equal. Illustratively, a first time interval t in FIG. 2 0 To t 1 The total duration is 10 milliseconds.
Referring to fig. 1, for a noise reduction method of a time domain processing section for noise reduction of an event stream output from an event imaging device, the event stream including events to be discriminated generated by a first pixel, each event in the event stream including at least coordinate information and time stamp information, the noise reduction method comprising the steps of: judging whether the event belongs to a first time interval or not according to the timestamp information of the event to be judged; and if the event to be judged does not belong to the first time interval, judging whether the event to be judged is blocked or not according to fluctuation between statistics of the event generated by the first pixel in the first time interval and statistics of the event generated by the first pixel in the previous time interval of the first time interval.
More specifically, a specific embodiment corresponding to the above technical solution includes the following steps (as shown in fig. 2):
step S1: judging whether the time stamp (information) t of the event to be judged generated by the pixel A belongs to a first time interval [ t ] 0 ，t 1 ]If so, adding 1 to the corresponding statistics of the pixel A in the event number matrix, and executing the step S6; otherwise, step S2 is performed.
Step S2: inquiring the corresponding statistics of the current pixel A in the event number matrix, if the statistics is 0, updating the state value of the corresponding pixel A in the state matrix into a first state value (0 in an example), and executing step S5; otherwise, step S3 is performed.
Step S3: calculating the difference between the statistics of the first time interval corresponding to the pixel A in the event quantity matrix and the statistics of the second time interval corresponding to the pixel A in the second event quantity matrix (namely, the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the previous time interval of the first time interval), if the absolute value of the difference between the statistics is greater than or equal to 3 (example), updating the state value corresponding to the pixel A in the state matrix to be a third state value (2 in example), and otherwise updating the state value to be a second state value (1 in example). Then, the process advances to step S4.
Step S4: when the state value corresponding to the pixel A in the state matrix is 1, the state value corresponding to the spatial neighboring pixel (abbreviated as neighboring pixel) in the neighboring area of the pixel A in the state matrix is checked, and whether the state value corresponding to the pixel A is updated to the third state value is determined according to the state values. For example, if the sum of the status values is greater than a predetermined value, the aforementioned update is performed.
For example, in the 3*3 neighborhood, if the sum of the state values corresponding to 8 neighboring pixels of the pixel a in the state matrix is greater than 3 (an example of the second state value being 1 is taken along), the state value corresponding to the pixel a in the state matrix is updated to the third state value (2 is taken as an example), otherwise, the state value is not updated. For another example, by counting the number of pixels whose state value is the third state value in the neighboring pixels around the pixel a, if the number of pixels in the state exceeds the preset value (for example, 3), the state value corresponding to the pixel a in the state matrix is updated to the third state value (for example, 2), otherwise, the state value is not updated. Step S5 is then performed.
The purpose of step S4 is to correct the corresponding state value of the pixel a in the state matrix (modify from the second state value to the third state value) if the neighboring pixel also generates a certain number (e.g., 1,2, 3) of pulse events when the pixel a generates the event to be discriminated, and give the opportunity to discriminate again in the next stage system (i.e., the spatial domain processing part). This step advantageously avoids false kills of some valid events.
Step S5: inquiring a state value corresponding to the pixel A in the state matrix, and if the state value is a third state value, updating a mark value corresponding to the pixel A in the mark matrix to be a first mark value (0 in an example); if the state value is the second state value, the corresponding flag value of the pixel a in the flag matrix is updated to the second flag value (1 in the example).
Step S6: inquiring a corresponding marking value of the pixel A in the marking matrix, if the value is a first marking value, releasing the event to be distinguished to enter a next-stage system, and if the value is a second marking value, blocking the event to be distinguished.
For steps S5 and S6, alternatively, the step of converting the state matrix into the flag matrix may be omitted, and the determination of whether to pass/block may be made directly from the state value.
In some embodiments, the next stage system, which is preferably the aforementioned spatial processing part, to which the event to be discriminated enters, refer to fig. 3.
And storing the time stamp of the event generated by a corresponding pixel in the pixel array in a storage unit of a certain storage space, and calling the time stamp set corresponding to the whole pixel array as a time stamp matrix.
And sending the events to be discriminated which are released (not blocked) by the time domain processing part to the airspace processing part. Preferably, the spatial processing part of the present invention uses the time stamp of the spatial neighbor pixels around the first pixel to provide enough support to allow the event to be determined to be released. Illustratively, the spatial domain processing part specifically processes as follows:
the event stream to be discriminated that can enter the next stage system is taken as a preliminarily filtered (filtered) event stream. The preliminarily filtered event stream enters the next-stage system, taking the pixel A as an example, updating the timestamp of the corresponding pixel A in the timestamp matrix (before updating, the timestamp of the event before the event to be distinguished is stored in the storage unit of the corresponding pixel A) into the timestamp t of the event to be distinguished, and marking the timestamps corresponding to a plurality of neighbor pixels of the pixel A stored in the timestamp matrix as t n （n=1，2，3，4，……）。
Calculating time stamps t and t n And comparing the time difference with a set threshold (for example, 10 milliseconds), and counting the number of neighbor pixels of which the difference is smaller than or equal to the set threshold. And if the counted number is larger than the second threshold value, releasing the event to be judged, otherwise, blocking the event to be judged.
For example, in the 5*5 neighborhood, time stamps t and t are calculated n The time difference between (n=1, 2,3,4, … …, 24) is set to a threshold of 10 ms, and if the difference is greater than 10 ms, the corresponding timestamp of the n-th neighbor pixel in the timestamp matrix is updated to 0. And counting pixels with time stamps not being 0 in the neighborhood range, if the sum of the values is larger than 4, releasing the event to be judged, otherwise, blocking the event to be judged. The zeroing of the time stamp in this example does not destroy the properties of the time stamp matrix and has the advantage of saving memory space/hardware overhead.
The airspace processing part has the advantages of high speed, event-driven processing and high noise reduction quality.
In certain classes of embodiments, the noise reduction method of the present invention is applied to a noise reduction device that is located in an event imaging device that can be deployed into a chip. An apparatus, such as an electronic device, comprising the aforementioned chip, event imaging device or noise reduction device, is shown in fig. 4.
In the device, an energy collection module, a perception module, a calculation module, a control module and a communication module are deployed. The energy harvesting module may receive environmental energy such as various known electricity generation modules that generate electricity from light energy, temperature differential energy, thermal energy, ocean wave energy, frictional energy, mechanical energy, piezoelectric materials, such as nano-generators.
The sensing module may sense the environmental signal and pass the environmental signal into the computing module for processing, such as an event imaging device, IMU, microphone, etc. The perceived signal is preferably inferred by a pulsed neural network to obtain a processed result.
And the processed results are respectively transmitted into a control module or/and a communication module. The energy collection module is connected with the sensing module, the calculation module, the control module and the communication module and provides energy for the sensing module, the calculation module, the control module and the communication module. The communication module can be connected with a cloud or other nodes for necessary information interaction, for example, the communication module can form an ad hoc network with other nodes. The control module is connected with the external equipment and used for controlling the external equipment to respond correspondingly; or the control module is deployed in the external device, and the control instruction is output based on the communication module to control the external device to respond correspondingly, so that the invention is not limited to the changes according to the needs of the specific application.
In alternative embodiments, the apparatus optionally includes one or more of the following modules: the system comprises an energy collection module, a control module and a communication module.
Preferably, the electronic device is an internet of things device/edge computing device/battery driven device.
Although the present invention has been described with reference to specific features and embodiments thereof, various modifications, combinations, substitutions can be made thereto without departing from the invention. The scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification, but rather, the methods and modules may be practiced in one or more products, methods, and systems of the associated, interdependent, inter-working, pre/post stages.
The specification and drawings are, accordingly, to be regarded in an abbreviated manner as an introduction to some embodiments of the technical solutions defined by the appended claims and are thus to be construed in accordance with the doctrine of greatest reasonable interpretation and are intended to cover as much as possible all modifications, changes, combinations or equivalents within the scope of the disclosure of the invention while also avoiding unreasonable interpretation.
Further improvements in the technical solutions may be made by those skilled in the art on the basis of the present invention in order to achieve better technical results or for the needs of certain applications. However, even if the partial improvement/design has creative or/and progressive characteristics, the technical idea of the present invention is relied on to cover the technical features defined in the claims, and the technical scheme shall fall within the protection scope of the present invention.
The features recited in the appended claims may be presented in the form of alternative features or in the order of some of the technical processes or the sequence of organization of materials may be combined. Those skilled in the art will readily recognize that such modifications, changes, and substitutions can be made herein after with the understanding of the present invention, by changing the sequence of the process steps and the organization of the materials, and then by employing substantially the same means to solve substantially the same technical problem and achieve substantially the same technical result, and therefore such modifications, changes, and substitutions should be made herein by the equivalency of the claims even though they are specifically defined in the appended claims.
The steps and components of the embodiments have been described generally in terms of functions in the foregoing description to clearly illustrate this interchangeability of hardware and software, and in terms of various steps or modules described in connection with the embodiments disclosed herein, may be implemented in hardware, software, or a combination of both. Whether such functionality is implemented as hardware or software depends upon the particular application or design constraints imposed on the solution. Those of ordinary skill in the art may implement the described functionality using different approaches for each particular application, but such implementation is not intended to be beyond the scope of the claimed invention.
Claims (16)
1. A noise reduction method for reducing noise of an event stream output from an event imaging device, the event stream including events to be discriminated generated by a first pixel, each event in the event stream including at least coordinate information and timestamp information, the noise reduction method comprising the steps of:
judging whether the event belongs to a first time interval or not according to the timestamp information of the event to be judged;
if the event to be judged does not belong to the first time interval, judging whether the event to be judged is blocked or not according to fluctuation between statistics of the event generated by the first pixel in the first time interval and statistics of the event generated by the first pixel in the previous time interval of the first time interval;
and if the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the statistics of the events generated by the first pixel in the previous time interval of the first time interval is greater than or equal to a first threshold value, releasing the event to be judged to enter a next stage system.
2. The noise reduction method according to claim 1, characterized in that: if the event to be discriminated belongs to the first time interval, adding one to the statistics of the events generated by the first pixel in the first time interval.
3. The noise reduction method according to claim 2, characterized in that:
after the statistics of the events generated by the first pixel in the first time interval is increased by one, whether to pass or block the event to be judged is judged according to the corresponding marking value of the first pixel in the marking matrix or the corresponding state value of the first pixel in the state matrix.
4. The noise reduction method according to claim 1, characterized in that:
if the fluctuation between the statistics of events generated by the first pixel in the first time interval and the statistics of events generated by the first pixel in the previous time interval of the first time interval is smaller than a first threshold value, then:
judging whether the fluctuation between statistics of events generated by more than a preset number of pixels in airspace neighbor pixels of the first pixel in the first time interval and the time interval before the first time interval is larger than or equal to a first threshold value, if so, releasing the event to be judged to enter a next-stage system, otherwise, blocking the event to be judged.
5. The noise reduction method according to claim 4, wherein:
if the statistics of the events generated by the first pixel in the first time interval is zero, updating the state value of the corresponding first pixel in the state matrix to be a first state value;
if the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the statistics of the events generated by the first pixel in the previous time interval of the first time interval is smaller than a first threshold value, updating the state value corresponding to the first pixel in the state matrix into a second state value;
if the fluctuation between the statistics of the events generated by the first pixel in the first time interval and the statistics of the events generated by the first pixel in the previous time interval of the first time interval is greater than or equal to a first threshold value, the state value corresponding to the first pixel in the state matrix is updated to be a third state value.
6. The noise reduction method according to claim 5, wherein:
if the fluctuation between the statistics of the events generated by more than a preset number of pixels in the airspace neighbor pixels of the first pixel in the first time interval and the time interval before the first time interval is greater than or equal to a first threshold value, updating the state value of the corresponding first pixel in the state matrix to be a third state value.
7. The noise reduction method according to claim 5 or 6, characterized in that:
if the state value corresponding to the first pixel in the state matrix is the third state value, updating the mark value corresponding to the first pixel in the mark matrix to be the first mark value;
if the state value corresponding to the first pixel in the state matrix is the second state value, updating the mark value corresponding to the first pixel in the mark matrix to be the second mark value.
8. The noise reduction method according to claim 7, wherein:
and if the mark value of the first pixel corresponding to the first pixel in the mark matrix is a first mark value, releasing the event to be judged to enter a next stage system, and if the mark value of the first pixel corresponding to the first pixel in the mark matrix is a second mark value, blocking the event to be judged.
9. The noise reduction method according to claim 3 or 4 or 8, characterized in that: the next-stage system is a noise reduction system, the noise reduction system counts the number of the airspace neighbor pixels with the time difference smaller than or equal to a preset threshold value based on the time difference between the time stamp of the airspace neighbor pixel corresponding to the first pixel in the time stamp matrix and the time stamp of the airspace neighbor pixel corresponding to the first pixel in the time stamp matrix, if the number exceeds a second threshold value, the event to be judged is released, otherwise, the event to be judged is blocked.
10. The noise reduction method according to claim 5 or 6, characterized in that:
and if the state value corresponding to the first pixel in the state matrix is a third state value, releasing the event to be judged to enter a next stage system, and if the state value corresponding to the first pixel in the state matrix is a second state value, blocking the event to be judged.
11. The noise reduction method according to any one of claims 1 to 6, 8, characterized in that: the length of the first time interval is the same as the length of the previous time interval of the first time interval, and the length of the first time interval is the same as the length of the next time interval of the first time interval.
12. A noise reduction device, characterized in that: the noise reduction device uses the noise reduction method as defined in any one of claims 1 to 11.
13. An event imaging apparatus, characterized by: the event imaging device comprising the noise reduction device of claim 12.
14. A chip, characterized in that: the chip being deployed with the event imaging device of claim 13.
15. An electronic device, characterized in that: the electronic device is provided with a chip as claimed in claim 14, or an event imaging apparatus as claimed in claim 13, or a noise reduction apparatus as claimed in claim 12.
16. The electronic device of claim 15, wherein: the electronic device further includes a computing module, and one or more of the following: the system comprises an energy collection module, a control module and a communication module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311041178.6A CN116757968B (en) | 2023-08-18 | 2023-08-18 | Noise reduction method and device, chip, event imaging device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311041178.6A CN116757968B (en) | 2023-08-18 | 2023-08-18 | Noise reduction method and device, chip, event imaging device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116757968A CN116757968A (en) | 2023-09-15 |
| CN116757968B true CN116757968B (en) | 2023-11-24 |
Family
ID=87953670
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311041178.6A Active CN116757968B (en) | 2023-08-18 | 2023-08-18 | Noise reduction method and device, chip, event imaging device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116757968B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709895A (en) * | 2015-11-16 | 2017-05-24 | 三星电子株式会社 | Image generating method and apparatus |
| CN113923319A (en) * | 2021-12-14 | 2022-01-11 | 成都时识科技有限公司 | Noise reduction device, noise reduction method, chip, event imaging device and electronic equipment |
| CN114257744A (en) * | 2021-12-20 | 2022-03-29 | 深圳锐视智芯科技有限公司 | Image processing method, device and equipment and readable storage medium |
| CN114285962A (en) * | 2021-12-14 | 2022-04-05 | 成都时识科技有限公司 | Noise processing device, method, chip, event imaging device and electronic equipment |
| CN115412686A (en) * | 2022-10-31 | 2022-11-29 | 深圳时识科技有限公司 | Fusion noise reduction method and device, sensor, chip and electronic equipment |
| WO2023092798A1 (en) * | 2021-11-25 | 2023-06-01 | 成都时识科技有限公司 | Noise filtering for dynamic vision sensor |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8178848B2 (en) * | 2009-12-21 | 2012-05-15 | General Electric Company | Systems and methods for filtering noise in pixelated image detectors |
| US9934557B2 (en) * | 2016-03-22 | 2018-04-03 | Samsung Electronics Co., Ltd | Method and apparatus of image representation and processing for dynamic vision sensor |
| KR20220076944A (en) * | 2020-12-01 | 2022-06-08 | 삼성전자주식회사 | Vision sensor, image processing device comprising thereof and operating method of vision sensor |
| US11843879B2 (en) * | 2020-12-15 | 2023-12-12 | Raytheon BBN Technologies, Corp. | Systems, devices, and methods for pixel frequency filtering for an event camera |
| KR20220164354A (en) * | 2021-06-04 | 2022-12-13 | 삼성전자주식회사 | Vision sensor and operating method of vision sensor |
-
2023
- 2023-08-18 CN CN202311041178.6A patent/CN116757968B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709895A (en) * | 2015-11-16 | 2017-05-24 | 三星电子株式会社 | Image generating method and apparatus |
| WO2023092798A1 (en) * | 2021-11-25 | 2023-06-01 | 成都时识科技有限公司 | Noise filtering for dynamic vision sensor |
| CN113923319A (en) * | 2021-12-14 | 2022-01-11 | 成都时识科技有限公司 | Noise reduction device, noise reduction method, chip, event imaging device and electronic equipment |
| CN114285962A (en) * | 2021-12-14 | 2022-04-05 | 成都时识科技有限公司 | Noise processing device, method, chip, event imaging device and electronic equipment |
| CN114257744A (en) * | 2021-12-20 | 2022-03-29 | 深圳锐视智芯科技有限公司 | Image processing method, device and equipment and readable storage medium |
| CN115412686A (en) * | 2022-10-31 | 2022-11-29 | 深圳时识科技有限公司 | Fusion noise reduction method and device, sensor, chip and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于事件相机的可视化及降噪算法;闫昌达;王霞;左一凡;李磊磊;陈家斌;《北京航空航天大学学报》;第47卷(第2期);第342-350页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116757968A (en) | 2023-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220191393A1 (en) | Method and apparatus of processing a signal from an event-based sensor | |
| US11631246B2 (en) | Method for outputting a signal from an event-based sensor, and event-based sensor using such method | |
| CN108922114A (en) | Security monitor method and system | |
| US8022987B2 (en) | Information-based self-organization of sensor nodes of a sensor network | |
| CN109600363A (en) | A kind of internet-of-things terminal network portrait and abnormal network access behavioral value method | |
| JP2020014144A (en) | Camera system and event record system | |
| GB2455837A (en) | Co-operative camera surveillance method | |
| CN106951849A (en) | A kind of unexpected monitoring method of pre- child-resistant and system | |
| GB1568496A (en) | Video alarm system | |
| CN103514694A (en) | Intrusion detection monitoring system | |
| CN112069939A (en) | Event detection method and device, electronic equipment and storage medium | |
| CN101567812A (en) | Method and device for detecting network attack | |
| DE60137365D1 (en) | METHOD AND DEVICE FOR REDUCING FALLEN ALARMS IN OUTPUT / INPUT SITUATIONS FOR SECURITY MONITORING OF RESIDENTIAL BUILDINGS | |
| CN114169362A (en) | Event stream data denoising method based on space-time correlation filtering | |
| CN116757968B (en) | Noise reduction method and device, chip, event imaging device and electronic equipment | |
| JPH06153202A (en) | Abnormality monitoring device | |
| US20110157360A1 (en) | Surveillance system and method | |
| CN117395062A (en) | Network defense method and device, electronic equipment and storage medium | |
| EP3559858A1 (en) | Rgbd sensing based object detection system and method thereof | |
| CN117115451B (en) | Adaptive threshold event camera denoising method based on space-time content correlation | |
| CN116233628A (en) | Flicker noise filtering system | |
| CN115412686B (en) | Fusion noise reduction method and device, sensor, chip and electronic equipment | |
| JP2923653B2 (en) | Moving object detection device | |
| US11765188B2 (en) | Real-time detection of network attacks | |
| CN113011216A (en) | Multi-classification threshold self-adaptive occlusion detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |