CN115412688B - Flicker noise filtering method and device, sensor, chip and electronic equipment - Google Patents
Flicker noise filtering method and device, sensor, chip and electronic equipment Download PDFInfo
- Publication number
- CN115412688B CN115412688B CN202211345510.3A CN202211345510A CN115412688B CN 115412688 B CN115412688 B CN 115412688B CN 202211345510 A CN202211345510 A CN 202211345510A CN 115412688 B CN115412688 B CN 115412688B
- Authority
- CN
- China
- Prior art keywords
- event
- flicker noise
- noise filtering
- filtering method
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000003384 imaging method Methods 0.000 claims description 44
- 230000008569 process Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 23
- 230000009467 reduction Effects 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 12
- 230000004048 modification Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000010365 information processing Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Landscapes
- Picture Signal Circuits (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
The invention discloses a flicker noise filtering method and device, a sensor, a chip and electronic equipment. In order to solve the problem of flicker noise caused by a flicker light source, the flicker noise filtering method disclosed by the invention receives a first event, and obtains the time difference between the first event and the previous event according to the time stamp of the first event and the time stamp of the previous event of the first event; if the time difference is smaller than a first threshold value, subtracting a non-zero constant from a first value in a first storage unit corresponding to a first event in the first storage space, otherwise, adding a non-zero constant to the first value to obtain an updated first value; and judging whether the first event is flicker noise or not according to the magnitude relation between the updated first numerical value and the second threshold. The invention takes credible counting as a technical means, solves the problem of flicker noise and obtains the noise reduction effect with low power consumption, low delay and high performance. The invention is suitable for the field of event cameras and brain-like computing.
Description
Technical Field
The invention relates to a flicker noise filtering method and device, a sensor, a chip and electronic equipment, in particular to a flicker noise filtering method and device, a sensor, a chip and electronic equipment for an event camera caused by a flicker light source.
Background
The conventional frame image sensor has the defects of delay, blur, high power consumption, etc., and an event camera (event camera) can well overcome the problems, thus becoming the leading edge and hot spot of current academic research. The event camera can be used in the fields of tracking, VR (eye tracking), obstacle avoidance, optical flow estimation, driver state detection, etc., and is good at capturing moving objects in the field of view, while still objects are not imaged in the field of view, so that imaging is completely event-driven, as shown in fig. 1.
Each pixel in the event camera works independently, and an event indicating the change direction of light is generated after the change of the light is detected, which is the bottom layer working principle, accords with the event driving characteristic, has the advantages of low power consumption and low delay, and is completely different from a frame image sensor.
However, ac lighting is commonly used in, for example, a home environment, and this can cause an ac-driven light source to flash (flickering), and this change in light brightness (light source itself and object reflected light) causes an event (referred to as flicker noise or flicker noise event) generated by an event camera to generate an unexpected false "motion" scene or cause extremely poor imaging quality, which in turn affects the difficulty and capability of subsequent information processing.
Included in fig. 2, sub-diagrams (a) and (b), respectively, show the imaging of a scene with a flashing billboard and pedestrian traffic by a low resolution event camera during two different very short time intervals. Moving pedestrians in the figure can be caught, but the constantly flashing billboard is always imaged, which is not desirable. The difficulty with this is that this portion is undesirably imaged, fully conforming to the underlying logic of the event camera capturing the varying light, which is all normal to the event camera itself but detrimental to practical use, so that conventional noise reduction schemes (such as those based on spatiotemporal characteristics) are difficult to adapt to this noise.
The sub-diagrams (a) and (b) included in fig. 3 then show the imaging of a person waving his palm in front of the event camera in two different environments, respectively. The strong and flickering background light source causes that the whole waving action is difficult to be clearly and reliably imaged, and the poor imaging quality obviously provides serious challenges for subsequent information processing.
Prior art 1: US10248222B2;
prior art 2: US11416759B2;
prior art 3: US20210067679A1.
The prior art 1-3 are solutions for eliminating flicker noise, but these solutions require buffering of consecutive events, counting of events by attenuation, and large amounts of memory space, complex high-precision calculations, etc., or acquisition of luminance information by means of other sensors, are not typically event-driven or require a clock, and thus have disadvantages in terms of cost/area, power consumption, delay, etc.
Based on this, there is a need in the art for a low-latency, low-cost, low-power, efficient, and reliable event camera flicker noise reduction scheme.
Disclosure of Invention
In order to solve or alleviate some or all of the technical problems, the invention is realized by the following technical scheme:
a flicker noise filtering method comprises the steps of receiving a first event, and obtaining the time difference between the first event and the previous event of the first event according to the time stamp of the first event and the time stamp of the previous event of the first event; if the time difference is smaller than a first threshold value, subtracting a non-zero constant from a first value in a first storage unit corresponding to a first event in the first storage space, otherwise, adding a non-zero constant to the first value to obtain an updated first value; and judging whether the first event is flicker noise or not according to the magnitude relation between the updated first numerical value and the second threshold.
In the present invention, the first event and the event previous to the first event are both from the first pixel.
In some class of embodiments, the subtracting a non-zero constant is the same as the adding a non-zero constant and is equal to 1.
In some class of embodiments, the first event and a previous event to the first event are both from a first pixel in an event imaging device; the first storage space comprises a plurality of storage units, and the storage units and a plurality of pixels in a pixel array of the event imaging device are in one-to-one correspondence.
In one class of embodiments, the first event is from a first pixel, and a first storage location in the first storage space corresponding to the first event is determined based on coordinates of the first pixel in a pixel array of the event imaging device.
In one class of embodiments, if the non-zero constant is a positive number, if the updated first value is smaller than a second threshold, it is determined that the first event is flicker noise; or,
and if the non-zero constant is a negative number, if the updated first numerical value is greater than a second threshold value, determining that the first event is flicker noise.
In a certain class of embodiments, the first storage unit has a storage length of 3 or 4 or 5 bits.
In some embodiments, if the first event is not judged to be flicker noise, the first event is handed to a post-stage for processing.
In some embodiments, the timestamp of the first event is stored in a second storage unit corresponding to the first event in the second storage space.
In a certain embodiment, after determining whether the first event is flicker noise or obtaining a time difference between the first event and a previous event, the time stamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space.
In some embodiments, the second storage space includes a plurality of storage units therein, and there is a one-to-one correspondence between the plurality of storage units and a plurality of pixels in a pixel array of the event imaging device.
In one class of embodiments, the first event is from a first pixel, and a second storage location in the second storage space corresponding to the first event is determined based on coordinates of the first pixel in a pixel array of the event imaging device.
In some embodiment, the timestamp of the previous event of the first event is taken from a second storage unit corresponding to the first event in the second storage space.
In one embodiment, if the time difference is smaller than the first threshold and is also larger than the third threshold, a non-zero constant is subtracted from the first value in the first storage location corresponding to the first event in the first storage space, otherwise, the non-zero constant is added to the first value.
A flicker noise filtering method comprises the steps of receiving a first event, and obtaining the time difference between the first event and the previous event according to the time stamp of the first event and the time stamp of the previous event of the first event; judging the size relationship between the time difference and a first threshold value, and writing a result representing the size relationship into a first position in a first list corresponding to a first event in a first storage space; and judging whether the first event is flicker noise or not according to a plurality of results which are stored in a plurality of positions of a first list corresponding to the first event in a first storage space and represent size relations.
In some embodiments, the first event and the event prior to the first event are both from a first pixel.
In some embodiments, if the first list has only the first location: if the result representing the magnitude relation is that the time difference is smaller than a first threshold value, judging that the first event is flicker noise, otherwise, judging that the first event is not flicker noise; or, if the first list has at least two positions: then, it is determined whether the first event is flicker noise or not according to the number of results indicating the same kind of magnitude relation in the results indicating the magnitude relation stored in the first position and the results indicating the magnitude relation stored in the first list at the other positions.
In some kind of embodiment, the first-in first-out principle is followed when storing the result representing the size relationship to a number of locations of the first list corresponding to the first event in the first storage space.
In some embodiments, the timestamp of the first event is stored in a second storage unit corresponding to the first event in the second storage space.
A flicker noise filtering apparatus, the flicker noise filtering apparatus comprising at least a first memory space and the flicker noise filtering apparatus noise-filtering a received first event according to the first memory space and the flicker noise filtering method as described in any one of the preceding items.
A sensor, the sensor being an event imaging device comprising a pixel array and a first memory space, the pixel array comprising first pixels generating a first event and generating a previous event of the first event, the first event being noise filtered according to the first memory space and the flicker noise filtering method as described in any of the preceding.
A chip comprising event imaging means and a processor and a first memory space on the chip, noise filtering at least a first event generated by the event imaging means according to the first memory space and the flicker noise filtering method as described in any one of the preceding items; and the processor processes the event generated by the event imaging device according to at least the first event after the noise filtering.
An electronic device, which is deployed with a chip as described before and is used for processing ambient signals.
In one class of embodiments, the processor is a brain-like processor (neuromorphic chip).
Some or all embodiments of the invention have the following beneficial technical effects:
1) The method is completely event-driven, and the scheme directly uses the time stamp of the event without clock participation;
2) There is no need to cache information for consecutive events to perform the comparison, nor to count these events;
3) By threshold comparison rather than a finite state machine, the latter often requiring a large amount of computation;
4) Low latency, the filtering operation occurs only locally and only once for each event.
Further advantages will be further described in the preferred embodiments.
The technical solutions/features disclosed above are intended to be summarized in the detailed description, and thus the ranges may not be completely the same. The technical features disclosed in this section, together with technical features disclosed in the subsequent detailed description and parts of the drawings not explicitly described in the specification, disclose further aspects in a mutually rational combination.
The technical scheme combined by all the technical features disclosed at any position of the invention is used for supporting the generalization of the technical scheme, the modification of the patent document and the disclosure of the technical scheme.
Drawings
FIG. 1 is a schematic diagram of the imaging effect of an event camera capturing a dynamic image;
FIG. 2 is a schematic diagram illustrating the effect of capturing images by an event camera with a flashing background light source for different moments;
FIG. 3 is a schematic diagram of the effect of an event camera capturing images with a strong background flicker light source for different shots;
FIG. 4 is a schematic overall view of an embodiment of the invention;
FIG. 5 is a flow chart of the flicker noise filtering scheme of the present invention;
FIG. 6 is a flow diagram of a flicker noise filtering scheme in an alternative embodiment of the present invention;
FIG. 7 is a comparison graph of imaging effect with a background flicker light source and no moving object under different schemes;
fig. 8 is a contrast diagram of imaging effect when there is a moving object with a background flicker light source under different schemes.
Detailed Description
Since various alternatives cannot be exhaustively described, the following will clearly and completely describe the main points in the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. It is to be understood that the invention is not limited to the details disclosed herein, which may vary widely from one implementation to another.
In the present invention, "/" at any position indicates a logical "or" unless it is a division meaning. The ordinal numbers "first," "second," etc. in any position of the invention are used merely as distinguishing labels in description and do not imply an absolute sequence in time or space, nor that the terms in which such a number is prefaced must be read differently than the terms in which it is prefaced by the same term in another definite sentence.
The present invention may be described in terms of various elements combined into various embodiments, which may be combined into various methods, articles of manufacture. In the present invention, even if the points are described only when introducing the method/product scheme, it means that the corresponding product/method scheme explicitly includes the technical features.
The presence or inclusion of a step, module, feature in any location in the disclosure does not imply that such presence is the only exclusive presence, and those skilled in the art are fully enabled to derive other embodiments based on the teachings herein, along with other techniques. The embodiments disclosed herein are generally for the purpose of disclosing preferred embodiments, but this does not imply that the opposite embodiment to the preferred embodiment is excluded/excluded from the present invention, and it is intended to cover the present invention as long as such opposite embodiment solves at least some technical problem of the present invention. Based on the point described in the embodiments of the present invention, those skilled in the art can completely apply the means of substitution, deletion, addition, combination, and order change to some technical features to obtain a technical solution still following the concept of the present invention. Such a configuration without departing from the technical idea of the present invention is also within the scope of the present invention.
Event cameras, which are essentially event-driven image sensors, are also known as Dynamic Vision Sensors (DVS). Based on this principle, there are some technical solutions to blend it with the conventional frame image pixels, and the obtained sensor can output events and pixel brightness, such as DAVIS sensor and ATIS sensor, and these event-based sensors (EBS) are collectively referred to as event-based imaging devices in the present invention, and belong to one of the sensors. The invention discloses a scheme for filtering flicker noise by taking an event camera as an example.
Referring to fig. 4, for the first event e generated by the event camera, without loss of generality, it may be any event generated by the event camera, which comes from the first pixel. The coordinates of the first pixel in the event camera pixel array are denoted as (x, y), and the time at which event e is generated is called the timestamp and denoted as ts (e). An event generated before the event e and generated by the same pixel (first pixel) is denoted as "e-1", which is a previous event (simply referred to as a previous event) of the event e, and the event is derived from the same pixel and has the same coordinates but different time stamp, and the time stamp of the previous event e-1 is denoted as ts (e-1). The first pixel generates the event e successively after the previous event e-1. The information of an event may also include the direction of the shading of the pixel's light exposure and is referred to as the polarity.
In addition, there are two memory spaces, which are referred to as a first memory space and a second memory space, respectively, and store data referred to as a boolean Map (boolean Map) and an activity Map (activity Map), respectively. There is a one-to-one correspondence between any memory cell corresponding to the boolean graph and the activity graph and any pixel of the event camera pixel array. In other words, each pixel has a corresponding storage unit in the first storage space and the second storage space, respectively. Without loss of generality, for any event e, the event e is respectively called as a first storage unit and a second storage unit. For simplicity of description, the generated pixel of any event e has coordinates (x, y), and the coordinates of the first storage unit and the second storage unit corresponding to the event e in the boolean diagram and the activity diagram are both (x, y) logically. This logical mapping may be in any reasonable manner, which is not a limitation of the present invention.
The Boolean graph stores a count corresponding to the storage unit in the first storage space, and the count reflects the credibility of the corresponding pixel. The greater the value of the count for event e, the less likely event e is due to a flickering light source. Preferably, the memory cell may be 3 bits in memory length, 1 bit in memory length is a very special case, and may also be 4 bits, 5 bits. In the case of 3 bits, the value stored in one embodiment is between-4 and 3, and the initial value is 0.
And the storage unit in the corresponding second storage space of the activity map stores the timestamp of the latest event generated by the corresponding pixel. In other words, the data in the activity map is the timestamp of the last event issued per pixel for the entire pixel array. Preferably, the memory cell may be 16-bit memory length. It is worth mentioning that the shorter the storage length, the less storage space is required. On the premise of meeting the precision requirement, the minimum storage length is selected to be beneficial to reducing the storage space/chip area. Therefore, the time stamp here may be a time stamp obtained by reducing the accuracy (original time stamp accuracy) of the pixel time stamp generated by the event camera, and it is needless to say that the time stamp may be stored with the original time stamp accuracy.
The first storage space and the second storage space are part of a chip storage area. The pixel array of the event camera is the light sensing portion of the chip and the pixel circuit may include photodiodes. The pixel array and the first and second storage spaces may constitute a chip that may be only an event imaging device, i.e. a sensor. The sensor and the processor may be connected together by the adapter board to form a "one-piece" chip, and the first memory space and the second memory space may be considered as part of the interface circuit as a module for constructing the noise reduction circuit.
Referring to fig. 5, there is shown a flicker noise filtering module 10 of an event camera, where the device generating the event may be any event imaging device. The timestamp ts (e) of the first event e is subtracted from the timestamp ts (e-1) of the previous event e-1, and the obtained result is compared with a first threshold value
Comparing if the difference between the two timestamps is at the first threshold
If the first value (x, y) is not equal to the first threshold value, then the boolean outcome is true, then a non-zero constant is subtracted from the first value, bolmap (x, y), stored in the first memory location in the boolean graph (essentially the confidence measure value), and otherwise (false), then the first memory location in the boolean graph is storedThe first value, boolMAP (x, y), stored in the cell performs an increment by a non-zero constant. The non-zero constant may be a positive number or a negative number, and if the constant is a negative number, it means that the logic for determining the event e is reversed after comparing the magnitude relationship between the first value and the second threshold value. The aforementioned non-zero constant is preferably 1.
As a further preferred embodiment of the foregoing embodiment, the result of the foregoing difference is further compared with a third threshold
(e.g., a second threshold/2) if: if the difference between the two timestamps is at the first threshold
Within, and greater than a third threshold
(the boolean result is true only at this time), a non-zero constant is subtracted from the first value boilmap (x, y) stored in the first memory location in the boolean diagram, and otherwise a non-zero constant is added.
Then according to the first value boolMAP (x, y) stored in the first storage unit in the Boolean diagram and the second threshold value
Comparing, if the first value is smaller than the second threshold
Then event e is considered to be a flicker noise caused by the flicker light source; otherwise, the event e is not considered to be flicker noise, and post-stage processing is performed.
Moreover, the terms greater than, less than, etc. are used in the present invention to essentially represent a logical comparison, and the results of the same logical comparison may be obtained by making slight modifications to the boundary values, but this is merely an equivalent routine alternative in the art, such as "≧ 2" and "> 1", in some cases the logical result of the comparison being equivalent. These basic logical transformations or boundary value modifications, etc., may be logically altered and replaced by those skilled in the art without departing from the basic concept of the present invention and still fall within the scope of the present invention.
Whether or not flicker noise is discriminated, the time stamp ts (e) of the event e is stored in the second storage unit having coordinates (x, y) in the second storage space, and its value is denoted as activityMap (x, y). The storing operation may be performed after obtaining a time difference between the first event and a previous event thereof, or after determining whether the first event is flicker noise. Subsequently, when a pixel with coordinates (x, y) in the pixel array issues a new event e +1 again, the aforementioned activityMap (x, y) value is read and used as the timestamp ts (e) of the previous event e of the aforementioned new event e + 1. In other words, in updating the activity map, the timestamp ts (e-1) of the previous event e-1 of event e is flushed by the timestamp ts (e) of event e, and thus is also why the data in the activity map is the last time stamp to issue an event per pixel for the entire pixel array.
As a special example, the boolean graph may also only have a 1-bit storage length. Such an embodiment would not require a second threshold comparison because the boolean diagram could only store 2 states, namely 0 or 1 (one form of result that represents a magnitude relationship). Therefore, in this embodiment, only the timestamp difference between the current event e and the previous event e-1 from the same pixel needs to be compared, if it is smaller than the second threshold
Event e is considered flicker noise, otherwise it is not. This particular scheme may not have the same noise reduction performance as the other schemes.
As an alternative embodiment, the boolean graph stored in the first storage space may further store the aforementioned boolean result of a plurality of events generated successively corresponding to the same pixel through a set of data. In other words, the foregoing embodiment counts up/down the first value boolMAP (x, y) based only on the foregoing Boolean result of the latest event e, while the alternative embodiment keeps a history of Boolean results a number of times before and after (in principle, keeps a number of times before and after each pixel)Time stamping of events is also feasible, but the requirement for storage space is greater), it is obvious that from a richer history, the corresponding first value and its corresponding second threshold value can be easily calculated
The magnitude relationship of (1). This solution may solve the flicker noise problem, but has the disadvantage of requiring a larger first memory space. These specific embodiments or alternative embodiments are within the technical idea of the present disclosure.
In other words, the flicker noise filtering method disclosed herein comprises the steps of: receiving a first event, and obtaining the time difference between the first event and the previous event of the first event according to the time stamp of the first event and the time stamp of the previous event of the first event; judging the size relationship between the time difference and a first threshold value, and writing a result representing the size relationship into a first position in a first list corresponding to a first event in a first storage space; and judging whether the first event is flicker noise or not according to a plurality of results which are positioned in a plurality of positions of a first list corresponding to the first event in a first storage space and represent size relations.
Preferably, the first event and the previous event of the first event are both from a first pixel.
Preferably, if the plurality of positions in the first list only have the first position (in this case, the first position may be a temporary position for storing the result of determining the size relationship): and if the result representing the magnitude relation is that the time difference is smaller than a first threshold value, judging that the first event is flicker noise, otherwise, judging that the first event is not flicker noise. This embodiment corresponds to the specific example described above.
Preferably, if the first list has at least two positions: then, according to the number of the results of the size relationship stored in the first position and the results of the size relationship in the same type stored in the other positions of the first list, whether the first event is flicker noise is judged. The reason why the results indicating the magnitude relation stored in the other positions most recently are selected here is that the magnitude relation is most recently triggered by the same pixel, and more likely to reflect whether the first event e is flicker noise. For example, the first list stores 5 values representing less than relations and 2 values representing more than relations, which is equivalent to the case where the aforementioned first numerical value is equal to 3.
Preferably, the storing of the result representing the size relationship to a number of locations of the first list corresponding to the first event in the first storage space follows a first-in-first-out principle. Following this principle, it is easier to find the most recently stored result of the size relationships at several other locations in the scheme, for example, the size relationships stored at all locations in the first list following the first-in-first-out principle are selected and must be the most recent result of the size relationships.
Preferably, the timestamp of the first event is stored in a second storage unit corresponding to the first event in the second storage space.
It should be noted that other features of the alternative embodiments or specific examples, which are the same as those described herein (without significantly violating logic), are hereby incorporated by reference.
Referring to fig. 6, another alternative embodiment is shown. The flicker noise filtering module 10 of the event camera in this implementation is the same as the scheme in fig. 5. The difference is that the step of reducing the accuracy of the time stamp is also performed before the flicker noise filtering method is performed. Referring to the foregoing, in order to reduce the size of the second storage space, the timestamp accuracy is reduced by shifting, but the performance requirement can still be met. For example, the timestamp ts (e) with low precision is obtained after shifting the timestamp to the left, and then the timestamp with low precision is sent to the flicker noise filtering module 10.
The flicker noise filtering module or the method flow can realize the corresponding information processing flow by designing the corresponding integrated circuit to form a final event imaging device or chip, and can also be realized by an FPGA (field programmable gate array) or software method, and the physical carrier for realizing the method is the flicker noise filtering device. It should be noted that, in the present invention, various parameters such as the threshold value are configurable, which is convenient to be adjusted according to different scenes. In addition, the storage units in the first storage space and the second storage space may be centralized or distributed.
The flicker noise filtering method or the event imaging device comprising the noise filtering device can have better flicker noise resistance. The flicker noise filtering method or the electronic equipment comprising the noise filtering device and the event imaging device can be easily operated under a light source driven by an alternating current power supply.
In other words, disclosed herein are: a flicker noise filtering apparatus, the flicker noise filtering apparatus comprising at least a first memory space and the flicker noise filtering apparatus noise-filtering a received first event according to the first memory space and the flicker noise filtering method as described in any one of the preceding items.
A sensor, the sensor being an event imaging device comprising a pixel array and a first storage space, the pixel array comprising first pixels generating a first event and a previous event generating the first event, the first event being noise filtered according to the first storage space and the flicker noise filtering method as described in any of the previous items.
A chip comprising event imaging means and a processor and a first memory space on which at least a first event generated by said event imaging means is noise filtered according to said first memory space and a flicker noise filtering method as described in any of the previous items; and the processor processes the event generated by the event imaging device according to at least the first event after the noise filtering.
An electronic device is provided with a chip as described above. The electronic device can be an edge device such as a toy and an electronic lock, and the chip is beneficial to realizing the intelligent detection of the always-on line.
In contrast, some prior art solutions filter the noise caused by the ac operating frequency by placing a band stop at the frequency, which is shown in fig. 7 (b) and fig. 8 (b).
Fig. 7 shows the event imaging effect corresponding to different schemes without any moving object and only with a flicker light source. Part (a) of fig. 7 corresponds to the imaging effect without any noise reduction measures, and the visible flicker light source and the reflected light on the object (within the ellipse in the figure) are imaged at all times. Part (b) of fig. 7 is the noise-reduced imaging effect of the above-described contrast noise reduction scheme, and the visible flicker light source and the reflected light on the object (within the ellipse in the figure) are still slightly visible in the background. Part (c) of fig. 7 corresponds to the imaging effect after noise reduction (without other noise reduction schemes) of the present invention, and most of the noise events caused by the background flicker light source are eliminated, and only part of the background random noise remains. This portion of the random noise can be eliminated by other specialized noise reduction schemes.
Fig. 8 shows the event imaging effect corresponding to different schemes in the case of a moving object and a flash lamp source. Part (a) of fig. 8 corresponds to the imaging effect without any noise reduction measure, the visible flicker light source and the reflected light on the object are imaged at all times, and the outline of one side of the mobile office chair (ellipse in the figure) is clearly visible. Part (b) of fig. 8 corresponds to the noise-reduced imaging effect of the above-described comparative noise reduction scheme, and a large amount of flicker noise (ellipse in the figure) remains in addition to the contour of one side of the office chair. Part (c) of fig. 8 corresponds to the noise reduction effect imaging of the noise reduction scheme of the present invention, except for the contour of one side of the office chair (ellipse in the figure), the flicker noise is basically eliminated, and only background random noise is left.
While the present invention has been described with reference to particular features and embodiments thereof, various modifications, combinations, and substitutions may be made thereto without departing from the invention. The scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification, and it is intended that the method, means, and method may be practiced in association with, inter-dependent on, inter-operative with, or after one or more other products, methods.
Therefore, the specification and drawings should be considered simply as a description of some embodiments of the technical solutions defined by the appended claims, and therefore the appended claims should be interpreted according to the principles of maximum reasonable interpretation and are intended to cover all modifications, variations, combinations, or equivalents within the scope of the disclosure as possible, while avoiding an unreasonable interpretation.
To achieve better technical results or for certain applications, a person skilled in the art may make further improvements on the technical solution based on the present invention. However, even if the partial improvement/design is inventive or/and advanced, the technical idea of the present invention is covered by the technical features defined in the claims, and the technical solution is also within the protection scope of the present invention.
Several technical features mentioned in the attached claims may be replaced by alternative technical features or the order of some technical processes, the order of materials organization may be recombined. Those skilled in the art will readily appreciate that various modifications, changes and substitutions can be made without departing from the scope of the present invention, and the technical problems and/or the sequences can be substantially solved by the same means.
The steps and components of the embodiments described in connection with the embodiments disclosed herein may be embodied in hardware, software, or a combination of both, and have been described in a functional generic sense in the foregoing description for the purpose of clearly illustrating the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application or design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (20)
1. A flicker noise filtering method, characterized by:
receiving a first event, and obtaining the time difference between the first event and the previous event of the first event according to the time stamp of the first event and the time stamp of the previous event of the first event;
if the time difference is smaller than a first threshold value, subtracting a non-zero constant from a first value in a first storage unit corresponding to a first event in the first storage space, otherwise, adding a non-zero constant to the first value to obtain an updated first value;
and judging whether the first event is flicker noise or not according to the magnitude relation between the updated first numerical value and the second threshold.
2. The flicker noise filtering method according to claim 1, wherein:
the first event and a previous event to the first event are both from a first pixel.
3. The flicker noise filtering method according to claim 2, wherein:
the subtracting a non-zero constant is the same as the adding a non-zero constant and is equal to 1.
4. The flicker noise filtering method according to claim 1, wherein:
the first event and the event before the first event are both from a first pixel in an event imaging device; the first storage space comprises a plurality of storage units, and the storage units and a plurality of pixels in a pixel array of the event imaging device are in one-to-one correspondence.
5. The flicker noise filtering method according to claim 1, wherein:
the first event is from a first pixel, and a first storage unit corresponding to the first event in the first storage space is determined according to the coordinate of the first pixel in a pixel array of the event imaging device.
6. The flicker noise filtering method according to claim 1, wherein:
if the non-zero constant is a positive number, if the updated first value is smaller than a second threshold, determining that the first event is flicker noise; or,
and if the non-zero constant is a negative number, if the updated first numerical value is greater than a second threshold value, determining that the first event is flicker noise.
7. The flicker noise filtering method according to claim 5, wherein:
the storage length of the first storage unit is 3 or 4 or 5 bits.
8. The flicker noise filtering method according to claim 2, wherein:
and if the first event is not judged to be flicker noise, handing the first event to a post-stage for processing.
9. The flicker noise filtering method according to claim 2, wherein:
and storing the timestamp of the first event in a second storage unit corresponding to the first event in a second storage space.
10. The flicker noise filtering method according to any one of claims 1 to 8, wherein:
and after judging whether the first event is flicker noise or not or obtaining the time difference between the first event and the previous event, storing the time stamp of the first event in a second storage unit corresponding to the first event in a second storage space.
11. The flicker noise filtering method according to claim 10, wherein:
the second storage space comprises a plurality of storage units, and the storage units and a plurality of pixels in a pixel array of the event imaging device are in one-to-one correspondence.
12. The flicker noise filtering method according to claim 10, wherein:
the first event comes from the first pixel, and a second storage unit corresponding to the first event and located in the second storage space is determined according to the coordinate of the first pixel in a pixel array of the event imaging device.
13. The flicker noise filtering method according to claim 10, wherein:
and the timestamp of the previous event of the first event is taken from a second storage unit which is positioned in the second storage space and corresponds to the first event.
14. The flicker noise filtering method according to any one of claims 1 to 8 or 11 to 13, wherein:
if the time difference is smaller than the first threshold and larger than the third threshold, then a non-zero constant is subtracted from a first value in a first storage unit corresponding to the first event in the first storage space, otherwise, the first value is increased by a non-zero constant.
15. The flicker noise filtering method according to any one of claims 9 or 11 to 13, wherein:
reducing the timestamp accuracy of the first event prior to performing a flicker noise filtering method.
16. A flicker noise filtering apparatus, characterized in that:
the flicker noise filtering means comprises at least a first memory space and the flicker noise filtering means noise filters the received first event in dependence of the first memory space and the flicker noise filtering method as claimed in any one of the claims 1 to 15.
17. A sensor, the sensor being an event imaging device comprising a pixel array and a first storage space, the pixel array comprising first pixels generating a first event and generating a previous event of the first event, characterized in that:
the flicker noise filtering method according to the first storage space and any one of claims 1 to 15 noise filters the first event.
18. A chip including an event imaging device, a processor, and a first memory space, the chip comprising:
the flicker noise filtering method of any one of claims 1 to 15, in accordance with the first memory space, at least noise-filtering a first event generated by the event imaging device;
and the processor processes the event generated by the event imaging device according to at least the first event after the noise filtering.
19. The chip of claim 18, wherein:
the processor is a brain-like processor.
20. An electronic device, characterized in that: the electronic device is provided with a chip as claimed in claim 18 or 19.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310003371.4A CN116233628A (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering system |
| CN202211345510.3A CN115412688B (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering method and device, sensor, chip and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211345510.3A CN115412688B (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering method and device, sensor, chip and electronic equipment |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310003371.4A Division CN116233628A (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115412688A CN115412688A (en) | 2022-11-29 |
| CN115412688B true CN115412688B (en) | 2023-02-03 |
Family
ID=84167527
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211345510.3A Active CN115412688B (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering method and device, sensor, chip and electronic equipment |
| CN202310003371.4A Pending CN116233628A (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering system |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310003371.4A Pending CN116233628A (en) | 2022-10-31 | 2022-10-31 | Flicker noise filtering system |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN115412688B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116546339B (en) * | 2023-07-07 | 2023-09-08 | 深圳时识科技有限公司 | Noise reduction device and method for distinguishing event polarity, chip and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111770290A (en) * | 2020-07-29 | 2020-10-13 | 中国科学院长春光学精密机械与物理研究所 | Noise reduction method for dynamic vision sensor output event stream |
| CN113923319A (en) * | 2021-12-14 | 2022-01-11 | 成都时识科技有限公司 | Noise reduction device, noise reduction method, chip, event imaging device and electronic equipment |
| CN114285962A (en) * | 2021-12-14 | 2022-04-05 | 成都时识科技有限公司 | Noise processing device, method, chip, event imaging device and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2018093275A (en) * | 2016-11-30 | 2018-06-14 | オリンパス株式会社 | Imaging apparatus and flicker determination method |
| CN114078100A (en) * | 2021-11-25 | 2022-02-22 | 成都时识科技有限公司 | Clustering noise reduction device, method, chip, event imaging device and electronic equipment |
-
2022
- 2022-10-31 CN CN202211345510.3A patent/CN115412688B/en active Active
- 2022-10-31 CN CN202310003371.4A patent/CN116233628A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111770290A (en) * | 2020-07-29 | 2020-10-13 | 中国科学院长春光学精密机械与物理研究所 | Noise reduction method for dynamic vision sensor output event stream |
| CN113923319A (en) * | 2021-12-14 | 2022-01-11 | 成都时识科技有限公司 | Noise reduction device, noise reduction method, chip, event imaging device and electronic equipment |
| CN114285962A (en) * | 2021-12-14 | 2022-04-05 | 成都时识科技有限公司 | Noise processing device, method, chip, event imaging device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116233628A (en) | 2023-06-06 |
| CN115412688A (en) | 2022-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107948465B (en) | A kind of method and apparatus that detection camera is disturbed | |
| US10755417B2 (en) | Detection system | |
| CN113923319B (en) | Noise reduction device, noise reduction method, chip, event imaging device and electronic equipment | |
| CN115412688B (en) | Flicker noise filtering method and device, sensor, chip and electronic equipment | |
| US20240064422A1 (en) | Noise filtering for dynamic vision sensor | |
| CN111064865B (en) | Background activity noise filter of dynamic vision sensor and processor | |
| US20220100658A1 (en) | Method of processing a series of events received asynchronously from an array of pixels of an event-based light sensor | |
| CN114285962B (en) | Noise processing device, method, chip, event imaging device and electronic equipment | |
| CN114885074B (en) | Event camera denoising method based on space-time density | |
| CN105824441B (en) | Navigation device and its trajectory smoothing method | |
| CN105554380A (en) | Day and night switching method and day and night switching device | |
| CN114078100A (en) | Clustering noise reduction device, method, chip, event imaging device and electronic equipment | |
| CN115412686B (en) | Fusion noise reduction method and device, sensor, chip and electronic equipment | |
| CN115442544A (en) | Dynamic visual event stream noise reduction method based on hot pixels and enhanced space-time correlation | |
| CN117115451B (en) | Adaptive threshold event camera denoising method based on space-time content correlation | |
| CN104782115A (en) | Image processing device for vehicle | |
| JP4763219B2 (en) | Method and apparatus for determining the center of gravity of an optical signal mark attached to a moving object by photogrammetry | |
| CN115412687B (en) | Active and passive combined noise reduction device and method, visual sensor and chip | |
| CN116546339B (en) | Noise reduction device and method for distinguishing event polarity, chip and electronic equipment | |
| CN110706227B (en) | Article state detection method, system, terminal device and storage medium | |
| CN116916164A (en) | Image strobe processing method, device, electronic equipment and storage medium | |
| WO2022001615A1 (en) | Method and system for automatically removing glare regions | |
| Jing-zhong et al. | A real-time detection of vehicle's speed based on vision principle and differential detection | |
| Li et al. | High-Resolution High-Sensitivity Human-Computer Algorithm and Gesture Control SOC Chip for Artificial Intelligence Applications | |
| CN115714923A (en) | Event information visualization method capable of adapting to time resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |