CN116957973A - Data set generation method for event stream noise reduction algorithm evaluation - Google Patents

Data set generation method for event stream noise reduction algorithm evaluation Download PDF

Info

Publication number
CN116957973A
CN116957973A CN202310920757.1A CN202310920757A CN116957973A CN 116957973 A CN116957973 A CN 116957973A CN 202310920757 A CN202310920757 A CN 202310920757A CN 116957973 A CN116957973 A CN 116957973A
Authority
CN
China
Prior art keywords
event
sub
sequence
pulse
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310920757.1A
Other languages
Chinese (zh)
Other versions
CN116957973B (en
Inventor
蔡宇杰
王明宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Yukan Technology Co ltd
Original Assignee
Shanghai Yukan Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Yukan Technology Co ltd filed Critical Shanghai Yukan Technology Co ltd
Priority to CN202310920757.1A priority Critical patent/CN116957973B/en
Publication of CN116957973A publication Critical patent/CN116957973A/en
Application granted granted Critical
Publication of CN116957973B publication Critical patent/CN116957973B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Image Analysis (AREA)

Abstract

The invention provides a data set generation method for event stream noise reduction algorithm evaluation, which comprises the following steps: acquiring video data under a preset illumination condition by using video acquisition equipment; performing preset operation on video data to obtain a plurality of sub-pulse sequences; acquiring multi-sequence space correlation values corresponding to all events in any sub-pulse sequence; classifying and counting all events in the sub-pulse sequence according to the multi-sequence space correlation value to obtain a multi-sequence space correlation-event quantity diagram; determining a judging threshold value of a pulse data set corresponding to the sub-pulse sequence according to the multi-sequence space correlation-event quantity diagram; generating a pulse data set corresponding to the sub-pulse sequence according to the judging threshold value and the multi-sequence space correlation value; performing the above operation on each sub-pulse sequence to obtain a pulse data set corresponding to each sub-pulse sequence; a target pulse data set for evaluation by the event stream noise reduction algorithm is generated from the pulse data set corresponding to each sub-pulse sequence.

Description

Data set generation method for event stream noise reduction algorithm evaluation
Technical Field
The invention relates to the technical field of dynamic sensor data generation, in particular to a data set generation method for event stream noise reduction algorithm evaluation.
Background
The dynamic vision sensor (Dynamic Vision Sensor, DVS) uses the higher biological vision system, and its pixel structure imitates a vision path composed of cone, ON and OFF bipolar cells and ON and OFF ganglion cells, and can independently sense the light intensity change and generate output. The DVS pixel array is composed of individual photosensitive pixels, each of which independently determines whether or not it is activated, and if so, generates an ON or OFF event, and if not, does not generate any output.
With the continuous development of artificial intelligence technology and sensor technology, the contradiction between mass data generated by the sensor and artificial intelligence computing power and power consumption is more and more prominent, and the dynamic vision sensor driven by the event based on the bionic vision principle can reduce the generation of data at the source by utilizing the unique working mechanism, so that the contradiction problem is solved. However, as the array expands and the pixel area and power consumption decrease, noise problems within the pixels and mismatch problems between pixels become non-negligible. This problem will cause the sensor to generate a large number of background noise events, thereby causing additional power consumption and also making processing and recognition of the sensor output data more difficult. Therefore, there is a need for efficient noise reduction of the output of dynamic vision sensor arrays. The effect evaluation of the event stream noise reduction algorithm needs to use event pulse data sets with 'signal' and 'noise' labels, and the quality of the labels directly influences the credibility of the evaluation result of the algorithm, however, how to obtain the pulse data sets with high-quality labels is always a difficult problem.
The generation method of the pulse data set in the prior art mainly comprises the following steps: (1) The event stream captured by the DVS camera is denoised using a very complex algorithm (denoted as algorithm a), and each event is tagged with a "signal" or "noise" according to the result of the denoising. However, using this approach to generate event pulse data sets to evaluate the noise reduction algorithm results in a noise reduction algorithm that is closer to algorithm a, which is clearly contrary to the original intent of noise reduction algorithm effect evaluation; (2) The event stream noise reduction algorithm is evaluated using the composite dataset. Since the dynamic vision sensor generates a low number of noise pulses in the case of excellent lighting conditions, the moving object is photographed first in the case of excellent lighting conditions to obtain an event pulse sequence X, and noise reduction processing is performed thereon to obtain a signal pulse sequence X'. And simultaneously, a noise sequence Y is obtained by shooting pictures without moving objects by a noise modeling method or under a certain low illumination condition, and the pulse sequence X' and the noise sequence Y are fused to obtain a synthetic data set Z with a signal label and a noise label. However, it was found through experiments that the sequence of event pulses captured by the event sensor under different lighting conditions is quite different. Fig. 2 (a) to 2 (c) are respectively event frames accumulated by events captured by using a prophensee EVK4 event camera under normal illumination, low illumination and very low illumination conditions, and it can be seen from the event frames: the clear edge outline of the palm and the details of the obvious sleeves can be seen in the event frame shot under the normal illumination condition; only the edge outline of the palm blurring can be seen in the event frame shot under the low illumination condition, but the details of the sleeves cannot be seen clearly; the shape of the palm can only be approximately seen in the event frames shot under the extremely low illumination condition, which shows that the time-space distribution of the signal pulses under different illumination conditions is different, so that the synthetic data set Z cannot truly evaluate the noise reduction effect of the event stream noise reduction algorithm under the low illumination condition.
The main evaluation method for the noise reduction of the event stream in the prior art comprises the following steps: CN115375581a discloses a dynamic visual event stream noise reduction effect evaluation method based on event time-space synchronization, which comprises the following steps: 1. reading an event stream output by a dynamic vision sensor, and acquiring pose information of the dynamic vision sensor; 2. three-dimensional reconstruction is carried out by combining the event stream with pose information, and events triggered at different moments are time-space synchronized to reference moments, so that a confidence map is obtained; 3. converting the confidence map into an event probability map, and representing the response probability of the dynamic vision sensor to the scene under ideal conditions; 4. based on the consistency of the event stream and the event probability map, calculating the rationality of the event stream; 5. and improving the rationality of the event stream by calculating the noise reduction algorithm to obtain a noise reduction precision index which is used for evaluating and comparing the noise reduction effects of different algorithms. The technical scheme realizes the evaluation of the noise reduction effect of the event stream, but is not suitable for different illumination conditions.
In summary, how to generate pulse data sets with high-quality labels under different illumination conditions, and further accurately evaluate the effect of the event stream noise reduction algorithm is one of the problems to be solved in the technical field of dynamic sensor data generation.
Disclosure of Invention
The present invention aims to solve at least some of the technical problems in the above-described technology. Therefore, the invention aims to provide a data set generation method for evaluating an event stream noise reduction algorithm, which is used for calculating a multi-sequence spatial correlation value of an event, further determining a judgment threshold value, determining a label of the event according to the multi-sequence spatial correlation value and the judgment threshold value, further generating a pulse data set, and further realizing the technical effects of generating the pulse data set with high-quality labels under different illumination conditions, and further accurately evaluating the effect of the event stream noise reduction algorithm.
The invention provides a data set generation method for event stream noise reduction algorithm evaluation, which comprises the following steps:
acquiring video data under a preset illumination condition by using video acquisition equipment;
performing preset operation on video data to obtain a plurality of sub-pulse sequences; wherein each sub-pulse sequence comprises a plurality of events;
selecting any one of a plurality of sub-pulse sequences, and acquiring multi-sequence space correlation values corresponding to all events in the sub-pulse sequences;
classifying and counting all events in the sub-pulse sequence according to the multi-sequence space correlation value to obtain a multi-sequence space correlation-event number diagram corresponding to the sub-pulse sequence;
Determining a judging threshold value of a pulse data set corresponding to the sub-pulse sequence according to the multi-sequence space correlation-event quantity diagram;
generating a pulse data set corresponding to the sub-pulse sequence according to the judging threshold value and the multi-sequence space correlation values of all events in the sub-pulse sequence;
performing the above operation on each sub-pulse sequence to obtain a pulse data set corresponding to each sub-pulse sequence;
a target pulse data set for evaluation by the event stream noise reduction algorithm is generated from the pulse data set corresponding to each sub-pulse sequence.
Preferably, the video acquisition device is a Propheee EVK4 event camera.
Preferably, the data set generating method for evaluating an event stream noise reduction algorithm acquires video data under a preset illumination condition by using a video acquisition device, and includes:
under the preset illumination condition, the shot object is fixed on a circular disc or an electric sliding rail which rotates periodically, the same movement process of the shot object is determined at the same time, and the movement process of the shot object is repeatedly recorded by utilizing video acquisition equipment to obtain video data.
Preferably, the data set generating method for evaluating an event stream noise reduction algorithm performs a preset operation on video data to obtain a plurality of sub-pulse sequences, including:
And editing the video data to obtain a plurality of sub-videos with completely consistent video content, wherein the plurality of sub-videos are used as a plurality of sub-pulse sequences, and each sub-video corresponds to one sub-pulse sequence.
Preferably, the data set generating method for evaluating the event stream noise reduction algorithm acquires multi-sequence spatial correlation values corresponding to all events in the sub-pulse sequence, including:
constructing a display array according to the sub-pulse sequence;
selecting any event in the sequence in the sub-pulse sequence as a target event, and acquiring the coordinates of the target event in the display array;
determining the polarity of a target event through a preset polarity function according to video data;
determining whether events are generated at adjacent coordinates of associated events corresponding to the target event in a plurality of sub-pulse sequences through a preset first judging function according to the video data, and obtaining a plurality of first judging results;
determining whether the polarity of the event generated at the adjacent coordinates of the related event of the plurality of sub-pulse sequences is the same as the polarity of the target event or not through a preset second judging function, and obtaining a plurality of second judging results;
setting a convolution kernel with the size of 3 multiplied by 3, and determining a convolution kernel weight;
Determining a multi-sequence space correlation numerical operation formula according to the number of the sub-pulse sequences, a preset first judging function, a preset second judging function and a convolution kernel;
bringing the first judgment results, the second judgment results and the convolution kernel weight into an operation formula to obtain a multi-sequence spatial correlation value corresponding to the target event;
and carrying out the processing process on all the events in the sub-pulse sequence to obtain the multi-sequence spatial correlation value corresponding to all the events in the sub-pulse sequence.
Preferably, the data set generating method for evaluating the event stream noise reduction algorithm determines a multi-sequence spatial correlation numerical operation formula according to the number of a plurality of sub-pulse sequences, a preset first judging function, a preset second judging function and a convolution kernel, wherein the multi-sequence spatial correlation numerical operation formula is as follows:
wherein, psi is N A multi-sequence spatial correlation value representing a target event; n represents the number of a plurality of sub-pulse sequences; m represents an mth sub-pulse sequence in a plurality of sub-pulse sequences; (x) 0 ,y 0 ) Coordinates representing a target event; i has the values of-1, 0 and 1; j has the values of-1, 0 and 1;representing the polarity of the target event; function K (m, x 0 +i,y 0 +j) is a preset first judging function for judging the coordinates in the mth sub-pulse sequence as (x) 0 +i,y 0 Whether an event is generated at the +j) position or not, and obtaining a first judging result; function->Is a preset second judging function for representing that the coordinates in the mth sub-pulse sequence are (x 0 +i,y 0 +j) generated eventsWhether or not the polarity of the member is equal to->The same, a second judgment result is obtained, wherein ∈>Representing the coordinates (x) in the mth sub-pulse sequence 0 +i,y 0 +j) polarity of the event generated at; omega i,j Representing the weight of the convolution kernel.
Preferably, the data set generating method for evaluating an event stream noise reduction algorithm performs classification statistics on all events in a sub-pulse sequence according to a multi-sequence spatial correlation value to obtain a multi-sequence spatial correlation-event number diagram corresponding to the sub-pulse sequence, and the method comprises the following steps:
classifying all events in the sub-pulse sequence according to the multi-sequence spatial correlation values, classifying the events with the same multi-sequence spatial correlation values into one class, and counting the number of each class of classified events to obtain an event counting result;
and taking the multi-sequence space correlation value as a horizontal axis and the event statistical result as a vertical axis to obtain a multi-sequence space correlation-event number graph corresponding to the sub-pulse sequence.
Preferably, the data set generating method for evaluating an event stream noise reduction algorithm determines a decision threshold of a pulse data set corresponding to a sub-pulse sequence according to a multi-sequence spatial correlation-event number graph, and includes:
determining a plurality of test thresholds according to the multi-sequence spatial correlation-event number graph;
selecting one test threshold value from the multiple test threshold values at will as a current test threshold value, and determining the label of each event in the sub-pulse sequence according to the current test threshold value; the tag comprises a signal tag and a noise tag;
drawing a space-time distribution diagram of a signal event and a noise event in a sub-pulse sequence under a current test threshold by using 3D drawing software; the signal event is an event with a signal tag, and the noise event is an event with a noise tag;
observing the definition of the motion trace of the signal event in the space-time distribution diagram;
when the definition of the motion trace of the signal event is higher than the first preset definition and the definition of the motion trace of the noise event is lower than the second preset definition, the current test threshold is used as a judgment threshold of a pulse data set corresponding to the sub-pulse sequence; wherein the first preset definition is higher than the second preset definition.
Preferably, the data set generating method for evaluating an event stream noise reduction algorithm generates a pulse data set corresponding to a sub-pulse sequence according to a decision threshold and a multi-sequence spatial correlation value of all events in the sub-pulse sequence, including:
comparing the multi-sequence spatial correlation value of each event in the sub-pulse sequence with a judging threshold value;
when the multi-sequence spatial correlation value of the event is smaller than the judgment threshold value, determining that the label of the event is noise; when the multi-sequence spatial correlation value of the event is not smaller than the judgment threshold value, determining the label of the event as a signal;
all events with noise tags and signal tags are generated as a corresponding pulse data set of the sub-pulse sequence.
Preferably, the data set generating method for evaluating an event stream noise reduction algorithm determines a plurality of test thresholds according to a multi-sequence spatial correlation-event number graph, including:
calculating to obtain an average value of the event number according to the multi-sequence space correlation-event number graph;
when the average value is an integer, taking the average value as a first event number, adding a preset value to the first event number to obtain a second event number, and subtracting the preset value from the first event number to obtain a third event number; acquiring a multi-sequence spatial correlation value corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set;
When the average value is a non-integer, rounding the average value upwards to obtain a rounded average value, taking the rounded average value as a first event number, adding a preset value to the first event number to obtain a second event number, and subtracting the preset value from the first event number to obtain a third event number; acquiring a multi-sequence spatial correlation value corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set;
taking the event number corresponding to the lowest point of the multi-sequence spatial correlation-event number graph as a fourth event number, adding a preset value to the fourth event number to obtain a fifth event number, and obtaining multi-sequence spatial correlation values corresponding to the fourth event number and the fifth event number to obtain a second test threshold set;
taking the event number corresponding to the highest point of the multi-sequence spatial correlation-event number graph as a sixth event number, subtracting a preset value from the sixth event number to obtain a seventh event number, and obtaining multi-sequence spatial correlation values corresponding to the sixth event number and the seventh event number to obtain a third test threshold set;
and taking the threshold value in the combined set of the first test threshold value set, the second test threshold value set and the third test threshold value set as a plurality of test threshold values.
The invention provides a data set generation method for evaluating an event stream noise reduction algorithm, which comprises the steps of acquiring video data under a preset illumination condition by utilizing video acquisition equipment; performing preset operation on video data to obtain a plurality of sub-pulse sequences; selecting any one of a plurality of sub-pulse sequences, and acquiring multi-sequence space correlation values corresponding to all events in the sub-pulse sequences; classifying and counting all events in the sub-pulse sequence according to the multi-sequence space correlation value to obtain a multi-sequence space correlation-event number diagram corresponding to the sub-pulse sequence; determining a judging threshold value of a pulse data set corresponding to the sub-pulse sequence according to the multi-sequence space correlation-event quantity diagram; generating a pulse data set corresponding to the sub-pulse sequence according to the judging threshold value and the multi-sequence space correlation values of all events in the sub-pulse sequence; performing the above operation on each sub-pulse sequence to obtain a pulse data set corresponding to each sub-pulse sequence; a target pulse data set for evaluation by the event stream noise reduction algorithm is generated from the pulse data set corresponding to each sub-pulse sequence. Therefore, the method and the device have the advantages that the pulse data set with the high-quality label is generated under different illumination conditions, and further, the effect of the event stream noise reduction algorithm is accurately evaluated.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flow chart of a method for generating a data set for event stream noise reduction algorithm evaluation in an embodiment of the invention;
FIG. 2 is a display diagram of an event frame accumulated by events captured by an optional event camera under different illumination conditions according to an embodiment of the present invention;
FIG. 3 is a diagram of an alternative display array in accordance with an embodiment of the present invention;
FIG. 4 is an alternative event storage block diagram in accordance with an embodiment of the present invention;
FIG. 5 is an alternative convolution kernel design diagram in accordance with an embodiment of the present invention;
FIG. 6 is a statistical plot of the mean versus standard deviation of single sequence spatial correlations of events generated by an alternative pixel A, B, C, D according to an embodiment of the invention;
FIG. 7 shows a multi-sequence spatial correlation ψ of events generated by a pixel A, B, C, D for an alternative pulse sequence number N equal to 10 and 100 in an embodiment of the invention N A statistical graph of the magnitude relation between the mean value and the standard deviation;
FIG. 8 is an alternative event spatiotemporal profile in accordance with an embodiment of the invention;
FIG. 9 is a diagram showing a time-space distribution of events during rotation of a fan blade under an alternative extreme dark condition according to an embodiment of the present invention;
FIG. 10 is a diagram of an alternative multi-sequence spatial correlation versus event number in an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
Referring to fig. 1, an embodiment of the present invention provides a data set generating method for event stream noise reduction algorithm evaluation, including:
step S1, acquiring video data under preset illumination conditions by using video acquisition equipment;
step S2, performing preset operation on the video data to obtain a plurality of sub-pulse sequences; wherein each sub-pulse sequence comprises a plurality of events;
Step S3, selecting any one of a plurality of sub-pulse sequences, and acquiring multi-sequence space correlation values corresponding to all events in the sub-pulse sequences;
step S4, classifying and counting all events in the sub-pulse sequence according to the multi-sequence space correlation value to obtain a multi-sequence space correlation-event number diagram corresponding to the sub-pulse sequence;
step S5, determining a judging threshold value of a pulse data set corresponding to the sub-pulse sequence according to the multi-sequence space correlation-event quantity diagram;
step S6, generating a pulse data set corresponding to the sub-pulse sequence according to the judging threshold value and the multi-sequence space correlation value of all events in the sub-pulse sequence;
step S7, performing the above operation on each sub-pulse sequence to obtain a pulse data set corresponding to each sub-pulse sequence;
and S8, generating a target pulse data set for evaluation of the event stream noise reduction algorithm according to the pulse data set corresponding to each sub-pulse sequence.
In this embodiment, the preset illumination may be strong light, weak light, dim light, or the like.
The technical principle of the technical scheme is as follows: acquiring video data under a preset illumination condition by using video acquisition equipment; performing preset operation on video data to obtain a plurality of sub-pulse sequences; selecting any one of a plurality of sub-pulse sequences, and acquiring multi-sequence space correlation values corresponding to all events in the sub-pulse sequences; classifying and counting all events in the sub-pulse sequence according to the multi-sequence space correlation value to obtain a multi-sequence space correlation-event number diagram corresponding to the sub-pulse sequence; determining a judging threshold value of a pulse data set corresponding to the sub-pulse sequence according to the multi-sequence space correlation-event quantity diagram; generating a pulse data set corresponding to the sub-pulse sequence according to the judging threshold value and the multi-sequence space correlation values of all events in the sub-pulse sequence; performing the above operation on each sub-pulse sequence to obtain a pulse data set corresponding to each sub-pulse sequence; a target pulse data set for evaluation by the event stream noise reduction algorithm is generated from the pulse data set corresponding to each sub-pulse sequence.
The technical effect of the scheme is as follows: the method comprises the steps of calculating the multi-sequence spatial correlation value of the event, further determining a judgment threshold, determining the label of the event according to the multi-sequence spatial correlation value and the judgment threshold, further generating a pulse data set, and further realizing the technical effects of generating the pulse data set with high-quality labels under different illumination conditions and further carrying out accurate evaluation on the effect of the event stream noise reduction algorithm.
The embodiment of the invention provides a data set generation method for evaluating an event stream noise reduction algorithm, and video acquisition equipment is a Propheee EVK4 event camera.
The technical principle of the technical scheme is as follows: video data was acquired using a prophaee EVK4 event camera.
The technical effects of the technical scheme are as follows: video data required for generating the data set is acquired, and source data is provided for the data set generation process.
The embodiment of the invention provides a data set generation method for evaluating an event stream noise reduction algorithm, which utilizes video acquisition equipment to acquire video data under preset illumination conditions, and comprises the following steps:
under the preset illumination condition, the shot object is fixed on a circular disc or an electric sliding rail which rotates periodically, the same movement process of the shot object is determined at the same time, and the movement process of the shot object is repeatedly recorded by utilizing video acquisition equipment to obtain video data.
The technical principle of the technical scheme is as follows: under the preset illumination condition, the shot object is fixed on a circular disc or an electric sliding rail which rotates periodically, the same movement process of the shot object is determined at the same time, and the movement process of the shot object is repeatedly recorded by utilizing video acquisition equipment to obtain video data.
The technical effects of the technical scheme are as follows: by repeatedly recording the motion process of the shot object, a plurality of video data with the same content are obtained, sufficient data are provided for the data set after processing, and the rationality and the correctness of the data stream noise reduction algorithm are ensured.
The embodiment of the invention provides a data set generation method for evaluating an event stream noise reduction algorithm, which is used for carrying out preset operation on video data to obtain a plurality of sub-pulse sequences and comprises the following steps:
and editing the video data to obtain a plurality of sub-videos with completely consistent video content, wherein the plurality of sub-videos are used as a plurality of sub-pulse sequences, and each sub-video corresponds to one sub-pulse sequence.
In this embodiment, the video data is clipped to obtain a plurality of sub-videos with completely consistent video content, and a specific implementation manner of using the plurality of sub-videos as the plurality of sub-pulse sequences may be: collecting a plurality of pieces of video data through a Propheee EVK4 event camera, and selecting a part of video in one piece of video data as a target sub-video; clipping operation is carried out on the rest video data to obtain videos corresponding to the same parts as the target sub-video content in each video data, wherein the videos corresponding to the same parts as the target sub-video content in the target sub-video and the rest video data are a plurality of sub-videos with completely consistent video content; each sub-video is used as a sub-pulse sequence, and a plurality of sub-videos are used as a plurality of sub-pulse sequences.
The technical principle of the technical scheme is as follows: and editing the video data to obtain a plurality of sub-videos with completely consistent video content, wherein the plurality of sub-videos are used as a plurality of sub-pulse sequences, and each sub-video corresponds to one sub-pulse sequence.
The technical effects of the technical scheme are as follows: and editing the video data to obtain a plurality of sub-videos with completely consistent contents, taking the sub-videos as sub-pulse sequences, providing sufficient data for the generated data set after processing, and ensuring the rationality and the correctness when evaluating the data stream noise reduction algorithm.
Referring to fig. 3, fig. 4, and fig. 5, an embodiment of the present invention provides a data set generating method for evaluating an event stream noise reduction algorithm, and obtains multiple sequence spatial correlation values corresponding to all events in a sub-pulse sequence, including:
constructing a display array according to the sub-pulse sequence;
selecting any event in the sequence in the sub-pulse sequence as a target event, and acquiring the coordinates of the target event in the display array;
determining the polarity of a target event through a preset polarity function according to video data;
determining whether events are generated at adjacent coordinates of associated events corresponding to the target event in a plurality of sub-pulse sequences through a preset first judging function according to the video data, and obtaining a plurality of first judging results;
Determining whether the polarity of the event generated at the adjacent coordinates of the related event of the plurality of sub-pulse sequences is the same as the polarity of the target event or not through a preset second judging function, and obtaining a plurality of second judging results;
setting a convolution kernel with the size of 3 multiplied by 3, and determining a convolution kernel weight;
determining a multi-sequence space correlation numerical operation formula according to the number of the sub-pulse sequences, a preset first judging function, a preset second judging function and a convolution kernel;
bringing the first judgment results, the second judgment results and the convolution kernel weight into an operation formula to obtain a multi-sequence spatial correlation value corresponding to the target event;
and carrying out the processing process on all the events in the sub-pulse sequence to obtain the multi-sequence spatial correlation value corresponding to all the events in the sub-pulse sequence.
In this embodiment, a specific implementation of constructing a display array according to a sub-pulse sequence may be: each pulse sequence corresponds to one sub-video, each sub-video is divided into multiple frames of videos, each frame of video corresponds to one picture, each pixel point in the picture is taken as an array element, each frame of video corresponds to one array, and the array is shown in fig. 3. In the array, the position of the lower left corner is selected as the origin of coordinates (0, 0), and the coordinates of the point A in the figure are (4, 4).
In this embodiment, the preset polarity function may be:
in this embodiment, when the target event is in the event frame a, the associated event corresponding to the target event is: the events in the event frame a in the remaining sub-pulse sequences are the same as the target event coordinates.
As shown in fig. 4, in this embodiment, a specific implementation manner of determining whether an event is generated at the adjacent coordinates of the associated event corresponding to the target event in the plurality of sub-pulse sequences by presetting the first judging function may be: by observing the event storage structure diagram, whether an event exists at the adjacent coordinates of the associated event corresponding to the target event in the plurality of sub-pulse sequences is confirmed, and whether the event is generated at the adjacent coordinates is further confirmed.
In the embodiment, when an event is generated at the adjacent coordinates of the associated event corresponding to the target event in the plurality of sub-pulse sequences, the first judgment result is 1; when no event is generated at the adjacent coordinates of the associated event corresponding to the target event in the plurality of sub-pulse sequences, the first judgment result is 0.
In this embodiment, determining whether the polarity of the event generated at the adjacent coordinates of the associated event of the plurality of sub-pulse sequences is the same as the polarity of the target event by presetting the second judging function may be: the polarity of the event generated at the adjacent coordinates of the associated event of the plurality of sub-pulse sequences is acquired through the dynamic vision sensor, the acquired polarity is compared with the polarity of the target event, and whether the acquired polarity and the polarity of the target event are identical is determined.
In this embodiment, when determining that the polarity of the event generated at the adjacent coordinates of the associated event of the plurality of sub-pulse sequences is the same as the polarity of the target event, the second determination result is 1; and when determining that the polarity of the event generated at the adjacent coordinates of the related events of the plurality of sub-pulse sequences is different from the polarity of the target event, the second judgment result is 0.
In this embodiment, a convolution kernel of 3×3 size and the convolution kernel weights are shown in fig. 5.
In this embodiment, the reason for selecting to calculate the multi-sequence spatial correlation value and further obtain the data set is:
as shown in fig. 3, the intensity of illumination perceived by pixels represented by black areas in the array varies from weak to strong due to movement of the object, while the intensity of illumination perceived by pixels represented by white areas does not. The probability of a black region pixel producing an ON event within a very small time slice dt is noted asThe probability of a black area pixel generating an OFF event is denoted +.>The probability of the white area generating an ON event is denoted +.>The probability of the white area generating an OFF event is denoted +.>According to the principle of operation of DVS, it is obvious +.>Is greater than->Is->But->And->The magnitude relation between the two is dependent on the characteristics of the dynamic vision sensor itself and can be changed due to the design of the DVS pixel circuit, and the following mathematical evidence is independent of +. >And->Magnitude relation between the two.
Considering the black area inside pixel a, the black area edge pixel B, the white area inside pixel C, and the white area edge pixel D, the conventional spatio-temporal correlation filter can easily distinguish the ON event generated by the pixel a as a "signal" event and the event generated by the pixel C as a "noise" event. However, since pixel B is at the black region edge and only one pixel in the surrounding 3×3 neighborhood is at the black region, and pixel D is at the white region edge and only one pixel in the surrounding 3×3 neighborhood is at the white region, the conventional spatio-temporal correlation filter cannot correctly label the labels of the events generated by the B pixels and the D pixels.
Evaluating events using convolution kernels as shown in FIG. 5The single sequence spatial correlation ψ within the time slice dt is shown in the following formula. Wherein (x) 0 ,y 0 ) Sitting representing eventsMark (I) of->Indicating the polarity, omega of the event i,j Representing the weights of the convolution kernel, function K (x 0 +i,y 0 +j) represents a coordinate of (x 0 +i,y 0 Whether an event is generated for a pixel of +j), functionFor determining the coordinates as (x) 0 +i,y 0 Whether the polarity of the event generated by the pixel of +j) is equal to +.>The same applies. Knowing the AND event +.>Homopolar events with smaller spatial distances may provide higher spatial correlation.
The mean value of the single sequence spatial correlation of the ON events generated by pixels A, B, C, D is respectively noted as And->Standard deviation is respectively marked as->And->Spatial correlation of single sequence of OFF events generated by pixel A, B, C, DThe mean values of (2) are respectively marked->And->Standard deviation is respectively marked as-> And (3) withThe mean and standard deviation expressions of the single sequence spatial correlation ψ of events generated by the pixel A, B, C, D can be obtained according to the mean and standard deviation characteristics and binomial distribution formula as shown in formulas (1) to (16).
Fig. 6 shows the magnitude of the mean and standard deviation of the single sequence spatial correlation of events generated by pixel A, B, C, D, as can be seen from the figure:
(1) The mean of the single-sequence spatial correlations of the ON events generated by pixel a is highest, and the mean of the single-sequence spatial correlations of the ON events generated by pixel D is inferior and greater than the single-sequence spatial correlations of the ON events generated by pixel B. It is therefore impossible to correctly distinguish the ON event (signal) of the orange region due to the variation of the illumination intensity from weak to strong from the ON event (noise) of the white region by setting a suitable single-sequence spatial correlation threshold;
(2) The standard deviation value of the spatial correlation of the event single sequence is comparable with the average value, and the value of the spatial correlation of the event single sequence is proved to have high randomness, so that the event single sequence is marked with wrong signal and noise labels with high probability.
In summary, conventional spatio-temporal correlation filters cannot generate correct labels for edge events, and using only a single event sequence cannot effectively distinguish signal events from noise events by spatial correlation of events within a slice of time. Therefore, the multi-sequence spatial correlation value is selected to mark the event for the event, and then a data set is generated.
The technical principle of the technical scheme is as follows: constructing a display array according to the sub-pulse sequence; selecting any event in the sequence in the sub-pulse sequence as a target event, and acquiring the coordinates of the target event in the display array; determining the polarity of a target event through a preset polarity function according to video data; determining whether events are generated at adjacent coordinates of associated events corresponding to the target event in a plurality of sub-pulse sequences through a preset first judging function according to the video data, and obtaining a plurality of first judging results; determining whether the polarity of the event generated at the adjacent coordinates of the related event of the plurality of sub-pulse sequences is the same as the polarity of the target event or not through a preset second judging function, and obtaining a plurality of second judging results; setting a convolution kernel with the size of 3 multiplied by 3, and determining a convolution kernel weight; determining a multi-sequence space correlation numerical operation formula according to the number of the sub-pulse sequences, a preset first judging function, a preset second judging function and a convolution kernel; bringing the first judgment results, the second judgment results and the convolution kernel weight into an operation formula to obtain a multi-sequence spatial correlation value corresponding to the target event; and carrying out the processing process on all the events in the sub-pulse sequence to obtain the multi-sequence spatial correlation value corresponding to all the events in the sub-pulse sequence.
The technical effects of the technical scheme are as follows: the operation formula of the multi-sequence space correlation value is determined through the acquired multiple parameters and the set multiple functions, a formula foundation is provided for acquiring the data set, and further the rationality and the correctness of the data flow noise reduction algorithm are guaranteed.
The embodiment of the invention provides a data set generation method for evaluating an event stream noise reduction algorithm, which is used for determining a multi-sequence spatial correlation numerical operation formula according to the number of a plurality of sub-pulse sequences, a preset first judgment function, a preset second judgment function and a convolution kernel, wherein the multi-sequence spatial correlation numerical operation formula is as follows:
wherein, psi is N A multi-sequence spatial correlation value representing a target event; n represents the number of a plurality of sub-pulse sequences; m represents an mth sub-pulse sequence in a plurality of sub-pulse sequences; (x) 0 ,y 0 ) Coordinates representing a target event; i has the values of-1, 0 and 1; j has the values of-1, 0 and 1;representing the polarity of the target event; function K (m, x 0 +i,y 0 +j) is a preset first judging function for judging the coordinates in the mth sub-pulse sequence as (x) 0 +i,y 0 If an event is generated at +j), a first determination result is obtained, wherein the coordinates (x 0 +i,y 0 +j) is the adjacent coordinates of the associated event corresponding to the target event; function of For presetting the second judgmentA function for representing the coordinates (x) in the mth sub-pulse sequence 0 +i,y 0 Whether the polarity of the event generated at +j) is equal to +.>The same, a second judgment result is obtained, wherein ∈>Representing the coordinates (x) in the mth sub-pulse sequence 0 +i,y 0 +j) polarity of the event generated at; omega i,j Representing the weight of the convolution kernel.
In this embodiment, the number N of sub-pulse sequences is as large as possible for the following reasons:
the mean value of the single sequence spatial correlation of the ON events generated by pixels A, B, C, D is respectively noted as And->Standard deviation is respectively marked as->And->The mean value of the spatial correlation of the single sequence of OFF events generated by pixel A, B, C, D is denoted as +.>And->Standard deviation is respectively marked as-> And (3) withThe multi-sequence spatial correlation ψ of events generated by the pixel A, B, C, D can be obtained according to the characteristics of the mean and standard deviation and the binomial distribution formula N The expression of the mean and standard deviation of (2) is shown in formulas (17) to (32)
From the observation of formulas (17) to (32), it can be seen that:
(1) The mean and standard deviation of the single sequence spatial correlation ψ are the multi-sequence spatial correlation ψ N A special form when the mean value and standard deviation of the (a) are 1;
(2) Multi-sequence spatial correlation psi N The average value and the standard deviation of the pulse sequence are increased along with the increase of the number N of the pulse sequences, but the increase speed of the average value is larger than that of the standard deviation, so that the multi-sequence spatial correlation psi is increased along with the increase of N N The randomness of the value can be reduced;
(3) Thanks to the convolution weights ω 0,0 The value of (2) is greater than the sum of the weighted values of other positions of the convolution, and the sum events in the multi-section pulse sequenceHomopolar events of the same coordinates may provide higher spatial correlation. Thus the multiple sequence of ON events generated by pixel B is spatially correlated ψ N Mean>The rate of increase with the number of sequences N is greater than the multi-sequence spatial correlation ψ of ON events generated by pixel D N Mean>The speed increasing along with the number N of sequences can solve the problem that the traditional time-space correlation filter can not accurately judge whether the event generated by the edge of the moving object is noise or not.
FIG. 7 shows the multi-sequence spatial correlation ψ of events generated by pixel A, B, C, D for a number N of pulse sequences equal to 10 and 100 N The magnitude relation between the mean value and standard deviation of (c) is shown in the figure:
(1) Multi-sequence spatial correlation ψ of ON events generated by pixel A and pixel B N The mean value of (a) is far higher than other events, which means that ON events (signals) generated by the change of illumination intensity from weak to strong can be distinguished from other events (noise) by setting a reasonable threshold;
(2) When the number of pulse sequences N is large, the event multisequence spatial correlation psi N Is much smaller than the mean, which means that the multi-sequence spatial correlation ψ N The randomness of the value is low, and the probability of attaching false signals and noise labels to the event is low.
The technical principle of the technical scheme is as follows: and obtaining the multi-sequence spatial correlation value of each event through a multi-sequence spatial correlation value operation formula.
The technical effects of the technical scheme are as follows: the multi-sequence spatial correlation value is obtained through calculation, and based on the multi-sequence spatial correlation value, the correctness of each data in the data set is ensured, and the rationality and the correctness of the data stream noise reduction algorithm are further ensured.
The embodiment of the invention provides a data set generation method for evaluating an event stream noise reduction algorithm, which carries out classification statistics on all events in a sub-pulse sequence according to a multi-sequence space correlation value to obtain a multi-sequence space correlation-event number diagram corresponding to the sub-pulse sequence, and comprises the following steps:
classifying all events in the sub-pulse sequence according to the multi-sequence spatial correlation values, classifying the events with the same multi-sequence spatial correlation values into one class, and counting the number of each class of classified events to obtain an event counting result;
And taking the multi-sequence space correlation value as a horizontal axis and the event statistical result as a vertical axis to obtain a multi-sequence space correlation-event number graph corresponding to the sub-pulse sequence.
The technical principle of the technical scheme is as follows: classifying all events in the sub-pulse sequence according to the multi-sequence spatial correlation values, classifying the events with the same multi-sequence spatial correlation values into one class, and counting the number of each class of classified events to obtain an event counting result; and taking the multi-sequence space correlation value as a horizontal axis and the event statistical result as a vertical axis to obtain a multi-sequence space correlation-event number graph corresponding to the sub-pulse sequence.
The technical effects of the technical scheme are as follows: the multi-sequence space correlation-event number graph can more intuitively see the number of events corresponding to a certain multi-sequence space correlation value, and is beneficial to the determination of a subsequent judgment threshold value.
Referring to fig. 8 and 9, an embodiment of the present invention provides a data set generating method for evaluating an event stream noise reduction algorithm, determining a decision threshold of a pulse data set corresponding to a sub-pulse sequence according to a multi-sequence spatial correlation-event number graph, including:
determining a plurality of test thresholds according to the multi-sequence spatial correlation-event number graph;
Selecting one test threshold value from the multiple test threshold values at will as a current test threshold value, and determining the label of each event in the sub-pulse sequence according to the current test threshold value; the tag comprises a signal tag and a noise tag;
drawing a space-time distribution diagram of a signal event and a noise event in a sub-pulse sequence under a current test threshold by using 3D drawing software; the signal event is an event with a signal tag, and the noise event is an event with a noise tag;
observing the definition of the motion trace of the signal event in the space-time distribution diagram;
when the definition of the motion trace of the signal event is higher than the first preset definition and the definition of the motion trace of the noise event is lower than the second preset definition, the current test threshold is used as a judgment threshold of a pulse data set corresponding to the sub-pulse sequence; wherein the first preset definition is higher than the second preset definition.
In this embodiment, the spatiotemporal profile is shown in FIG. 8, with the values on the x-axis and y-axis representing the coordinates of the event and the timestamp axis representing the point in time at which the event is located.
As shown in fig. 9, in this embodiment, the manner of observing the definition of the trace of the signal event and the definition of the trace of the noise event in the spatiotemporal profile may be:
FIG. 9 is a time-space distribution diagram of events during the rotation of a fan blade under extremely dark conditions, and FIG. 9 (a) is a time-space distribution diagram of all events during the rotation of the fan blade when a certain test threshold is used as the current test threshold; from a review of fig. 9 (b), it can be seen that under extremely dim lighting conditions, noise events can occur in large numbers around the object motion trajectory, while random high density noise event blocks can occur in the data set. This means that noise events also have high spatio-temporal correlation (and single sequence spatial correlation ψ) under extremely dim illumination conditions, so it is difficult to effectively distinguish signal events from noise events only by a single shot event sequence; the time-space distribution diagram of the events with the "signal" labels shown in FIG. 9 (c) clearly depicts the rotation process of the fan blades, from which it can be seen that the spatial correlation ψ is based on multiple sequences N The generated signal event contains little noise, and the high-density noise event block is effectively removed due to randomness. Because noise events can appear around the motion trail of the object in a large quantity under the extremely dark illumination condition, the motion trail of the object can be contained in the space-time distribution diagram of the noise events to a certain extent; from an examination of the spatiotemporal profile of events with "noise" labels shown in FIG. 9 (d), it can be seen that the spatial correlation ψ is based on multiple sequences N The trace of the motion track of the object in the generated noise event is shallow, which indicates that almost no signal event is marked with an error label, and indicates that the current test threshold can be used as a judgment threshold.
The technical principle of the technical scheme is as follows: determining a plurality of test thresholds according to the multi-sequence spatial correlation-event number graph; selecting one test threshold value from the multiple test threshold values at will as a current test threshold value, and determining the label of each event in the sub-pulse sequence according to the current test threshold value; the tag comprises a signal tag and a noise tag; drawing a space-time distribution diagram of a signal event and a noise event in a sub-pulse sequence under a current test threshold by using 3D drawing software; the signal event is an event with a signal tag, and the noise event is an event with a noise tag; observing the definition of the motion trace of the signal event in the space-time distribution diagram; when the definition of the motion trace of the signal event is higher than the first preset definition and the definition of the motion trace of the noise event is lower than the second preset definition, the current test threshold is used as a judgment threshold of a pulse data set corresponding to the sub-pulse sequence; wherein the first preset definition is higher than the second preset definition.
The technical effects of the technical scheme are as follows: and determining a judging threshold according to the definition of the motion trace of the signal event and the definition of the motion trace of the noise event, and providing a judging basis for the generation of the next data set.
The embodiment of the invention provides a data set generation method for evaluating an event stream noise reduction algorithm, which generates a pulse data set corresponding to a sub-pulse sequence according to a judgment threshold value and multi-sequence space correlation values of all events in the sub-pulse sequence, and comprises the following steps:
comparing the multi-sequence spatial correlation value of each event in the sub-pulse sequence with a judging threshold value;
when the multi-sequence spatial correlation value of the event is smaller than the judgment threshold value, determining that the label of the event is noise; when the multi-sequence spatial correlation value of the event is not smaller than the judgment threshold value, determining the label of the event as a signal;
all events with noise tags and signal tags are generated as a corresponding pulse data set of the sub-pulse sequence.
The technical principle of the technical scheme is as follows: comparing the multi-sequence spatial correlation value of each event in the sub-pulse sequence with a judging threshold value; when the multi-sequence spatial correlation value of the event is smaller than the judgment threshold value, determining that the label of the event is noise; when the multi-sequence spatial correlation value of the event is not smaller than the judgment threshold value, determining the label of the event as a signal; all events with noise tags and signal tags are generated as a corresponding pulse data set of the sub-pulse sequence.
The technical effects of the technical scheme are as follows: and generating labels for all events according to the judging threshold value, so that the correctness of the labels is ensured, the correctness of the pulse data set is further ensured, and the rationality and the correctness of the data stream noise reduction algorithm are ensured.
Referring to fig. 10, an embodiment of the present invention provides a data set generating method for evaluating an event stream noise reduction algorithm, determining a plurality of test thresholds according to a multi-sequence spatial correlation-event number graph, including:
calculating to obtain an average value of the event number according to the multi-sequence space correlation-event number graph;
when the average value is an integer, taking the average value as a first event number, adding a preset value to the first event number to obtain a second event number, and subtracting the preset value from the first event number to obtain a third event number; acquiring a multi-sequence spatial correlation value corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set;
when the average value is a non-integer, rounding the average value upwards to obtain a rounded average value, taking the rounded average value as a first event number, adding a preset value to the first event number to obtain a second event number, and subtracting the preset value from the first event number to obtain a third event number; acquiring a multi-sequence spatial correlation value corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set;
Taking the event number corresponding to the lowest point of the multi-sequence spatial correlation-event number graph as a fourth event number, adding a preset value to the fourth event number to obtain a fifth event number, and obtaining multi-sequence spatial correlation values corresponding to the fourth event number and the fifth event number to obtain a second test threshold set;
taking the event number corresponding to the highest point of the multi-sequence spatial correlation-event number graph as a sixth event number, subtracting a preset value from the sixth event number to obtain a seventh event number, and obtaining multi-sequence spatial correlation values corresponding to the sixth event number and the seventh event number to obtain a third test threshold set;
and taking the threshold value in the combined set of the first test threshold value set, the second test threshold value set and the third test threshold value set as a plurality of test threshold values.
In this embodiment, the preset value may be any value between 10 and 50, and a plurality of test thresholds are obtained by adding and subtracting preset thresholds, so that the test process is increased, and the rationality of the obtained decision threshold is ensured.
The technical principle of the technical scheme is as follows: calculating to obtain an average value of the event number according to the multi-sequence space correlation-event number graph; when the average value is an integer, taking the average value as a first event number, adding a preset value to the first event number to obtain a second event number, subtracting the preset value from the first event number to obtain a third event number, and obtaining multi-sequence space correlation values corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set; when the mean value is a non-integer, rounding the mean value upwards to obtain a rounded mean value, taking the rounded mean value as a first event number, adding a preset value to the first event number to obtain a second event number, subtracting the preset value from the first event number to obtain a third event number, and obtaining multi-sequence space correlation values corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set; taking the event number corresponding to the lowest point of the multi-sequence spatial correlation-event number graph as a fourth event number, adding a preset value to the fourth event number to obtain a fifth event number, and obtaining multi-sequence spatial correlation values corresponding to the fourth event number and the fifth event number to obtain a second test threshold set; taking the event number corresponding to the highest point of the multi-sequence spatial correlation-event number graph as a sixth event number, subtracting a preset value from the sixth event number to obtain a seventh event number, and obtaining multi-sequence spatial correlation values corresponding to the sixth event number and the seventh event number to obtain a third test threshold set; and taking the threshold value in the combined set of the first test threshold value set, the second test threshold value set and the third test threshold value set as a plurality of test threshold values.
The technical effects of the technical scheme are as follows: by selecting a plurality of test thresholds in different ranges, the accidental is avoided, the correctness of the pulse data set is ensured, and the rationality and the correctness of the data stream noise reduction algorithm are ensured.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (10)

1. A data set generation method for event stream noise reduction algorithm evaluation, comprising:
acquiring video data under a preset illumination condition by using video acquisition equipment;
performing preset operation on the video data to obtain a plurality of sub-pulse sequences; wherein each sub-pulse sequence comprises a plurality of events;
selecting any one of a plurality of sub-pulse sequences, and acquiring multi-sequence space correlation values corresponding to all events in the sub-pulse sequences;
classifying and counting all events in the sub-pulse sequence according to the multi-sequence space correlation value to obtain a multi-sequence space correlation-event number diagram corresponding to the sub-pulse sequence;
Determining a judging threshold value of a pulse data set corresponding to the sub-pulse sequence according to the multi-sequence space correlation-event quantity diagram;
generating a pulse data set corresponding to the sub-pulse sequence according to the judging threshold value and the multi-sequence space correlation value of all events in the sub-pulse sequence;
performing the above operation on each sub-pulse sequence to obtain a pulse data set corresponding to each sub-pulse sequence;
and generating a target pulse data set for evaluation of the event stream noise reduction algorithm according to the pulse data set corresponding to each sub-pulse sequence.
2. The data set generation method for event stream noise reduction algorithm evaluation of claim 1, wherein the video acquisition device is a prophensee EVK4 event camera.
3. The data set generating method for event stream noise reduction algorithm evaluation according to claim 1, wherein capturing video data under preset lighting conditions with a video capturing device comprises:
under the preset illumination condition, the shot object is fixed on a circular disc or an electric sliding rail which rotates periodically, the same movement process of the shot object is determined at the same time, and the movement process of the shot object is repeatedly recorded by utilizing video acquisition equipment to obtain video data.
4. The method for generating a data set for evaluation of an event stream noise reduction algorithm according to claim 1, wherein performing a preset operation on the video data to obtain a plurality of sub-pulse sequences comprises:
and editing the video data to obtain a plurality of sub-videos with completely consistent video content, wherein the plurality of sub-videos are used as a plurality of sub-pulse sequences, and each sub-video corresponds to one sub-pulse sequence.
5. The method for generating a data set for evaluation of an event stream noise reduction algorithm according to claim 1, wherein obtaining a multi-sequence spatial correlation value corresponding to all events in the sub-pulse sequence comprises:
constructing a display array according to the sub-pulse sequence;
selecting any event in a sequence in the sub-pulse sequence as a target event, and acquiring coordinates of the target event in a display array;
determining the polarity of the target event through a preset polarity function according to video data;
determining whether events are generated at adjacent coordinates of associated events corresponding to the target event in a plurality of sub-pulse sequences through a preset first judging function according to the video data, and obtaining a plurality of first judging results;
Determining whether the polarity of an event generated at the adjacent coordinates of the related event of a plurality of sub-pulse sequences is the same as the polarity of the target event or not through a preset second judging function, and obtaining a plurality of second judging results;
setting a convolution kernel with the size of 3 multiplied by 3, and determining a convolution kernel weight;
determining a multi-sequence space correlation numerical operation formula according to the number of the sub-pulse sequences, the preset first judging function, the preset second judging function and the convolution kernel;
bringing a plurality of first judgment results, a plurality of second judgment results and convolution kernel weights into the operation formula to obtain a multi-sequence spatial correlation value corresponding to the target event;
and carrying out the processing process on all the events in the sub-pulse sequence to obtain the multi-sequence space correlation value corresponding to all the events in the sub-pulse sequence.
6. The method for generating a dataset for an evaluation of a noise reduction algorithm of an event stream as claimed in claim 5, wherein a multi-sequence spatial correlation numerical operation formula is determined according to a number of sub-pulse sequences, the preset first judgment function, the preset second judgment function, and the convolution kernel, the multi-sequence spatial correlation numerical operation formula being:
Wherein, psi is N A multi-sequence spatial correlation value representing a target event; n represents the number of a plurality of sub-pulse sequences; m represents an mth sub-pulse sequence in a plurality of sub-pulse sequences; (x) 0 ,y 0 ) Coordinates representing a target event; i has the values of-1, 0 and 1; j has the values of-1, 0 and 1;representing the polarity of the target event; function K (m, x 0 +i,y 0 +j) is a preset first judging function for judging the coordinates in the mth sub-pulse sequence as (x) 0 +i,y 0 Whether an event is generated at the +j) position or not, and obtaining a first judging result; function->Is a preset second judging function for representing that the coordinates in the mth sub-pulse sequence are (x 0 +i,y 0 Whether the polarity of the event generated at +j) is equal to +.>The same, a second judgment result is obtained, wherein ∈>Representing the coordinates (x) in the mth sub-pulse sequence 0 +i,y 0 +j) polarity of the event generated at; omega i,j Representing the weight of the convolution kernel.
7. The method for generating a data set for evaluating an event stream noise reduction algorithm according to claim 1, wherein classifying and counting all events in the sub-pulse sequence according to the multi-sequence spatial correlation value to obtain a multi-sequence spatial correlation-event number map corresponding to the sub-pulse sequence, comprising:
classifying all events in the sub-pulse sequence according to the multi-sequence spatial correlation values, classifying the events with the same multi-sequence spatial correlation values into one class, and counting the number of each class of classified events to obtain an event counting result;
And taking the multi-sequence space correlation value as a horizontal axis and the event statistical result as a vertical axis to obtain a multi-sequence space correlation-event number graph corresponding to the sub-pulse sequence.
8. The data set generating method for event stream noise reduction algorithm evaluation according to claim 1, wherein determining a decision threshold of a pulse data set corresponding to the sub-pulse sequence from the multi-sequence spatial correlation-event number graph comprises:
determining a plurality of test thresholds according to the multi-sequence spatial correlation-event number graph;
selecting one test threshold value from a plurality of test threshold values at will as a current test threshold value, and determining the label of each event in the sub-pulse sequence according to the current test threshold value; the tag comprises a signal tag and a noise tag;
drawing a space-time distribution diagram of a signal event and a noise event in the sub-pulse sequence under the current test threshold by using 3D drawing software; the signal event is an event with a signal tag, and the noise event is an event with a noise tag;
observing the definition of the motion trace of the signal event and the definition of the motion trace of the noise event in the space-time distribution diagram;
when the definition of the motion trace of the signal event is higher than the first preset definition and the definition of the motion trace of the noise event is lower than the second preset definition, the current test threshold is used as a judgment threshold of the pulse data set corresponding to the sub-pulse sequence; wherein the first preset definition is higher than the second preset definition.
9. The method for generating a data set for evaluation of an event stream noise reduction algorithm according to claim 1, wherein generating a pulse data set corresponding to a sub-pulse sequence based on the decision threshold and a multi-sequence spatial correlation value for all events in the sub-pulse sequence, comprises:
comparing the multi-sequence spatial correlation value of each event in the sub-pulse sequence with the decision threshold;
when the multi-sequence spatial correlation value of the event is smaller than the judging threshold value, determining that the label of the event is noise; when the multi-sequence spatial correlation value of the event is not smaller than the judging threshold value, determining the label of the event as a signal;
all events with noise tags and signal tags are taken as corresponding pulse data sets generated by the sub-pulse sequences.
10. The method for generating a dataset for an evaluation of an event stream noise reduction algorithm as claimed in claim 8, wherein determining a plurality of test thresholds from the multi-sequence spatial correlation-event number graph comprises:
calculating to obtain an average value of the event number according to the multi-sequence space correlation-event number graph;
when the average value is an integer, taking the average value as a first event number, adding a preset value to the first event number to obtain a second event number, and subtracting the preset value from the first event number to obtain a third event number; acquiring a multi-sequence spatial correlation value corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set;
When the average value is a non-integer, rounding the average value upwards to obtain a rounded average value, taking the rounded average value as a first event number, adding a preset value to the first event number to obtain a second event number, and subtracting the preset value from the first event number to obtain a third event number; acquiring a multi-sequence spatial correlation value corresponding to the first event number, the second event number and the third event number to obtain a first test threshold set;
taking the event number corresponding to the lowest point of the multi-sequence spatial correlation-event number graph as a fourth event number, adding a preset value to the fourth event number to obtain a fifth event number, and obtaining multi-sequence spatial correlation values corresponding to the fourth event number and the fifth event number to obtain a second test threshold set;
taking the event number corresponding to the highest point of the multi-sequence spatial correlation-event number graph as a sixth event number, subtracting a preset value from the sixth event number to obtain a seventh event number, and obtaining a multi-sequence spatial correlation value corresponding to the sixth event number and the seventh event number to obtain a third test threshold set;
and taking the threshold value in the combined set of the first test threshold value set, the second test threshold value set and the third test threshold value set as a plurality of test threshold values.
CN202310920757.1A 2023-07-25 2023-07-25 Data set generation method for event stream noise reduction algorithm evaluation Active CN116957973B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310920757.1A CN116957973B (en) 2023-07-25 2023-07-25 Data set generation method for event stream noise reduction algorithm evaluation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310920757.1A CN116957973B (en) 2023-07-25 2023-07-25 Data set generation method for event stream noise reduction algorithm evaluation

Publications (2)

Publication Number Publication Date
CN116957973A true CN116957973A (en) 2023-10-27
CN116957973B CN116957973B (en) 2024-03-15

Family

ID=88444153

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310920757.1A Active CN116957973B (en) 2023-07-25 2023-07-25 Data set generation method for event stream noise reduction algorithm evaluation

Country Status (1)

Country Link
CN (1) CN116957973B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140046659A1 (en) * 2012-08-09 2014-02-13 Plantronics, Inc. Context Assisted Adaptive Noise Reduction
CN111770290A (en) * 2020-07-29 2020-10-13 中国科学院长春光学精密机械与物理研究所 Noise reduction method for dynamic vision sensor output event stream
CN113724297A (en) * 2021-08-31 2021-11-30 中国科学院长春光学精密机械与物理研究所 Event camera-based tracking method
CN114881070A (en) * 2022-04-07 2022-08-09 河北工业大学 AER object identification method based on bionic hierarchical pulse neural network
CN114885074A (en) * 2022-05-06 2022-08-09 中国科学院光电技术研究所 Event camera denoising method based on space-time density
CN115375581A (en) * 2022-09-05 2022-11-22 东南大学 Dynamic visual event stream noise reduction effect evaluation method based on event time-space synchronization
CN115442544A (en) * 2022-09-05 2022-12-06 东南大学 Dynamic visual event stream noise reduction method based on hot pixels and enhanced space-time correlation
WO2023092798A1 (en) * 2021-11-25 2023-06-01 成都时识科技有限公司 Noise filtering for dynamic vision sensor

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140046659A1 (en) * 2012-08-09 2014-02-13 Plantronics, Inc. Context Assisted Adaptive Noise Reduction
CN111770290A (en) * 2020-07-29 2020-10-13 中国科学院长春光学精密机械与物理研究所 Noise reduction method for dynamic vision sensor output event stream
CN113724297A (en) * 2021-08-31 2021-11-30 中国科学院长春光学精密机械与物理研究所 Event camera-based tracking method
WO2023092798A1 (en) * 2021-11-25 2023-06-01 成都时识科技有限公司 Noise filtering for dynamic vision sensor
CN114881070A (en) * 2022-04-07 2022-08-09 河北工业大学 AER object identification method based on bionic hierarchical pulse neural network
CN114885074A (en) * 2022-05-06 2022-08-09 中国科学院光电技术研究所 Event camera denoising method based on space-time density
CN115375581A (en) * 2022-09-05 2022-11-22 东南大学 Dynamic visual event stream noise reduction effect evaluation method based on event time-space synchronization
CN115442544A (en) * 2022-09-05 2022-12-06 东南大学 Dynamic visual event stream noise reduction method based on hot pixels and enhanced space-time correlation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
YANG FENG ET AL.: "Event Density Based Denoising Method for Dynamic Vision Sensor", MDPI, 16 March 2020 (2020-03-16) *
徐亮: "脉冲序列式图像传感器的噪声消除研究", 中国优秀硕士学位论文全文数据库 信息科技辑, no. 2, 15 February 2023 (2023-02-15) *

Also Published As

Publication number Publication date
CN116957973B (en) 2024-03-15

Similar Documents

Publication Publication Date Title
Zhang et al. Multi-level fusion and attention-guided CNN for image dehazing
Hu et al. Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries
Wang et al. CDnet 2014: An expanded change detection benchmark dataset
Senst et al. Crowd violence detection using global motion-compensated lagrangian features and scale-sensitive video-level representation
Kim et al. N-imagenet: Towards robust, fine-grained object recognition with event cameras
Schraml et al. A spatio-temporal clustering method using real-time motion analysis on event-based 3D vision
US10909424B2 (en) Method and system for object tracking and recognition using low power compressive sensing camera in real-time applications
Hu et al. Optical flow estimation for spiking camera
Deng et al. Learning from images: A distillation learning framework for event cameras
Zhang et al. Application research of YOLO v2 combined with color identification
CN110414558A (en) Characteristic point matching method based on event camera
CN112084826A (en) Image processing method, image processing apparatus, and monitoring system
Ghosh et al. Multi‐Event‐Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion
Viguier et al. Automatic video content summarization using geospatial mosaics of aerial imagery
Dong et al. Semi-supervised domain alignment learning for single image dehazing
Li et al. Event stream super-resolution via spatiotemporal constraint learning
Boettiger A comparative evaluation of the detection and tracking capability between novel event-based and conventional frame-based sensors
CN105957060B (en) A kind of TVS event cluster-dividing method based on optical flow analysis
Li et al. Sodformer: Streaming object detection with transformer using events and frames
Miao et al. Ds-depth: Dynamic and static depth estimation via a fusion cost volume
CN116957973B (en) Data set generation method for event stream noise reduction algorithm evaluation
Delussu et al. Investigating synthetic data sets for crowd counting in cross-scene scenarios
CN115496920B (en) Adaptive target detection method, system and equipment based on event camera
Sakkos et al. Image editing-based data augmentation for illumination-insensitive background subtraction
Haji-Esmaeili et al. Large-scale monocular depth estimation in the wild

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant