WO2024001970A1 - 一种事件数据处理方法及相关设备 - Google Patents

一种事件数据处理方法及相关设备 Download PDF

Info

Publication number
WO2024001970A1
WO2024001970A1 PCT/CN2023/102176 CN2023102176W WO2024001970A1 WO 2024001970 A1 WO2024001970 A1 WO 2024001970A1 CN 2023102176 W CN2023102176 W CN 2023102176W WO 2024001970 A1 WO2024001970 A1 WO 2024001970A1
Authority
WO
WIPO (PCT)
Prior art keywords
event
matrix
electronic device
events
roi
Prior art date
Application number
PCT/CN2023/102176
Other languages
English (en)
French (fr)
Inventor
王耀园
龙明盛
游凯超
张子阳
王瀛
占云龙
王建民
Original Assignee
清华大学
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学, 华为技术有限公司 filed Critical 清华大学
Publication of WO2024001970A1 publication Critical patent/WO2024001970A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Definitions

  • the present application relates to the field of computers, and in particular, to an event data processing method and related equipment.
  • the event camera detects changes in light intensity within the shooting range through the sensor, and then outputs the coordinates of the pixels where the brightness changes, the timestamp, and the positive and negative polarity of the brightness change as event data.
  • the coordinates of the pixels where the brightness changes, the timestamp, and the positive and negative polarity of the brightness change as event data.
  • event data obtained by event cameras is usually at the microlevel, so more data is obtained and more storage space is occupied.
  • event data is stored as a single pixel. If you only need to read events of pixels in a certain area, you need to traverse all event data, which increases the difficulty of processing event data and reduces the efficiency of event data processing.
  • This application provides an event data processing method and related equipment, which can reduce the storage space occupied by event data and improve the efficiency of reading event data.
  • an event data processing method including: a first electronic device obtains event data captured by an event camera, wherein the event data includes a timestamp of event occurrence, pixel point coordinates, and brightness changes; the first electronic device obtains event data according to Multiple event data within a preset time period generate an event frame matrix, in which the position of the matrix element in the event frame matrix corresponds to the pixel point coordinates, and the value of the matrix element indicates whether the corresponding pixel has an event and whether the pixel has an event.
  • Event polarity if a pixel has an event, it means that the brightness of the pixel has changed within the preset time period, and if there is no event on the pixel, it means that the brightness of the pixel has not changed within the preset time period.
  • the event polarity includes the brightness becoming higher or the brightness becoming lower.
  • the event polarity of a pixel point coordinate can be used to characterize the event data of the pixel point within the preset time period, without the need to record all the event data of the pixel point coordinates within the preset time period one by one, reducing The amount of recorded data is reduced. For example, if the brightness of a certain pixel first becomes high and then becomes low within a preset time period, the event frame matrix only needs to store the result that the brightness of the pixel remains unchanged. There is no need to store each time in the preset time period one by one. event data, thereby reducing the amount of data that needs to be stored.
  • the preset time period range includes 1 to 100 milliseconds.
  • the preset time period can be set according to the requirements of the application scenario, and can be specifically between 1 and 100 milliseconds. For application scenarios with higher accuracy requirements, the preset time can be set shorter, and for application scenarios with lower accuracy requirements, the preset time can be set longer. Furthermore, the preset time period can not only meet the accuracy requirements of the application scenario, but also reduce the amount of data that needs to be stored.
  • the first electronic device determines the storage space occupied by multiple preset storage methods to respectively store the event frame matrix according to the number of pixels; the first electronic device determines the size of the storage space occupied by the event frame matrix from the multiple preset storage methods. Determine a manner of storing the event frame matrix that occupies the smallest storage space; the first electronic device stores the event frame matrix according to the manner of storing the event frame matrix that occupies the smallest storage space.
  • the first electronic device provides multiple storage methods, and can determine the storage method that occupies the smallest storage space according to the number of pixels in the event frame matrix to store the event frame matrix. Furthermore, the most appropriate storage method can be selected according to the situation of each event frame matrix, which reduces the space occupied by storing the event frame matrix.
  • the second electronic device acquires the region of interest RoI; the second electronic device reads the presence or absence of pixels within the RoI from the first electronic device based on the RoI and the method of storing the event frame matrix. The event or the event polarity of the pixel with the event.
  • the second electronic device can directly determine the range of pixels to be read from the event frame matrix based on the RoI, and then read out whether there are events in the pixels within the RoI and the specific event polarity. In this way, the second electronic device does not need to traverse the event data and determine whether the coordinates of each event data are within the RoI. Furthermore, the efficiency of reading the event polarity of pixels within the RoI is improved.
  • the preset storage method includes: the first electronic device stores the coordinates and event polarity of pixel points with events in the event frame matrix.
  • the first electronic device stores the event data within the preset time period according to pixel point coordinates, so that the event frame matrix
  • the position of the matrix element in corresponds to the pixel point coordinates, and the value of the matrix element indicates whether the corresponding pixel has an event and the event polarity of the pixel with an event.
  • the event polarity of the pixel to be read can be read based on the pixel coordinates, which improves the efficiency of reading the event polarity of the pixels within the RoI.
  • only the coordinates and event progress of the pixels with events are stored, instead of storing each pixel in turn, reducing the amount of data that needs to be stored.
  • the second electronic device determines the matrix element coordinate range corresponding to the RoI in the event frame matrix based on the RoI, and the second electronic device reads the RoI from the first electronic device based on the matrix element coordinate range.
  • the event polarity of the pixel to be read can be read according to the pixel coordinates.
  • the second electronic device does not need to traverse the event data and determine the event polarity of each event data in turn. Whether the coordinates are within the RoI. Furthermore, the efficiency of reading the event polarity of the pixels within the RoI is improved, and the efficiency of reading the event polarity of the pixels within the RoI is improved.
  • the preset storage method includes: the first electronic device stores the number of pixels with events in each row of the event frame matrix and in all previous rows, and the number of pixels with events in the event frame matrix.
  • the column coordinates of the pixels and the event polarity or the first electronic device stores the number of pixels with events in each row of the event frame matrix, and the column coordinates of the pixels with events in the event frame matrix and the corresponding event polarity. sex.
  • the second electronic device determines the rows that intersect with the RoI according to the event frame matrix and reads the number of pixels with events in the rows that intersect with the RoI.
  • the second electronic device According to the number of event pixels in the rows that intersect with the RoI, read the column coordinates and event polarity of the event pixels in the RoI from the first electronic device, and then obtain the column coordinates and event polarity of the event pixels in the RoI. Coordinates and event polarity.
  • the number of pixels with events in the first i rows can be stored, and the number of pixels with events in the i-th row can also be stored.
  • the column coordinates of the pixels with events in the i-th row and the storage location of the event polarity can be determined.
  • the column of pixels with events in the i-th row can be determined The location where coordinates and event polarity are stored.
  • the second electronic device does not need to traverse the event data and determine whether the coordinates of each event data are within the RoI. Furthermore, the efficiency of reading the event polarity of the pixels within the RoI is improved, and the efficiency of reading the event polarity of the pixels within the RoI is improved.
  • the preset storage method includes: the first electronic device divides the event frame matrix into multiple sub-matrices according to the event frame matrix, and stores the previous sub-matrix and the total storage occupied by all previous sub-matrices. The size of the space and the coordinates of the pixels with events in each sub-matrix and the event polarity, or the first electronic device stores the size of the storage space occupied by each sub-matrix and the coordinates of the pixels with events in each sub-matrix and the event polarity.
  • the event frame matrix is divided into multiple sub-matrices for storage, which can be to store the storage space occupied by the first m sub-matrices in sequence, or to store the storage space occupied by the m-th sub-matrix in sequence, and then each sub-matrix There are the coordinates of the pixels of the event and the polarity of the event.
  • the maximum number of bits stored for each item of data is smaller, thus reducing the amount of data that needs to be stored. quantity.
  • the second electronic device determines, based on the RoI, the sub-matrix that intersects with the RoI and the storage space occupied by the sub-matrix that intersects with the RoI.
  • the size of the storage space of the sub-matrix determines the storage location where the data of the sub-matrix that intersects with the RoI is stored.
  • the second electronic device reads the pixels with events in the sub-matrix that intersects with the RoI from the first electronic device based on the storage location.
  • the coordinates and event polarity of the point are then obtained to obtain the coordinates and event polarity of the pixels with events in the RoI.
  • the second electronic device does not need to traverse the event data and sequentially determines whether the coordinates of each event data are within the RoI. Improved the efficiency of reading the event polarity of pixels within the RoI, and improved the efficiency of reading the event polarity of the pixels within the RoI.
  • the preset storage method includes: the first electronic device stores an event frame matrix.
  • the complete event frame matrix can be directly stored.
  • the event frame matrix is stored according to pixel coordinates and event polarity. , so that when reading part of the data in the event frame matrix, the low-pixel data that needs to be read can be directly read through the coordinates of the matrix elements in the event frame matrix.
  • the second electronic device determines the matrix element coordinate range corresponding to the RoI in the event frame matrix based on the RoI, and the second electronic device reads the RoI from the first electronic device based on the matrix element coordinate range. Whether each pixel has an event and the event polarity of the pixel with an event.
  • the second electronic device marks the pixels without events as having unchanged brightness according to the coordinates and event polarity of pixels with events in the RoI, and obtains whether each pixel in the RoI has The event and the event polarity of the pixel with the event.
  • the pixels without events within the reading range can also be marked as unchanged brightness and output together.
  • the present application provides an electronic device, including a processor and a memory.
  • the memory is used to store instructions
  • the processor is used to execute the instructions.
  • the processor executes the instructions, the method described in the first aspect is executed.
  • the present application provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are run on an electronic device, the method described in the first aspect is executed.
  • the present application provides a computer program product.
  • the computer program product includes computer instructions. When executed by a computing device, the computing device performs the method described in the first aspect.
  • the event data processing method and related equipment obtain event data within a preset time period and count the brightness changes of each pixel within the preset time period to obtain each pixel.
  • the overall brightness change result of the point is then saved as an event frame matrix according to the position of each pixel point. In this way, the amount of data is reduced, thereby reducing the storage space occupied.
  • the event frame matrix determine the storage method that takes up the smallest space to store the event frame matrix, which can also reduce the storage space occupied.
  • the event frame matrix is stored according to the coordinates of the matrix elements in the event frame matrix. Therefore, when reading some data in the event frame matrix, there is no need to traverse all the stored data and can be read directly as needed.
  • the range determines the storage location of matrix element coordinates and event polarity, improving the efficiency of reading event data.
  • Figure 1A is a schematic diagram of a monitoring screen provided by an embodiment of the present application.
  • Figure 1B is a schematic diagram of a grayscale image generated by an event camera provided by an embodiment of the present application.
  • Figure 2 is a schematic structural diagram of a system for executing an event data processing method provided by an embodiment of the present application
  • FIG. 3 is a schematic flowchart of an event data processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of another event data processing method provided by an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of an event data processing system provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the event camera detects changes in light intensity within the shooting range through the sensor, and then outputs the coordinates of the pixels where the brightness changes, the timestamp, and the positive and negative polarity of the brightness change as event data. Specifically, the brightness value changes of all pixels are continuously obtained through the photosensitive device at consecutive moments. When the brightness value of a certain pixel changes and the brightness change reaches a certain threshold, the camera will output an event data.
  • the threshold of pixel brightness is the preset value of the event camera, and the event data can be represented by elements: x, y, t, p.
  • x and y are used for the location where the event occurs, that is, the coordinates of the pixel
  • t is the time stamp when the event occurs
  • P is used to represent the change in the brightness of the pixel, including the brightness becoming higher and the brightness becoming lower.
  • (x, y, p, t) can represent "at which moment and at which pixel the brightness increased or decreased.”
  • Output in Events stream mode When objects in the scene move or lighting changes cause a large number of pixel brightness changes, a series of events will be generated. Output in Events stream mode. For example, when there are N events, the event stream can be To express, k and N are positive integers. When the event stream accumulates a certain amount, an event frame can be generated, which can reflect the overall change in brightness of each pixel during that time period.
  • DVS cameras Dynamic Vision Sensor (DVS) cameras and Dynamic and Active Pixel Vision Sensor (DAVIS) cameras.
  • DVS cameras are ordinary event cameras that only transmit event data; DAVIS cameras can also transmit grayscale or color images while transmitting event data.
  • Event cameras can capture events at high frame rates, and the time resolution of the event data they output is microseconds. Therefore, event cameras have the characteristics of low time delay and large dynamic range (that is, large differences between light and dark), and can be used in machine vision fields such as autonomous driving, drone visual navigation, industrial inspection, and video surveillance.
  • Figure 1A shows the scene diagram of the monitoring screen when the event camera is used for video surveillance. When there is a change in lighting in the scene that causes a large number of pixel brightness changes, a series of event data will be generated.
  • Figure 1B shows a grayscale image determined based on the event frame, in which dark points indicate that the brightness of the pixel becomes brighter during this time period, and light points indicate that the brightness of the pixel becomes darker during this time period.
  • event data is stored as a single pixel event according to the time when the event occurs. If you only need to read the events of pixel points within a certain coordinate range, you need to traverse all the event data, which increases the processing of the event data. difficulty, reducing the efficiency of event data processing.
  • this application provides an event data processing method, which can be applied to the system architecture as shown in Figure 2.
  • the architecture includes an event camera 110, a storage device 120, and a reading device 130.
  • the storage device 120 can also be called the first electronic device, and the reading device 130 can also be called the second electronic device.
  • the event camera 110 is used to detect changes in light intensity within the shooting range through a sensor, and then output the coordinates of pixels where brightness changes occur, timestamps, and brightness changes as event data. For details, please refer to the detailed description of the event camera above, and will not be repeated here.
  • the storage device 120 includes a memory and a processor.
  • the processor is used to obtain event data from the event camera 110, record the event data of a pixel within a preset time period as the event polarity of the pixel, and then generate the preset time period.
  • the event frame matrix within the event frame matrix includes the event polarity and pixel coordinates of each pixel.
  • the processor is also used to select the storage method that takes up the least space to store the event frame matrix from multiple preset storage methods, and then sends the event frame matrix to the memory, and the memory stores the event frame matrix according to the storage method selected by the processor.
  • Storage devices can be physical servers or cloud devices, such as mobile personal computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA), etc. This application does not place any restrictions on the specific type of the storage device 120.
  • the reading device 130 is used to read the event frame matrix from the storage device 120 .
  • the reading device 130 is also used to obtain the user's Region of Interest (RoI) in the picture captured by the event camera, or to obtain the image of the picture captured by the event camera by an application or other electronic device.
  • RoI Region of Interest
  • the reading device 130 can read the event polarity of the pixels within the RoI range from the storage device 120 according to the storage method of the RoI and the event frame matrix. After the reading device 130 reads out the data, the reading device 130 or other devices can perform further image processing or complete other operations.
  • the reading device 130 may be a mobile phone, a tablet computer, a wearable device, a VR device, a vehicle-mounted device, a notebook computer, a UMPC, a netbook, a PDA, etc. This application does not place any restrictions on the specific type of the reading device 130.
  • the storage device 120 may be a memory and a processor in an event camera. After the event camera generates event data, the processor will directly generate an event frame matrix and determine a storage method, and then store the event frame matrix in in memory.
  • the storage device 120 is the memory of the reading device 130. After the event camera sends the event data to the reading device 130, the processor of the reading device 130 generates an event frame matrix and determines the storage method, and then stores the event frame matrix into the memory. middle. When the reading device 130 needs to use the event data, the event frame matrix will be read from the memory.
  • this application provides an event data processing method, which obtains events within a preset time period from the event camera 110 through the storage device 120 data, convert each pixel The overall change in brightness within the preset time period is recorded as the event polarity.
  • the event polarity includes higher brightness and lower brightness. If the brightness of the pixel does not change, it means there is no event, and the event frame matrix is obtained.
  • the position of each matrix element in the event frame matrix corresponds to the pixel coordinates, and the value of the matrix element corresponds to whether there is an event at the pixel and the specific event polarity.
  • the event polarity of a pixel point coordinate can be used to characterize the event data of the pixel point within the preset time period, without the need to record all the event data of the pixel point coordinates within the preset time period one by one, reducing the need for recording the amount of data. For example, if the brightness of a certain pixel first becomes high and then becomes low within a preset time period, the event frame matrix only needs to store the result that the brightness of the pixel remains unchanged. There is no need to store each time in the preset time period one by one. event data, thereby reducing the amount of data that needs to be stored.
  • the storage device 120 can select a storage method that occupies the smallest space from multiple preset storage methods to store the event frame matrix, which can further reduce the storage space occupied by event data.
  • the storage method of the event frame matrix includes: storing the coordinates and event polarity of the matrix elements with events in the event frame matrix; or, storing the cumulative number of matrix elements with events in each row of the event frame matrix and all rows preceding the row.
  • the event frame matrix into multiple sub-matrices, first determine the storage method of the storage sub-matrix and the storage required to store the sub-matrix space size, and then store the coordinates of the location where each sub-matrix is stored in the storage space, and store the matrix elements with events in each sub-matrix and the corresponding event polarity according to the determined storage method; or, directly store the event frame matrix.
  • the specific storage methods of each storage method will be introduced below and will not be described in detail here.
  • the reading device 130 can also obtain the RoI, and can read the pixels within the RoI in the event frame matrix from the storage device 120 according to the storage method of the RoI and the event frame matrix. Whether there is an event and the specific event polarity. That is to say, the reading device 130 can directly determine the range of pixels to be read from the event frame matrix according to the RoI, and then read out whether there are events in the pixels within the RoI and the specific event polarity. In this way, the reading device 130 does not need to traverse the event data and determine whether the coordinates of each event data are within the RoI. Furthermore, the efficiency of reading the event polarity of pixels within the RoI is improved.
  • S310 The event camera 110 generates multiple event data.
  • the event camera 110 detects changes in light intensity within the shooting range through the sensor, and then outputs the coordinates of pixels with brightness changes, timestamps, and brightness changes as event data.
  • Event data includes the position of the pixel, timestamp, and changes in the brightness of the pixel.
  • a single event data can be represented by (x, y, p, t). Assuming that within the preset time T, there are a total of N event data, then the N event data can be used express.
  • P can be represented by "0, 1", 0 means that the brightness of the pixel has become lower, and 1 means that the brightness of the pixel has become higher; or, P can also be represented by "-1, 1", and -1 represents the pixel. The brightness of the point becomes lower, and 1 means that the brightness of the pixel becomes higher. It should be understood that this application does not specifically limit the method of expressing changes in pixel brightness.
  • S320 The storage device 120 obtains multiple event data within a preset time period from the event camera 110.
  • the event camera 110 continuously generates event data and sends the event data to the storage device 120 for storage.
  • the storage device 120 stores multiple event data within a preset time period together. Therefore, the storage device 120 may first obtain the event data within the preset time period, and then store the event data within the preset time period. , and then obtain the event data of the next preset time period.
  • the storage device 120 may also divide all event data into multiple event data groups according to preset time periods, and store each event data group in sequence. It should be understood that this application does not specifically limit the manner in which the storage device 120 obtains event data.
  • the storage device 120 generates an event frame matrix based on multiple event data within a preset time period.
  • the storage device 120 determines whether there is an event at each pixel according to the overall brightness change of each pixel within the preset time period, and if there is an event, determines the specific event polarity, and then obtains the event frame matrix.
  • the event polarity includes the brightness becoming high or the brightness becoming low. If the brightness of the pixel does not change, there is no event.
  • the position of each matrix element in the event frame matrix represents the coordinates of the pixel, and the value of the matrix element represents whether there is an event at the corresponding pixel and the specific event polarity.
  • the preset time period can be set according to the requirements of the application scenario, and can be specifically between 1 and 100 milliseconds. For application scenarios with higher accuracy requirements, the preset time can be set shorter, and for application scenarios with lower accuracy requirements, the preset time can be set longer. Furthermore, the preset time period can meet the accuracy requirements of the application scenario.
  • the event data the brightness of the pixels becoming higher is called a positive event, and the brightness of the pixels becoming lower is called a negative event, where, the event data It is generated when the brightness change of the pixel exceeds the threshold. Specifically, it is obtained by taking the logarithm of the brightness value and comparing it. It can be understood that the amount of brightness increase corresponding to each positive event is the same, and the amount corresponding to each negative event is The amount by which the brightness becomes lower is the same.
  • the event polarity of the pixel within the preset time T is that the brightness becomes higher; when each pixel is at the preset time T When the number of positive events is less than the number of negative events, the event polarity of the pixel within the preset time T is that the brightness becomes lower; when the number of positive events for each pixel within the preset time T is equal to the number of negative events, including none In the case of positive events and negative events, that is, the number of positive events and the number of negative events are both zero, and the pixel has no events within the preset time T.
  • the event frame matrix can be determined based on whether there is an event at each pixel and the specific event polarity and pixel location.
  • the value of each matrix element in the event frame matrix can be calculated using a symbolic function, as shown in formula (1).
  • the image pixels acquired by the event camera are W ⁇ H. That is to say, the event frame matrix F has a total of H rows. Column W, H and W are positive integers.
  • Formula (1) shows the calculation method of matrix element F[i, j] in the i-th row and j-th column, 0 ⁇ i ⁇ H, 0 ⁇ j ⁇ W, and i and j are positive integers.
  • the storage device 120 counts the brightness changes of each pixel within a certain period of time to obtain the overall brightness change result of each pixel, and then calculates the overall brightness change corresponding to the pixel according to the position of each pixel.
  • the results are saved as event frame matrices, reducing the amount of data and thus the storage space occupied.
  • the storage device 120 determines the event frame matrix storage method according to the number of pixels, and stores the event frame matrix.
  • the storage device 120 determines how to store the event frame matrix based on the number of pixels, that is, the number of matrix elements in the event frame matrix. This method provides a variety of storage methods for preset event frame matrices. The storage device 120 can first calculate the storage space occupied by each storage method, and then select the method that occupies the smallest storage space to store the event frame matrix.
  • the storage method of the event frame matrix includes: storing the coordinates of the matrix elements with events in the event frame matrix and the corresponding event polarity; or, storing the cumulative matrix elements with events in each row of the event frame matrix and all rows preceding the row. number, as well as the column coordinates of the matrix elements with events in the event frame matrix and the corresponding event polarity; alternatively, the event frame matrix is divided into multiple sub-matrices, and the coordinates of the starting matrix element of each sub-matrix and the coordinates of the starting matrix elements in each sub-matrix are stored. The matrix elements of the event and the corresponding event polarity; or, directly store the event frame matrix.
  • each event frame matrix and the space occupied by each storage method are described in detail below.
  • the event frame matrix F has a total of H rows and W columns.
  • E is a positive integer.
  • the bit width of W be denoted as B W
  • the bit width of H as B H
  • the bit width of the total amount of events E as B E
  • the event polarity of the matrix elements with events as Q′
  • the bit width of Q′ Denote it as B Q′ .
  • the bit width is the minimum number of bits to record the positive integer, which can be obtained by taking the logarithm of 2 of the positive integer. It should be understood that there are two possibilities for the event polarity Q′, including the brightness becoming lower and the brightness becoming higher. Usually Q′ is recorded with one bit, that is, B Q′ is equal to 1.
  • the storage method When the storage method is to store the coordinates and event polarity of matrix elements with events in the event frame matrix, that is, storing the abscissa, ordinate, and event polarity of matrix elements with events, ⁇ bits of data are required for each pixel.
  • ⁇ is equal to the sum of B W , B H , and B Q′ .
  • the space S 1 occupied by this storage method is the product of E and ⁇ .
  • the calculation method of ⁇ can refer to the following formula (2)
  • the calculation method of S 1 can refer to the following formula (3).
  • S 1 E* ⁇ (3)
  • the storage method is to store the cumulative number of matrix elements with events in each row of the event frame matrix and all rows before the row, as well as the column coordinates of the matrix elements with events in the event frame matrix and the corresponding event polarity. Specifically, it includes: first using H+1 numbers to record the number of matrix elements with events in the first i rows, where the i+1th number records the number of matrix elements with events in the first i rows of the matrix (0 ⁇ i ⁇ H), each of the H+1 numbers requires B E bits. Then use E numbers to record the column coordinates and event polarity of the matrix element with the event. Each number in the E numbers requires B W + B Q' bits. Then the space occupied by this storage method is S 2.
  • the calculation method can refer to the following formula (4).
  • S 2 (H+1)*B E +E*(B W +B Q′ ) (4)
  • the storage method is to divide the event frame matrix into multiple sub-matrices, first determine the storage method of the sub-matrix and the size of the storage space required to store the sub-matrix, and then store the coordinates of the location where each sub-matrix is stored in the storage space, and according to The determined storage method stores the matrix elements with events in each sub-matrix and the corresponding event polarity.
  • the event frame matrix F can be divided into n h ⁇ n w sub-matrices, and the number of matrix elements with events in each sub-matrix is recorded as e′, where , n h is equal to H/h′, n w is equal to W/w′, the bit width of w′ is B w′ , and the bit width of h′ is B h′ .
  • h′ ⁇ w′ bits can be used to record whether each matrix element in the sub-matrix has an event, and then e′ bits can be used to record the event polarity of the matrix element with an event, which requires a total of w′ ⁇ h′+ e′ bits.
  • ⁇ bits are used to record the coordinates of the matrix elements with events in the sub-matrix and the event polarity, where ⁇ is equal to the sum of B w′ , B h′ , and B Q′ .
  • the method of storing sub-matrices can be determined based on the number e' of matrix elements with events in each sub-matrix.
  • e' w′h′/B w′ +B h′ +B Q′ -1 (6)
  • h′ ⁇ w′ bits are used to record whether each matrix element in the submatrix has an event, and then e′ bits are used to record the matrix elements with events.
  • the event polarity storage method occupies less storage space.
  • the storage method of using ⁇ bits to record the coordinates of the matrix elements with events in the sub-matrix and the polarity of the events takes up less storage space.
  • the size of the storage space occupied by all sub-matrices can be determined. First, use n h′ ⁇ n w′ +1 numbers to record the starting position of each sub-matrix in the storage space, then The starting and ending positions of the m-th submatrix in the storage space can be obtained by reading the m-th number and the m+1-th number. The matrix elements in each sub-matrix are then stored separately according to the sub-matrix storage method determined above. Among them, the total occupied space is recorded as D, and the bit width of D is recorded as B D . Then the space occupied by this storage method is S 3. The calculation method can refer to the following formula (7).
  • the size of the storage space occupied by each sub-matrix can be recorded using n h′ ⁇ n w′ numbers.
  • the matrix elements in each sub-matrix are then stored separately according to the sub-matrix storage method determined above. Among them, the bit width of the space occupied by each sub-matrix is B 2*h′*w′ . Then the space occupied by this storage method is S′ 3.
  • the calculation method can refer to the following formula (8).
  • the storage device 120 after storing the event frame matrix, the storage device 120 also reads the event frame matrix when it needs to process the event frame matrix, or reads part of the content in the event frame matrix, for example, reading the event frame matrix.
  • Event polarity of matrix elements within the region of interest RoI The event polarity for reading the matrix elements within the RoI is described in detail below. It should be understood that reading the complete data of the event frame matrix can refer to the following reading of event polarity of matrix elements within the RoI, and reading the complete data of the event frame matrix will not be described again here.
  • the reading device 130 obtains the RoI and reads the events of the pixels in the RoI from the storage device 120.
  • the reading device 130 obtains the RoI.
  • the RoI can be any area in the picture captured by the event camera, such as the area selected by the user when viewing the picture captured by the event camera through the reading device 130.
  • the application program (such as image viewing) in the reading device 130 program, image processing program, etc.).
  • the reading device 130 determines the matrix element range of the RoI in the event frame matrix based on the RoI, and reads the event polarity of each pixel in the RoI based on the RoI and the method of storing the event frame matrix.
  • the reading device 130 can determine the abscissa and ordinate range of the matrix element to be obtained based on the RoI. For the RoI, The reading device 130 reads the event polarity of the matrix element and outputs the read result.
  • this storage method can only read the event polarity of the matrix element with an event, whether the brightness becomes lower or the brightness becomes higher.
  • the matrix element points in the RoI that do not read the event polarity the pixel corresponding to the matrix element The brightness remains unchanged.
  • the matrix element points for which the event polarity is not read are also marked as unchanged brightness, and then the event data of all matrix element points within the RoI are output together.
  • the storage method is to store the cumulative number of matrix elements with events in each row of the event frame matrix and all rows before the row, as well as the column coordinates of the matrix elements with events in the event frame matrix and the corresponding event polarity.
  • the reading device 130 is Read in row units. Specifically, it includes: first, determine the rows that intersect with the RoI. For the i-th row, directly read the i-th number and i+1-th number to get the number of events in the row, column coordinates, and event polarity storage. position, read the column coordinates and event polarity of each matrix element in the row of data, and then determine the columns that intersect with the RoI, and then determine the matrix elements that need to be output and the corresponding column coordinates.
  • the reading device 130 You need to read the first i+1 number to get the number of events in the row, the location where the column coordinates and event polarity are stored, and read the column coordinates and event polarity of each matrix element in the row of data. , and then determine the columns that intersect with the RoI, and then determine the matrix elements that need to be output and the corresponding column coordinates.
  • the matrix elements read by this storage method only include matrix elements with events.
  • matrix element points whose event polarity is not read are also marked as unchanged brightness, and then, The event data of all matrix element points within the RoI are output together.
  • the storage method is to divide the event frame matrix into multiple sub-matrices, first determine the storage method of the sub-matrix and the size of the storage space required to store the sub-matrix, and then store the coordinates of the location where each sub-matrix is stored in the storage space, and according to When storing in the determined storage mode, when each sub-matrix contains matrix elements of events and corresponding event polarities, the reading device 130 reads in sub-matrix units.
  • the details include: first determine the sub-matrix that intersects with the RoI. For the m-th matrix, directly read the m-th number and m+1-th number to get the starting position and end position of the sub-matrix. According to the storage method of each intersection sub-matrix, read the coordinates and event polarity of the matrix elements of the intersection sub-matrices.
  • the reading device 130 can determine the storage method used by the sub-matrix based on the size of the space occupied by the storage sub-matrix.
  • the critical point of the space occupied by the two sub-matrices is S 0 .
  • the storage method used is It is h′ ⁇ w′ bits to record whether the value of each matrix element in the sub-matrix has an event, and then e′ bits are used to record the event polarity of the matrix element with an event.
  • the storage space occupied by the sub-matrix is less than S 0 , a storage method is adopted in which ⁇ bits are used to record the coordinates of the matrix elements with events in the sub-matrix and the polarity of the events.
  • h′ ′ bits determine the matrix elements with events, and then use the last e′ bits to determine whether the specific event polarity becomes higher or lower in brightness. For matrix elements whose brightness does not change among h′ ⁇ w′ bits, it is directly The corresponding pixel brightness of the output matrix element remains unchanged. Furthermore, the event polarity of each matrix element point in the sub-matrix with intersection of RoI can be obtained.
  • the storage method is to directly store the event frame matrix F
  • the data of all pixels corresponding to the RoI area can be directly read, and then the event polarity of each matrix element point is determined and output.
  • the event data processing method obtains event data within a preset time period through the storage device 120, and counts the brightness changes of each pixel within the preset time period to obtain each pixel.
  • the overall brightness change result of the point is then saved as an event frame matrix according to the position of each pixel point. In this way, the amount of data is reduced, thereby reducing the storage space occupied.
  • the event frame matrix determine the storage method that takes up the smallest space to store the event frame matrix, which can also reduce the storage space occupied.
  • the event frame matrix is stored according to the coordinates of the matrix elements in the event frame matrix.
  • the reading device 130 when reading part of the data in the event frame matrix, the reading device 130 does not need to traverse all the stored data and can directly
  • the storage location of matrix element coordinates and event polarity is determined according to the range that needs to be read, which improves the efficiency of reading event data.
  • S410 The storage device 120 obtains the event data stream.
  • the storage device 120 obtains the event data stream output by the event camera.
  • Each event data in the event data stream includes the position of the pixel, the timestamp, and the brightness change of the pixel.
  • the event data can be represented by (x, y, p, t).
  • S420 The storage device 120 compresses the event data stream into frames and generates an event frame matrix.
  • the number of microseconds in the timestamp of event 1 is 262600 ⁇ s, that is, 262.6ms.
  • the time window includes all events from 262.6ms to 267.6ms, that is, the events falling within the time window are event numbers 1, 2, 3, and 4. of four events. After frame compression, a 2 ⁇ 2 event frame as shown in Table 2 can be obtained.
  • event 1 and event 3 are both events of the pixel point with coordinates (1, 1). Adding the brightness changes of event 1 and event 3, we can get that within the preset time period of 5 milliseconds, the coordinates are ( The event polarity of the pixels 1, 1) is 0, that is, the brightness remains unchanged.
  • the event frame shown in Figure 2 can also be expressed as an event frame matrix F', and F' is shown in the following formula (11).
  • the storage device 120 determines the storage method of the event frame matrix, and stores the event frame matrix.
  • step S340 the storage device 120 determines the event frame matrix storage method according to the number of pixels, and stores the event frame matrix. Next, the event frame matrix F′ is analyzed using the storage space occupied by each storage method. Line details.
  • the storage device 120 stores the cumulative number of matrix elements with events in each row of the event frame matrix and all rows preceding the row, as well as the column coordinates of the matrix elements with events in the event frame matrix and the corresponding event polarity, because the event frame matrix F′ has a total of 2 rows, so it is necessary to first use 3 numbers to record the number of matrix elements of the cumulative brightness change in each row, and each number requires 2 bits. There are two matrix elements with events in the event frame matrix F′. It is necessary to use 2 numbers to record the column coordinates and event polarity of the matrix elements whose brightness changes. Each number requires 2 bits. Then according to formula (4), it can be determined that the storage device 120 requires 10 bits to store the event frame matrix F'.
  • the storage device 120 divides the event frame matrix into multiple sub-matrices, and when storing the starting matrix element coordinates of each sub-matrix and the matrix elements with events in each sub-matrix and the corresponding event polarity, the sub-matrix size may be 2 ⁇ 2, that is, There is only one submatrix, and 2 numbers are needed to store the starting matrix element position of the data block.
  • the size of the submatrix can be 2 ⁇ 2, 4 bits can be used to record whether the value of each matrix element in the submatrix has an event, and because there are two matrix elements with events, 2 can be used Bits record the event polarity of the matrix element of the event.
  • This sub-matrix requires a total of 6 bits. Or, for each matrix element in the sub-matrix, store the coordinates and event polarity of the matrix element with an event in the sub-matrix. There are two matrix elements. Each matrix element requires 1 bit to record the abscissa and 1 bit to record the event. On the ordinate, 1 bit of event polarity also requires 6 bits.
  • the storage device 120 requires 6 bits to store the sub-matrix, and 3 bits are needed to record the sub-matrix storage space size. According to formula (7), it can be determined that the storage device 120 requires 12 bits to store the event frame matrix F′. bits.
  • the storage device 120 directly stores the event frame matrix F', because the event frame matrix F' has a total of 2 rows and 2 columns. According to formula (9), the storage device 120 requires 8 bits to store the event frame matrix F'.
  • the storage device 120 uses the first storage method to store the event frame matrix F′, that is, the electronic device stores the coordinates and event polarity of the matrix elements with events in the event frame matrix F′, which takes up less storage space.
  • the reading device 130 obtains the RoI and reads the events of the pixels in the RoI.
  • the reading device 130 obtains that the coordinates of the RoI are (1, 1), because the storage device 130 uses the first storage method to store the event frame matrix F′, which stores the coordinates and event polarity of the matrix elements of the brightness change, that is, the storage are (2, 1, -1) and (1, 2, + 1), because the coordinate (1, 1) is not in the stored event list, the reading device 130 can determine the pixel at the coordinate (1, 1) The brightness of the point remains unchanged. Furthermore, the event of the pixel in the RoI can be output as (1, 1, 0).
  • the event data processing method counts the brightness changes of each pixel within a preset time period, obtains the overall brightness change result of each pixel, and generates an event frame matrix. In this way, the amount of data is reduced, thereby reducing the storage space occupied. Then, according to the event frame matrix, determine the storage method that takes up the smallest space to store the event frame matrix, which can also reduce the storage space occupied. Moreover, the event frame matrix is stored according to the coordinates of the matrix elements in the event frame matrix. Therefore, when reading some data in the event frame matrix, there is no need to traverse all the stored data and can be read directly as needed. The range determines the storage location of matrix element coordinates and event polarity, improving the efficiency of reading event data.
  • the first matrix element of each submatrix has an event.
  • the storage device 120 stores according to the cumulative number of matrix elements with events in each row of the event frame matrix, as well as the column coordinates and corresponding event polarity of the matrix elements with events in the event frame matrix, 721 numbers are needed to record each event.
  • the cumulative number of matrix elements with events in a row requires 13 bits for each number.
  • 7200 numbers are used to record the column coordinates and event polarity of the matrix elements with events.
  • the column coordinates require 10 bits, 1 bit is required for event polarity.
  • For each matrix element in the sub-matrix, to record the coordinates and event polarity of the matrix element with an event in the sub-matrix, only B w′ +B h′ +B Q′ 8 bits are needed. Therefore, the second submatrix storage method takes up less space.
  • choosing the second storage method to store the event frame matrix takes up less space.
  • the event data processing method provided by this application can determine the storage method that occupies the smallest space for storing the event frame matrix based on the event frame matrix, and can also reduce the occupation of storage space.
  • the event data processing system 1000 includes an event camera input module 1010, Event data frame compression module 1020, event frame compression module 1030, and event data reading module 1040.
  • the event camera input module 1010 is used to obtain event data output by the event camera.
  • the event data includes the position of the pixel, the timestamp, and the change in brightness of the pixel. It is specifically used to execute the method described in step S310 and step S320, which will not be described again here.
  • the event data frame compression module 1020 is used to determine the event polarity of each pixel point based on the overall brightness change of each pixel point within a preset time period, and obtain an event frame matrix. It is specifically used to execute the method described in step S330, which will not be described again here.
  • the event frame compression module 1030 is used to determine a method of storing the event frame matrix based on the number of pixels, that is, the number of matrix elements in the event frame matrix. It is specifically used to execute the method described in step S340, which will not be described again here.
  • the event data reading module 1040 is used to obtain the RoI, determine the range of matrix elements corresponding to the RoI in the event frame matrix, and read the event polarity of each pixel in the RoI according to the RoI and the method of storing the event frame matrix. It is specifically used to execute the method described in step S350, which will not be described again here.
  • the event data processing system counts the brightness changes of each pixel within a preset time period, obtains the overall brightness change result of each pixel, and generates an event frame matrix. In this way, the amount of data is reduced, thereby reducing the storage space occupied. Then, according to the event frame matrix, determine the storage method that takes up the smallest space to store the event frame matrix, which can also reduce the storage space occupied. Moreover, the event frame matrix is stored according to the coordinates of the matrix elements in the event frame matrix. Therefore, when reading some data in the event frame matrix, there is no need to traverse all the stored data and can be read directly as needed. The range determines the storage location of matrix element coordinates and event polarity, improving the efficiency of reading event data.
  • FIG. 6 is a schematic structural diagram of an electronic device 1200 provided by this application.
  • the electronic device 1200 may be the event camera 110, the storage device 120, and the reading device 130 mentioned above, or a combination of the above devices.
  • the electronic device 1200 includes: a processor 1210, a communication interface 1220, a memory 1230, and an event camera 1250.
  • the processor 1210, the communication interface 1220, the memory 1230 and the event camera 1250 can be connected to each other through the internal bus 1240, or can communicate through other means such as wireless transmission.
  • the embodiment of this application takes the connection through bus 1240 as an example.
  • the bus 1240 can be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus ( unified bus, Ubus or UB), computer express link (compute express link, CXL), cache coherent interconnect for accelerators (CCIX), etc.
  • PCIe Peripheral Component Interconnect Express
  • EISA extended industry standard architecture
  • a unified bus unified bus, Ubus or UB
  • computer express link compute express link
  • CXL cache coherent interconnect for accelerators
  • Bus 1240 can be divided into address bus, data bus, control bus, etc.
  • the bus 1240 may also include a power bus, a control bus, a status signal bus, etc.
  • the various buses are labeled bus 1240 in the figure.
  • the processor 1210 may include at least one general-purpose processor, such as a central processing unit (Central Processing Unit, CPU), or a combination of a CPU and a hardware chip.
  • the above-mentioned hardware chip may be an application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), a programmable logic device (Programmable Logic Device, PLD), or a combination thereof.
  • the above-mentioned PLD can be a complex programmable logic device (CPLD) or a field programmable gate array. (Field-Programmable Gate Array, FPGA), Generic Array Logic (GAL), or any combination thereof.
  • the processor 1210 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 1230, which enable the electronic device 1200 to provide a variety of services.
  • the memory 1230 is used to store program codes and event frame matrices, and is controlled and executed by the processor 1210 to perform the processing steps of the event data processing method in the above embodiment.
  • the program code may include one or more software modules, and the one or more software modules may be the software modules provided in the embodiment of FIG. 6 .
  • this embodiment can be implemented by a general-purpose physical server, such as an ARM server or an A complete computer system with complete hardware system functions and running in a completely isolated environment is not specifically limited in this application.
  • the memory 1230 may include volatile memory (Volatile Memory), such as random access memory (Random Access Memory, RAM); the memory 1230 may also include non-volatile memory (Non-Volatile Memory), such as read-only memory (Read-Only Memory). Only Memory (ROM), Flash Memory (Flash Memory), Hard Disk Drive (HDD) or Solid-State Drive (SSD); the memory 1230 may also include a combination of the above types.
  • volatile memory volatile memory
  • RAM random access memory
  • non-Volatile Memory Non-Volatile Memory
  • Read-Only Memory read-only memory
  • Only Memory ROM
  • Flash Memory Flash Memory
  • HDD Hard Disk Drive
  • SSD Solid-State Drive
  • the memory 1230 may store program codes to specifically execute steps S310 to S350 and their optional steps in the embodiment of FIG. 3, which will not be described again here.
  • the communication interface 1220 may be a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIe) bus interface), a wired interface (such as an Ethernet interface), or a wireless interface (such as a cellular network interface or using a wireless LAN interface) for communicating with other devices or modules.
  • a wired interface such as an Ethernet interface
  • PCIe Peripheral Component Interconnect express
  • FIG. 6 is only a possible implementation manner of the embodiment of the present application.
  • the electronic device 1200 may also include more or fewer components, which is not limited here.
  • content not shown or described in the embodiment of the present application please refer to the relevant explanations in the embodiment of FIG. 3 and will not be described again here.
  • the electronic device shown in Figure 6 can also be a computer cluster composed of at least one server, which is not specifically limited in this application.
  • Embodiments of the present application also provide a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instruction is run on a processor, the method flow shown in Figure 3 is implemented.
  • An embodiment of the present application also provides a computer program product.
  • the computer program product is run on a processor, the method flow shown in Figure 3 is implemented.
  • the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination.
  • the above-described embodiments may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer Readable storage media can be any available media that can be accessed by a computer or data storage devices such as servers and data centers that contain one or more sets of available media.
  • the available media can be magnetic media (for example, floppy disks, hard disks, tapes ), optical media (for example, high-density digital video disc (DVD)), or semiconductor media.
  • the semiconductor media may be SSD.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供了一种事件数据处理方法，包括：获取事件相机拍摄的事件数据，其中，事件数据包括事件发生的时间戳、像素点坐标和亮度变化情况；根据预设时间段内的多个事件数据，生成事件帧矩阵，其中，事件帧矩阵中的矩阵元素的位置对应像素点坐标，矩阵元素的值表示对应像素点有无事件以及有事件的像素点的事件极性，像素点有事件表示像素点在预设时间段内亮度发生变化，像素点无事件表示像素点在预设时间段内亮度无变化，事件极性包括亮度变高、亮度变低。这样，无需逐个记录该像素点坐标在预设时间段内的所有事件数据，减小了需要记录的数据的数量。

Description

一种事件数据处理方法及相关设备
本申请要求于2022年06月30日提交中国专利局、申请号为202210778563.8、申请名称为“一种事件数据处理方法及相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机领域，尤其涉及一种事件数据处理方法及相关设备。
背景技术
事件相机(event camera)通过传感器检测拍摄范围内的光强度变化，然后将发生亮度变化的像素点坐标、时间戳和亮度变化的正负极性输出为事件数据。当场景中有物体运动或光照改变造成大量像素亮度变化时，则会产生一系列的事件，这些事件以事件流(Events stream)方式输出。
但是，事件相机获得的事件数据时间分辨率通常为微妙级，因此获得的数据会较多，会占用较多存储空间。并且，事件数据是以单个像素点进行存储的，如果只需要读取某一区域的像素点的事件，需要遍历全部事件数据，增加了对事件数据的处理难度，降低了事件数据处理效率。
因此，如何减少事件数据占用存储空间以及提高读取事件数据的效率，是亟待解决的问题。
发明内容
本申请提供了一种事件数据处理方法及相关设备，能减少事件数据占用存储空间以及提高读取事件数据的效率。
第一方面，提供一种事件数据处理方法，包括：第一电子设备获取事件相机拍摄的事件数据，其中，事件数据包括事件发生的时间戳、像素点坐标和亮度变化情况；第一电子设备根据预设时间段内的多个事件数据，生成事件帧矩阵，其中，事件帧矩阵中的矩阵元素的位置对应像素点坐标，矩阵元素的值表示对应像素点有无事件以及有事件的像素点的事件极性，像素点有事件表示像素点在预设时间段内亮度发生变化，像素点无事件表示像素点在预设时间段内亮度无变化，事件极性包括亮度变高、亮度变低。
在上述方案中，通过一个像素点坐标的事件极性，即可表征预设时间段内该像素点的事件数据，而无需逐个记录该像素点坐标在预设时间段内的所有事件数据，减小了记录的数据的数量。比如，某像素点在预设时间段内亮度先变高，再变低，事件帧矩阵中只用存储该像素点亮度不变这个结果即可，不用一一存储该预设时间段内每一次事件数据，进而减少了需要存储的数据量。
结合第一方面，在一些实现方式中，预设时间段范围包括1至100毫秒。
在上述方案中，预设时间段可以根据应用场景需求进行设定，具体可以是在1至100毫秒之间。对于精度要求较高的应用场景，预设时间可以设置得较短，对于精度要求较低的应用场景，预设时间可以设置得较长。进而，使得预设时间段既能满足应用场景的精度需求，又能减少需要存储的数据量。
结合第一方面，在一些实现方式中，第一电子设备根据像素点数量，确定多个预设存储方式分别存储事件帧矩阵占用的存储空间大小；第一电子设备从多个预设存储方式中确定存储事件帧矩阵占用的存储空间最小的方式；第一电子设备根据存储事件帧矩阵占用的存储空间最小的方式，存储事件帧矩阵。
在上述方案中，第一电子设备提供了多种存储方式，能根据事件帧矩阵中像素点的个数，确定占用存储空间最小的存储方式来存储事件帧矩阵。进而，能根据每个事件帧矩阵的情况，选择最合适的存储方式，减少了存储事件帧矩阵的占用空间。
结合第一方面，在一些实现方式中，第二电子设备获取感兴趣区域RoI；第二电子设备根据RoI和存储事件帧矩阵的方式，从第一电子设备中读取RoI内的像素点有无事件或者有事件的像素点的事件极性。
在上述方案中，第二电子设备能直接根据RoI从事件帧矩阵中确定要读取的像素点范围，然后读取出RoI内像素点是否有事件以及具体的事件极性。这样，第二电子设备不用遍历事件数据，依次判断每个事件数据的坐标是否在RoI内。进而，提高了读取RoI内像素点的事件极性的效率。
结合第一方面，在一些实现方式中，预设存储方式包括：第一电子设备存储事件帧矩阵中有事件的像素点的坐标和事件极性。
在上述方案中，第一电子设备将预设时间段内的事件数据按像素点坐标进行存储，使得事件帧矩阵 中的矩阵元素的位置对应像素点坐标，矩阵元素的值表示对应像素点有无事件以及有事件的像素点的事件极性。进而，在读取像素点的事件极性时，能根据像素点坐标读取需要读取的像素点的事件极性，提高了读取RoI内像素点的事件极性的效率。并且，只存储有事件的像素点的坐标和事件进行，不用依次存储每个像素点，减少了需要存储的数据量。
结合第一方面，在一些实现方式中，第二电子设备根据RoI，确定事件帧矩阵中RoI对应的矩阵元素坐标范围，第二电子设备根据矩阵元素坐标范围，从第一电子设备中读取RoI内有事件的像素点的坐标和事件极性。
在上述方案中，在读取像素点的事件极性时，能根据像素点坐标读取需要读取的像素点的事件极性，第二电子设备不用遍历事件数据，依次判断每个事件数据的坐标是否在RoI内。进而，提高了读取RoI内像素点的事件极性的效率，提高了读取RoI内像素点的事件极性的效率。
结合第一方面，在一些实现方式中，预设存储方式包括：第一电子设备存储事件帧矩阵中每一行以及前面所有行中有事件的像素点的个数，以及事件帧矩阵中有事件的像素点的列坐标和事件极性，或者，第一电子设备存储事件帧矩阵中每行有事件的像素点的个数，以及事件帧矩阵中有事件的像素点的列坐标和对应的事件极性。
在上述方案中，根据每行中有事件的像素点的个数、有事件的像素点的列坐标和事件极性进行存储，不用依次存储每个有事件的像素点，减少了需要存储的数据量。
结合第一方面，在一些实现方式中，第二电子设备根据事件帧矩阵，确定与RoI有交集的行并读取与RoI有交集的行中有事件的像素点的个数，第二电子设备根据与RoI有交集的行中有事件的像素点的个数，从第一电子设备中读取RoI内有事件的像素点的列坐标和事件极性，进而得到RoI内有事件的像素点的坐标和事件极性。
在上述方案中，可以存储前i行有事件的像素点的个数，也能存储第i行有事件的像素点的个数。存储前i行有事件的像素点的个数时，通过确定需要读取的行，就能确定第i行有事件的像素点的列坐标和事件极性的存储位置。存储第i行有事件的像素点的个数时，通过确定需要读取的行，将前i行有事件的像素点的个数相加，就能确定第i行有事件的像素点的列坐标和事件极性的存储位置。第二电子设备不用遍历事件数据，依次判断每个事件数据的坐标是否在RoI内。进而，提高了读取RoI内像素点的事件极性的效率，提高了读取RoI内像素点的事件极性的效率。
结合第一方面，在一些实现方式中，预设存储方式包括：第一电子设备根据事件帧矩阵，将事件帧矩阵分为多个子矩阵，存储前一个子矩阵以及前面所有子矩阵总共占用的存储空间大小以及每个子矩阵中有事件的像素点的坐标以及事件极性，或者，第一电子设备存储每个子矩阵占用的存储空间大小以及每个子矩阵中有事件的像素点的坐标以及事件极性。
在上述方案中，将事件帧矩阵分为多个子矩阵进行存储，可以是依次存储前m个子矩阵占用的存储空间大小，也可以是依次存储第m个子矩阵占用的存储空间大小，然后每个子矩阵中有事件的像素点的坐标以及事件极性。这样，存储子矩阵中有事件的像素点的坐标以及事件极性时，因为数据量小于完整的事件帧矩阵的数据量，因此存储每项数据最大比特量较小，进而减少了需要存储的数据量。
结合第一方面，在一些实现方式中，第二电子设备根据RoI，确定与RoI有交集的子矩阵以及与RoI有交集的子矩阵占用的存储空间大小，第二电子设备根据与RoI有交集的子矩阵的存储空间大小，确定存储与RoI有交集的子矩阵的数据的存储位置，第二电子设备根据存储位置，从第一电子设备中读取与RoI有交集的子矩阵中有事件的像素点的坐标以及事件极性，进而得到RoI内有事件的像素点的坐标和事件极性。
在上述方案中，当依次存储前m个子矩阵占用的存储空间大小时，通过读取第m个数和第m+1个数就能知道第m个子矩阵有事件的像素点的坐标以及事件极性的存储位置。当依次存储第m个子矩阵占用的存储空间大小时，通过将前m个数相加，和将第m+1个数相加，就能知道第m个子矩阵有事件的像素点的坐标以及事件极性的存储位置。进而，第二电子设备不用遍历事件数据，依次判断每个事件数据的坐标是否在RoI内。提高了读取RoI内像素点的事件极性的效率，提高了读取RoI内像素点的事件极性的效率。
结合第一方面，在一些实现方式中，预设存储方式包括：第一电子设备存储事件帧矩阵。
在上述方案中，可以直接存储完整的事件帧矩阵，事件帧矩阵是按像素点坐标和事件极性进行存储 的，使得在读取事件帧矩阵中的部分数据时，能直接通过事件帧矩阵中矩阵元素的坐标读取到需要读取的像素低的数据。
结合第一方面，在一些实现方式中，第二电子设备根据RoI，确定事件帧矩阵中RoI对应的矩阵元素坐标范围，第二电子设备根据矩阵元素坐标范围，从第一电子设备中读取RoI内每个像素点是否有事件以及有事件的像素点的事件极性。
在上述方案中，因为每个像素点的数据是根据像素点的坐标进行存储的，因此，在读取RoI范围内的数据时，能直接根据RoI对应的矩阵元素坐标范围读取出需要读取的像素点的事件极性。
结合第一方面，在一些实现方式中，第二电子设备根据RoI内有事件的像素点坐标和事件极性，将没有事件的像素点标记为亮度不变，得到RoI内每个像素点是否有事件以及有事件的像素点的事件极性。
在上述方案中，当只存储了有事件的像素点的坐标和事件极性时，在输出读取结果前，还可以将读取范围内没有事件的像素点标记为亮度不变，并一起输出。
第二方面，本申请提供了一种电子设备，包括处理器和存储器，存储器用于存储指令，处理器用于执行指令，当处理器执行该指令时，执行如第一方面所描述的方法。
第三方面，本申请提供一种计算机可读存储介质，计算机可读存储介质中存储有指令，当指令在电子设备上运行时，执行如第一方面所描述的方法。
第四方面，本申请提供了一种计算机程序产品，计算机程序产品包括计算机指令，在被计算设备执行时，计算设备执行如第一方面所描述的方法。
综上所述，本申请提供的事件数据处理方法及相关设备，通过获取预设时间段内的事件数据，并将每个像素点在预设时间段内的亮度变化进行统计，得到每个像素点的总体亮度变化结果，然后根据每个像素点的位置，将像素点对应的总体亮度变化结果保存为事件帧矩阵。这样，减少了数据量，进而减少了占用的存储空间。然后，根据事件帧矩阵，确定存储事件帧矩阵占用空间最小的存储方式，也能减少存储空间的占用。并且，对于事件帧矩阵的存储是根据事件帧矩阵中矩阵元素的坐标进行存储的，因此，在读取事件帧矩阵中的部分数据时，不需要遍历所有存储的数据，能直接根据需要读取的范围确定矩阵元素坐标和事件极性的存储位置，提高了读取事件数据的效率。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍。
图1A是本申请实施例提供的一种监控画面的示意图；
图1B是本申请实施例提供的一种事件相机生成的灰度图的示意图；
图2是本申请实施例提供的一种执行事件数据处理方法的系统结构示意图；
图3是本申请实施例提供的一种事件数据处理方法的流程示意图；
图4是本申请实施例提供的另一种事件数据处理方法的流程示意图；
图5是本申请实施例提供的一种事件数据处理系统的结构示意图；
图6是本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
以下将参考附图详细说明本申请的具体实施方式。
为便于理解，首先介绍本申请中涉及的事件相机应用技术。
事件相机(event camera)通过传感器检测拍摄范围内的光强度变化，然后将发生亮度变化的像素点坐标、时间戳和亮度变化的正负极性输出为事件数据。具体地，在连续时刻内持续通过感光器件获取所有像素点的亮度值变化，当某个像素所处位置的亮度值发生变化，且亮度变化达到一定阈值后时，相机就会输出一个事件数据。
其中，像素亮度的阈值为事件相机的预设值，事件数据可以用要素：x，y，t，p进行表示。其中，x和y用于事件发生的位置即像素点坐标、t为事件发生的时刻戳，P用于表示像素点亮度变化情况，包括亮度变高、亮度变低。总之，(x，y，p，t)可以表示“在什么时刻，哪个像素点，发生了亮度的增加或减小”。
当场景中有物体运动或光照改变造成大量像素亮度变化时，则会产生一系列的事件，这些事件以事 件流(Events stream)方式输出。举例来说，当有N个事件时，事件流可以用进行表示，k、N为正整数。当事件流积累一定数量之后，可以生成事件帧，事件帧能反应每个像素点在该时间段内亮度的整体变化情况。
事件相机主要有两种：动态视觉传感器(Dynamic Vision Sensor，DVS)相机以及动态有源像素视觉传感器(Dynamic and Active Pixel Vision Sensor，DAVIS)相机。DVS相机是普通的事件相机，仅传输事件数据；DAVIS相机是在回传事件数据的同时还可以回传灰度图或者彩色图。事件相机能以高帧率拍摄事件，其输出的事件数据的时间分辨率为微秒级。因此，事件相机具有时间延迟低、动态范围大(即明暗差别大)的特点，能应用于自动驾驶、无人机视觉导航、工业检测及视频监控等机器视觉领域。图1A示出了事件相机应用于视频监控时监控画面的场景图，当该场景中有光照改变造成大量像素亮度变化时，则会产生一系列的事件数据，在一定时间后，最终生成事件帧。如图1B所示为根据事件帧确定的灰度图，其中，深色点表示该像素点在该时间段内亮度变亮，浅色点表示该像素点在该时间段内亮度变暗。
但是，目前由于事件相机获取事件数据的时间分辨率过高，通常为微秒级别，得到的事件数据量过大，会占用较大存储空间。并且，事件数据是按事件发生的时间，以单个像素点的事件进行存储的，如果只需要读取某一坐标范围内的像素点的事件，需要遍历全部事件数据，增加了对事件数据的处理难度，降低了事件数据处理效率。
为了解决事件数据占用存储空间较多以及读取事件数据效率较低的问题，本申请提供了一种事件数据处理方法，该事件数据处理方法可以应用于如图2所示的系统架构中，该架构包括事件相机(event camera)110、存储设备120以及读取设备130。其中，存储设备120也可成为第一电子设备，读取设备130也可称为第二电子设备。
事件相机110用于通过传感器检测拍摄范围内的光强度变化，然后将发生亮度变化的像素点坐标、时间戳和亮度变化情况输出为事件数据。具体可参考上面对事件相机的详细描述，此处不再赘述。
存储设备120包括存储器和处理器，处理器用于从事件相机110处获取事件数据，并将预设时间段内像素点的事件数据记为该像素点的事件极性，然后生成该预设时间段内的事件帧矩阵，事件帧矩阵中包括每个像素点的事件极性和像素点坐标。处理器还用于从多个预设存储方式中选择存储该事件帧矩阵占用空间最小的存储方式，然后将该事件帧矩阵发送给存储器，存储器将该事件帧矩阵按处理器选择的存储方式进行存储。存储设备可以是物理服务器或云端设备，比如X86服务器、ARM服务器等等，也可以是终端，比如手机、平板电脑、可穿戴设备、VR设备、车载设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等，本申请对该存储设备120的具体类型不作任何限制。
读取设备130用于从存储设备120中读取事件帧矩阵。在一些实施例中，读取设备130还用于获取用户对事件相机拍摄的画面的感兴趣区域(Region of Interest，RoI)，或者获取应用程序或者其他电子设备在对事件相机拍摄的画面进行图像处理后得到的RoI，读取设备130能根据RoI和事件帧矩阵的存储方式，从存储设备120中读取RoI范围内的像素点的事件极性。读取设备130读取出来数据后，读取设备130或其他设备能进行进一步图像处理，或完成其他操作。读取设备130可以是手机、平板电脑、可穿戴设备、VR设备、车载设备、笔记本电脑、UMPC、上网本、PDA等，本申请对该读取设备130的具体类型不作任何限制。
应理解，上述系统结构还可以包括其他设备，或者可以根据需要对上述应用场景中的各个设备进行合并或拆分。例如，在一些实施例中，存储设备120可以是事件相机中的存储器和处理器，事件相机生成事件数据后，将直接由处理器生成事件帧矩阵和确定存储方式，然后将事件帧矩阵存储到存储器中。或者，存储设备120是读取设备130的存储器，事件相机将事件数据发送给读取设备130后，读取设备130的处理器生成事件帧矩阵和确定存储方式，然后将事件帧矩阵存储到存储器中。当读取设备130需要使用事件数据时，将从存储器中读取事件帧矩阵。
下面结合图3对本申请提供的事件数据处理方法进行详细描述。
为了解决事件数据占用存储空间较多以及读取事件数据效率较低的问题，本申请提供了一种事件数据处理方法，该方法通过存储设备120从事件相机110中获取预设时间段内的事件数据，将每个像素点 在预设时间段内亮度变化的整体情况记为事件极性，事件极性包括亮度变高、亮度变低，如果像素点亮度不变即为没有事件，进而得到事件帧矩阵。事件帧矩阵中每个矩阵元素的位置对应像素点坐标，矩阵元素的值对应像素点是否有事件以及具体的事件极性。如此，通过一个像素点坐标的事件极性，即可表征预设时间段内该像素点的事件数据，而无需逐个记录该像素点坐标在预设时间段内的所有事件数据，减小了记录的数据的数量。比如，某像素点在预设时间段内亮度先变高，再变低，事件帧矩阵中只用存储该像素点亮度不变这个结果即可，不用一一存储该预设时间段内每一次事件数据，进而减少了需要存储的数据量。
并且，在存储事件帧矩阵时，存储设备120可以从预设的多种存储方式中，选择占用空间最小的一种存储方式来存储事件帧矩阵，可以进一步降低事件数据占用的存储空间。其中，事件帧矩阵的存储方法包括：存储事件帧矩阵中有事件的矩阵元素的坐标和事件极性；或者，存储事件帧矩阵中每行以及该行前面所有行有事件的矩阵元素累计个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性；或者，将事件帧矩阵分为多个子矩阵，先判断存储子矩阵的存储方式和存储子矩阵所需要占用的存储空间大小，然后存储每个子矩阵在存储空间中存储的位置的坐标，以及根据确定后的存储方式存储每个子矩阵中有事件的矩阵元素以及对应的事件极性；或者，直接存储事件帧矩阵。各存储方法的具体存储方式将在下文进行介绍，在此不做赘述。
在一些实施例中，本申请提供的事件数据处理方法，读取设备130还能获取RoI，并能根据RoI以及事件帧矩阵的存储方法，从存储设备120中读取事件帧矩阵中RoI内像素点的是否有事件以及具体的事件极性。也即是说，读取设备130能直接根据RoI从事件帧矩阵中确定要读取的像素点范围，然后读取出RoI内像素点是否有事件以及具体的事件极性。这样，读取设备130不用遍历事件数据，依次判断每个事件数据的坐标是否在RoI内。进而，提高了读取RoI内像素点的事件极性的效率。
下面结合图3对本申请提供的事件数据处理方法进行详细描述。如图3所示，该方法包括如下步骤：
S310：事件相机110生成多个事件数据。
事件相机110通过传感器检测拍摄范围内的光强度变化，然后将有亮度变化的像素点坐标、时间戳和亮度变化情况输出为事件数据。
事件数据包括像素点的位置、时间戳以及像素点亮度变化情况。其中，单个事件数据可以用(x，y，p，t)进行表示，x，y为事件发生的位置，即像素点坐标，P表示像素点亮度变化情况，t为事件发生的时刻戳。假设预设时间T内，共有N个事件数据，则该N个事件数据可以用于表示。
举例来说，P可以用“0、1”来表示，0表示像素点亮度变低，1表示像素点亮度变高；或者，P还可以用“-1、1”来表示，-1表示像素点亮度变低，1表示像素点亮度变高。应理解，本申请对像素点亮度变化情况的表示方法不做具体限定。
S320：存储设备120从事件相机110中获取预设时间段内的多个事件数据。
事件相机110在持续性地生成事件数据，并将事件数据发送给存储设备120进行存储。存储设备120是将预设时间段内的多个事件数据存储在一起的，因此，存储设备120可以是先获取预设时间段内的事件数据，将该预设时间段内的事件数据存储后，再获取下一个预设时间段的事件数据。存储设备120还可以是把所有事件数据按预设时间段分为多个事件数据组，依次对每个事件数据组进行存储。应理解，本申请对存储设备120获取事件数据的方式不做具体限定。
S330：存储设备120根据预设时间段内的多个事件数据，生成事件帧矩阵。
存储设备120根据每个像素点在该预设时间段内的总体亮度变化情况，确定每个像素点的是否有事件，以及如果有事件，确定具体的事件极性，进而得到事件帧矩阵。事件极性包括亮度变高、亮度变低，如果像素点亮度不变即为没有事件。事件帧矩阵中的每个矩阵元素的位置表示像素点坐标，矩阵元素的值表示对应像素点是否有事件以及具体的事件极性。其中，预设时间段可以根据应用场景的需求进行设定，具体可以是在1至100毫秒之间。对于精度要求较高的应用场景，预设时间可以设置得较短，对于精度要求较低的应用场景，预设时间可以设置得较长。进而，使得预设时间段能满足应用场景的精度需求。
具体地，将事件数据中像素点亮度变高称为正事件，像素点亮度变低称为负事件，其中，事件数据 是在像素点亮度变化超过阈值后就会生成，具体是将亮度值取对数后进行比较得到的，可以理解为每个正事件对应的亮度变高的量是相同的，每个负事件对应的亮度变低的量是相同的。因此，当每个像素点在预设时间T内正事件数量多于负事件数量时，像素点在预设时间T内的事件极性为亮度变高；当每个像素点在预设时间T内正事件数量少于负事件数量时，像素点在预设时间T内的事件极性为亮度变低；当每个像素点在预设时间T内正事件数量等于负事件数量时，包括没有正事件和负事件的情况，即正事件数量和负事件数量都为零，像素点在预设时间T内没有事件。最终，根据每个像素点是否有事件以及具体的事件极性和像素点位置，就能确定事件帧矩阵。
举例来说，当P用-1来表示像素点亮度变低，P等于1表示像素点亮度变高，矩阵元素的值Q用-1来表示像素点亮度变低，Q等于0表示像素点亮度不变，Q等于1表示像素点亮度变高时，事件帧矩阵中矩阵元素的值Q是根据该像素点在预设时间T内每个事件数据中的P值的和确定的。当P值的和小于0，矩阵元素的值Q为-1；当P值的和等于0，矩阵元素的值Q为0；当P值的和大于0，矩阵元素的值Q为1。则事件帧矩阵中每个矩阵元素的值可以用符号函数计算得到，如公式(1)所示，假设事件相机获取的图像像素为W×H，也即是说，事件帧矩阵F共有H行W列，H和W为正整数。公式(1)示出了将第i行第j列的矩阵元素F[i，j]的计算方式，0≤i＜H，0≤j＜W，且i和j为正整数。
综上，存储设备120将每个像素点在一定时间段内的亮度变化进行统计，得到每个像素点的总体亮度变化结果，然后根据每个像素点的位置，将像素点对应的总体亮度变化结果保存为事件帧矩阵，减少了数据量，进而减少了占用的存储空间。
S340：存储设备120根据像素点数量，确定事件帧矩阵存储方式，并对事件帧矩阵进行存储。
存储设备120根据像素点数量，即事件帧矩阵中矩阵元素的数量，确定存储事件帧矩阵的方式。本方法提供了多种预设事件帧矩阵的存储方法，存储设备120可以先计算出采用各个存储方式所占用的存储空间，再选择占用存储空间最小的方法对事件帧矩阵进行存储。
其中，事件帧矩阵的存储方法包括：存储事件帧矩阵中有事件的矩阵元素的坐标和对应的事件极性；或者，存储事件帧矩阵中每行以及该行前面所有行有事件的矩阵元素累计个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性；或者，将事件帧矩阵分为多个子矩阵，存储每个子矩阵起始矩阵元素坐标以及每个子矩阵中有事件的矩阵元素以及对应的事件极性；或者，直接存储事件帧矩阵。
下面对每种事件帧矩阵的存储方法以及每种存储方法占用的空间进行详细说明。
假设事件帧矩阵F共有H行W列，事件帧矩阵F中有事件的矩阵元素共有E个，E为正整数。将W的位宽记为BW，H的位宽记为BH，事件总量E的位宽记为BE，将有事件的矩阵元素的事件极性记为Q′，Q′位宽记为BQ′。其中，位宽为记录该正整数的最小比特数，可以通过该正整数取2的对数得到。应理解，事件极性Q′共有两种可能，包括亮度变低和亮度变高，则通常Q′用一个比特进行记录，即BQ′等于1。
存储方式为存储事件帧矩阵中有事件的矩阵元素的坐标和事件极性时，即存储有事件的矩阵元素的横坐标、纵坐标和事件极性，对于每个像素点需要β个比特的数据进行记录，β等于BW、BH、BQ′的和。有事件的矩阵元素共有E个，则该存储方法占用的空间S1为E和β的乘积。示例性地，β的计算方式可以参考下述公式(2)，S1的计算方式可以参考下述公式(3)。
β＝BW+BH+BQ′＝BW+BH+1      (2)
S1＝E*β      (3)
存储方式为存储事件帧矩阵中每行以及该行前面所有行有事件的矩阵元素累计个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性。具体包括：先用H+1个数记录前i行有事件的矩阵元素的个数，其中，第i+1个数记录的是矩阵前i行有事件的矩阵元素的个数(0≤i＜H)，H+1个数中的每个数需要BE个比特。再用E个数记录有事件的矩阵元素的列坐标和事件极性，E个数中每个数需要BW+BQ′个比特。则该存储方法占用的空间为S2的计算方式可以参考下述公式(4)。
S2＝(H+1)*BE+E*(BW+BQ′)      (4)
在一些实施例中，该存储方式还可以是存储事件帧矩阵中每行中有事件的矩阵元素个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性。具体包括：先用H个数记录第i行的有事件的矩阵元素的个数，其中，第i+1个数记录的是矩阵第i行中有事件的矩阵元素的个数(0≤i＜H)，H个数中的每个数需要BW个比特。再用E个数记录有事件的矩阵元素的列坐标和事件极性，E个数中每个数需要BW+BQ′个比特。则该存储方法占用的空间为S′2的计算方式可以参考下述公式(5)。
S′2＝H×BW+E×(BW+BQ′)      (5)
存储方式为将事件帧矩阵分为多个子矩阵，先判断存储子矩阵的存储方式和存储子矩阵所需要占用的存储空间大小，然后存储每个子矩阵在存储空间中存储的位置的坐标，以及根据确定后的存储方式存储每个子矩阵中有事件的矩阵元素以及对应的事件极性。
具体包括：假设子矩阵的大小为h′行w′列，进而事件帧矩阵F可以被分为nh×nw个子矩阵，每个子矩阵中有事件的矩阵元素个数记为e′，其中，nh等于H/h′，nw等于W/w′，w′的位宽为Bw′，h′的位宽为Bh′。
对于子矩阵，可以用h′×w′个比特记录子矩阵中每一个矩阵元素是否有事件，再用e′个比特记录有事件的矩阵元素的事件极性，共需要w′×h′+e′个比特。或者，对于子矩阵，使用α个比特记录子矩阵中有事件的矩阵元素的坐标及事件极性，其中，α等于Bw′、Bh′、BQ′的和。可以根据每个子矩阵中有事件的矩阵元素个数e′确定存储子矩阵的方式，两种子矩阵的存储方式的占用空间相等时，e′的值可参考公式(6)中e0的结果。
e0＝w′h′/Bw′+Bh′+BQ′-1      (6)
当子矩阵中有事件的矩阵元素个数e′大于e0时，采用h′×w′个比特记录子矩阵中每一个矩阵元素是否有事件，再用e′个比特记录有事件的矩阵元素的事件极性的存储方式占用的存储空更小。当有事件的矩阵元素个数e′小于或等于e0时，采用α个比特记录其在子矩阵中有事件的矩阵元素的坐标及事件极性的存储方式占用的存储空更小。
当所有的子矩阵的存储方式都确定了之后，可以确定所有子矩阵占用的存储空间大小，先用nh′×nw′+1个数记录每个子矩阵在存储空间中起始位置，则第m个子矩阵在存储空间的起始、结束位置可通过读取第m个数和第m+1个数得到。再分别对每个子矩阵中的矩阵元素根据上述确定后的子矩阵存储方式进行存储。其中，将总占用空间大小记为D，D的位宽记为BD。则该存储方法占用的空间为S3的计算方式可以参考下述公式(7)。
在一些实施例中，当所有的子矩阵的存储方式都确定了之后，可以先用nh′×nw′个数记录每个子矩阵占用的存储空间大小。再分别对每个子矩阵中的矩阵元素根据上述确定后的子矩阵存储方式进行存储。其中，每个子矩阵占用空间大小的位宽为B2*h′*w′。则该存储方法占用的空间为S′3的计算方式可以参考下述公式(8)。
存储方式为直接存储事件帧矩阵F时，该存储方法占用的空间为S4的计算方式可以参考下述公式(9)。
S4＝H×W×2      (9)
在一些实施例中，存储设备120在存储事件帧矩阵后，在需要对事件帧矩阵进行处理时，还会读取事件帧矩阵，或者读取事件帧矩阵中的部分内容，例如，读取感兴趣区域RoI内的矩阵元素的事件极性。下面对读取RoI内的矩阵元素的事件极性进行详细描述。应理解，读取事件帧矩阵的完整数据可以参考下述读取RoI内的矩阵元素的事件极性，此处不再对读取事件帧矩阵的完整数据进行赘述。
S350：读取设备130获取RoI，并从存储设备120中读取RoI中像素点的事件。
读取设备130获取RoI，RoI可以是事件相机拍摄的画面中的任意区域，例如用户通过读取设备130查看事件相机拍摄的画面时选中的区域，读取设备130中的应用程序(例如图像查看程序、图像处理程序等)生成的区域。读取设备130根据RoI，确定RoI在事件帧矩阵中的矩阵元素范围，根据RoI和存储事件帧矩阵的方式，读取RoI内的每个像素点的事件极性。
下面对每种事件帧矩阵存储方式对应的读取RoI内的每个像素点的事件极性的方法，进行详细描述。
存储方式为存储事件帧矩阵中有事件的矩阵元素对应的像素点坐标和事件极性时，读取设备130根据RoI可以判断需要获取的矩阵元素的横坐标、纵坐标范围，对于在RoI内的矩阵元素，读取设备130读取该矩阵元素的事件极性，并输出读取后的结果。
应理解，该存储方式只能读取有事件的矩阵元素的事件极性为亮度变低还是亮度变高，对于RoI内没有读取到事件极性的矩阵元素点，其矩阵元素对应的像素点为亮度不变。在一些实施例中，还会将没有读取到事件极性的矩阵元素点标记为亮度不变，进而，将RoI内所有矩阵元素点的事件数据一起输出。
存储方式为存储事件帧矩阵中每行以及该行前面所有行有事件的矩阵元素累计个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性，读取设备130是以行为单位进行读取的。具体包括：首先，确定与RoI有交集的行，对于第i行，直接读取第i个数和第i+1个数，就能得到该行的事件个数以及列坐标、事件极性存储的位置，将该行数据中每个矩阵元素的列坐标、事件极性读取出来，然后再确定与RoI有交集的列，进而确定需要输出的矩阵元素以及对应的列坐标。
当该存储方式是存储事件帧矩阵中每行中有事件的矩阵元素个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性时，对于第i行，读取设备130需要读取前i+1个数，就能得到该行的事件个数以及列坐标、事件极性存储的位置，将该行数据中每个矩阵元素的列坐标、事件极性读取出来，然后再确定与RoI有交集的列，进而确定需要输出的矩阵元素以及对应的列坐标。
同样地，该存储方式读取的矩阵元素的也只包括有事件的矩阵元素，在一些实施例中，还会将没有读取到事件极性的矩阵元素点标记为亮度不变，进而，将RoI内所有矩阵元素点的事件数据一起输出。
存储方式为将事件帧矩阵分为多个子矩阵，先判断存储子矩阵的存储方式和存储子矩阵所需要占用的存储空间大小，然后存储每个子矩阵在存储空间中存储的位置的坐标，以及根据确定后的存储方式存储时，每个子矩阵中有事件的矩阵元素以及对应的事件极性时，读取设备130是以子矩阵为单位进行读取的。具体包括：先确定与RoI有交集的子矩阵，对于第m个矩阵，直接读取第m个数和第m+1个数，就能得到该子矩阵的起始位置和结束位置。根据每个有交集的子矩阵的存储方式，读取有交集的子矩阵的矩阵元素的坐标和事件极性。
读取设备130可以根据存储子矩阵占用的空间大小判断得到子矩阵采用的存储方式，两种子矩阵的占用空间的临界点为S0，当子矩阵所占用的存储空间大于S0时，采用的是h′×w′个比特记录子矩阵中每一个矩阵元素的值是否有事件，再用e′个比特记录有事件的矩阵元素的事件极性。当子矩阵所占用的存储空间小于S0时，采用的是α个比特记录子矩阵中有事件的矩阵元素的坐标及事件极性的存储方式。示例性地，S0的计算方式可以参考下述公式(10)。
S0＝(w′h′/Bw′+Bh′+BQ′-1)(Bw′+Bh′+1)      (10)
其中，当用h′×w′个比特记录子矩阵中每一个矩阵元素的值是否有事件，再用e'个比特记录有事件的矩阵元素的事件极性时，先根据前h′×w′个比特确定有事件的矩阵元素，再根据后e′个比特确定具体的事件极性为亮度变高还是亮度变低，对于h′×w′个比特中亮度不变的矩阵元素，则直接输出为该矩阵元素的对应的像素点亮度不变。进而，可以得到RoI有交集的子矩阵中每个矩阵元素点的事件极性。
当用α个比特记录子矩阵中有事件的矩阵元素的坐标及事件极性时，直接读取每个子矩阵的数据就 能得到子矩阵中有事件的矩阵元素的坐标和事件极性。对于没有记录的矩阵元素，其亮度不变。进而，可以得到RoI有交集的子矩阵中每个矩阵元素点的事件极性。
其中，如果是用nh′×nw′个数记录每个子矩阵占用的存储空间大小，则对于第m个矩阵，需要读取前m+1个数，才能得到该子矩阵的起始位置和结束位置。然后根据每个有交集的子矩阵的存储方式，读取有交集的子矩阵的矩阵元素的坐标和事件极性。
存储方式为直接存储事件帧矩阵F时，可直接读取RoI区域对应的所有像素的数据，进而确定并输出每个矩阵元素点的事件极性。
综上所述，本申请提供的事件数据处理方法，通过存储设备120获取预设时间段内的事件数据，并将每个像素点在预设时间段内的亮度变化进行统计，得到每个像素点的总体亮度变化结果，然后根据每个像素点的位置，将像素点对应的总体亮度变化结果保存为事件帧矩阵。这样，减少了数据量，进而减少了占用的存储空间。然后，根据事件帧矩阵，确定存储事件帧矩阵占用空间最小的存储方式，也能减少存储空间的占用。并且，对于事件帧矩阵的存储是根据事件帧矩阵中矩阵元素的坐标进行存储的，因此，读取设备130在读取事件帧矩阵中的部分数据时，不需要遍历所有存储的数据，能直接根据需要读取的范围确定矩阵元素坐标和事件极性的存储位置，提高了读取事件数据的效率。
为了便于理解本申请提供的事件数据处理方法，下面结合图4对该事件数据处理方法进行举例说明。
S410：存储设备120获取事件数据流。
存储设备120获取事件相机输出的事件数据流，事件数据流中每个事件数据包括像素点的位置、时间戳以及像素点亮度变化情况，事件数据可以用(x，y，p，t)进行表示，具体可以参考上述步骤S310：事件相机110生成多个事件数据和步骤S320：存储设备120从事件相机110中获取预设时间段内的多个事件数据。如表1所示，表1为获取到的事件数据流，共包括5个事件数据，事件相机的分辨率为H＝2，W＝2，时间戳t的格式为年份-日期-时分秒-微秒，P用“-1，1”来表示，-1表示像素点亮度变低，1表示像素点亮度变高。
表1
S420：存储设备120将事件数据流压帧，生成事件帧矩阵。
存储设备120将事件数据流压帧的预设时间段为5毫秒，从第一个事件开始的5毫秒之内的数据进行压帧(即时间窗口参数T＝5ms)。事件1的时间戳的微秒数为262600μs，即262.6ms，则时间窗口包括了262.6ms至267.6ms内的所有事件，即落在时间窗内的事件为事件序号为1、2、3、4的四个事件。经过压帧之后可以得到如表2所示的2×2的事件帧。
其中，事件1和事件3都是坐标为(1，1)的像素点的事件，将事件1和事件3的亮度变化情况相加，则可以得到在预设时间段5毫秒内，坐标为(1，1)的像素点的事件极性为0，即亮度不变。
表2
如图表2所示的事件帧，还可以表示为事件帧矩阵F'，F′如下公式(11)所示。
S430：存储设备120确定事件帧矩阵的存储方式，并将事件帧矩阵进行存储。
事件帧矩阵的存储方式具体可以参考上述步骤S340：存储设备120根据像素点数量，确定事件帧矩阵存储方式，并对事件帧矩阵进行存储。下面对事件帧矩阵F′采用每种存储方式所占用的存储空间进 行详细说明。
存储设备120存储事件帧矩阵F′中有事件的矩阵元素的坐标和事件极性时，因为事件帧矩阵F′共有2行2列，有事件的矩阵元素也只有两个，即H＝2，W＝2，Q'＝2，则BW＝1，BH＝1，BQ′＝1。进而，根据公式(2)可以确定，每个像素点需要3个比特进行存储。事件帧矩阵F′中有事件的矩阵元素有两个，即(2，1)和(1，2)的矩阵元素，则E＝2。进而，根据公式(3)可以确定，存储设备120存储事件帧矩阵F′需要6个比特。
存储设备120存储事件帧矩阵中每行以及该行前面所有行有事件的矩阵元素累计个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性时，因为事件帧矩阵F′共有2行，则需要先用3个数记录每行累计的亮度改变的矩阵元素的个数，每个数需要2个比特。事件帧矩阵F′中有事件的矩阵元素有两个，需要用2个数记录亮度改变的矩阵元素的列坐标和事件极性，每个数需要2个比特。则根据公式(4)，可以确定存储设备120存储事件帧矩阵F′需要10个比特。
存储设备120将事件帧矩阵分为多个子矩阵，存储每个子矩阵起始矩阵元素坐标以及每个子矩阵中有事件的矩阵元素以及对应的事件极性时，子矩阵大小可以为2×2，即只有一个子矩阵，需要用2个数来存储该数据块的起始矩阵元素位置。
然后，因为子矩阵大小为子矩阵大小可以为2×2，则可以用4个比特记录子矩阵中每一个矩阵元素的值是否有事件，又因为共有两个矩阵元素有事件，则再用2个比特记录有事件的矩阵元素的事件极性，该子矩阵共需要6个比特。或者，对于子矩阵中的每个矩阵元素，存储子矩阵中有事件的矩阵元素的坐标及事件极性，共有两个矩阵元素，每个矩阵元素需要1个比特记录横坐标，1个比特记录纵坐标，1个比特事件极性，则也是需要6个比特。
因此，对于该存储方式，存储设备120存储子矩阵需要6个比特，则需要3个比特记录子矩阵存储空间大小，根据公式(7)，可以确定存储设备120存储事件帧矩阵F′需要12个比特。
存储设备120直接存储事件帧矩阵F′，因为事件帧矩阵F′共有2行2列，则根据公式(9)，存储设备120存储事件帧矩阵F′需要8个比特。
综上，存储设备120采用第一种存储方式存储事件帧矩阵F′，即电子设备存储事件帧矩阵F′中有事件的矩阵元素的坐标和事件极性，占用的存储空间更小。
S440：读取设备130获取RoI，读取RoI中像素点的事件。
读取设备130获取到RoI为坐标为(1，1)，因为存储设备130采用第一种存储方式存储事件帧矩阵F′，存储的是亮度改变的矩阵元素的坐标和事件极性，即存储的是(2，1，-1)和(1，2，+1)，因为坐标(1，1)不在存储的事件列表中，则读取设备130可以确定坐标(1，1)处的像素点为亮度不变。进而，可以输出RoI中像素点的事件为(1，1，0)。
综上所述，本申请提供的事件数据处理方法，将每个像素点在预设时间段内的亮度变化进行统计，得到每个像素点的总体亮度变化结果，并生成为事件帧矩阵。这样，减少了数据量，进而减少了占用的存储空间。然后，根据事件帧矩阵，确定存储事件帧矩阵占用空间最小的存储方式，也能减少存储空间的占用。并且，对于事件帧矩阵的存储是根据事件帧矩阵中矩阵元素的坐标进行存储的，因此，在读取事件帧矩阵中的部分数据时，不需要遍历所有存储的数据，能直接根据需要读取的范围确定矩阵元素坐标和事件极性的存储位置，提高了读取事件数据的效率。
下面再结合另一个示例对本申请提供的事件数据处理方法中的各种存储方式进行举例说明。
其中，事件帧矩阵为1280*720(宽度W＝1280，高度H＝720)。事件帧矩阵被划分为大小为16*8的子矩阵(块的宽度w′＝16，高度h′＝8)，一共可以得到1280/16*720/8＝7200个子矩阵。假设每一个子矩阵的第一个矩阵元素有一个事件。此时，BW＝11，BH＝10，Bw′＝4，Bh′＝3，nh′＝90，nw′＝80，E＝7200，BE＝13。
存储设备120存储事件帧矩阵中有事件的矩阵元素的坐标和事件极性时，每个有事件的矩阵元素需要11个比特记录横坐标，10个比特记录纵坐标，1个比特记录事件极性。因为共有7200个有事件的矩阵元素。进而，该存储方式共需要7200*22＝158400个比特。
存储设备120根据事件帧矩阵中每行有事件的矩阵元素的累计个数，以及事件帧矩阵中有事件的矩阵元素的列坐标和对应的事件极性进行存储时，需要用721个数记录每行的累计有事件的矩阵元素个数，每个数需要13个比特，然后用7200个数记录有事件的矩阵元素的列坐标和事件极性，列坐标需要10 个比特，事件极性需要1个比特。进而，该存储方式则共需要721*13+7200*11＝88573个比特。
存储设备120将事件帧矩阵分为多个子矩阵，存储每个子矩阵起始矩阵元素坐标以及每个子矩阵中有事件的矩阵元素以及对应的事件极性时，每个子矩阵采用h′×w′＝128个比特记录子矩阵中每一个矩阵元素的值是否对应亮度变化，再用1个比特记录有事件的矩阵元素的事件极性，共需要129个比特。而对于子矩阵中的每个矩阵元素，记录其在子矩阵中有事件的矩阵元素的坐标及事件极性，只需要用Bw′+Bh′+BQ′＝8个比特。因此，采用第二种子矩阵存储方式占用空间更少。
共有7200个子矩阵，每个子矩阵需要8个比特，则共需要57600个比特，则需要16个比特存储占用空间大小。该存储方式则共需要(90*80+1)*16+7200*8＝172816个比特。
存储设备120直接存储事件帧矩阵，则需要1280*720*2＝1843200个比特。
因此，选用第二种存储方式存储事件帧矩阵占用的空间更小。
综上所述，本申请提供的事件数据处理方法，能根据事件帧矩阵，确定存储事件帧矩阵占用空间最小的存储方式，也能减少存储空间的占用。
为了解决事件数据占用存储空间较多以及读取事件数据效率较低的问题，本申请提供了一种事件数据处理系统1000，如图5所示，事件数据处理系统1000包括事件相机输入模块1010、事件数据压帧模块1020、事件帧压缩模块1030以及事件数据读取模块1040。
事件相机输入模块1010用于获取事件相机输出的事件数据，事件数据包括像素点的位置、时间戳以及像素点亮度变化情况。具体用于执行步骤S310和步骤S320所描述的方法，此处不再赘述。
事件数据压帧模块1020用于根据每个像素点在预设时间段内的总体亮度变化情况，确定每个像素点的事件极性，得到事件帧矩阵。具体用于执行步骤S330所描述的方法，此处不再赘述。
事件帧压缩模块1030用于根据像素点数量，即事件帧矩阵中矩阵元素的数量，确定存储事件帧矩阵的方式。具体用于执行步骤S340所描述的方法，此处不再赘述。
事件数据读取模块1040用于获取RoI，确定RoI对应在事件帧矩阵中的矩阵元素范围，根据RoI和存储事件帧矩阵的方式，读取RoI内的每个像素点的事件极性。具体用于执行步骤S350所描述的方法，此处不再赘述。
综上所述，本申请提供的事件数据处理系统，将每个像素点在预设时间段内的亮度变化进行统计，得到每个像素点的总体亮度变化结果，并生成为事件帧矩阵。这样，减少了数据量，进而减少了占用的存储空间。然后，根据事件帧矩阵，确定存储事件帧矩阵占用空间最小的存储方式，也能减少存储空间的占用。并且，对于事件帧矩阵的存储是根据事件帧矩阵中矩阵元素的坐标进行存储的，因此，在读取事件帧矩阵中的部分数据时，不需要遍历所有存储的数据，能直接根据需要读取的范围确定矩阵元素坐标和事件极性的存储位置，提高了读取事件数据的效率。
上述详细阐述了本申请实施例的方法，为了便于更好的实施本申请实施例上述方案，相应地，下面还提供用于配合实施上述方案的相关设备。
图6是本申请提供的一种电子设备1200的结构示意图，该电子设备1200可以是前述内容中的事件相机110、存储设备120和读取设备130，或者上述设备的组合。如图所示，电子设备1200包括：处理器1210、通信接口1220、存储器1230以及事件相机1250。其中，处理器1210、通信接口1220、存储器1230以及事件相机1250可以通过内部总线1240相互连接，也可通过无线传输等其他手段实现通信。
本申请实施例以通过总线1240连接为例，总线1240可以是快捷外围部件互连标准(Peripheral Component Interconnect Express，PCIe)总线，或扩展工业标准结构(extended industry standard architecture，EISA)总线、统一总线(unified bus，Ubus或UB)、计算机快速链接(compute express link，CXL)、缓存一致互联协议(cache coherent interconnect for accelerators，CCIX)等。总线1240可以分为地址总线、数据总线、控制总线等。总线1240除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线1240。
处理器1210可以包括至少一个通用处理器，例如中央处理器(Central Processing Unit，CPU)，或者CPU和硬件芯片的组合。上述硬件芯片可以是专用集成电路(Application-Specific Integrated Circuit，ASIC)、可编程逻辑器件(Programmable Logic Device，PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(Complex Programmable Logic Device，CPLD)、现场可编程门阵列 (Field-Programmable Gate Array，FPGA)、通用阵列逻辑(Generic Array Logic，GAL)或其任意组合。处理器1210执行各种类型的数字存储指令，例如存储在存储器1230中的软件或者固件程序，它能使电子设备1200提供多种服务。
存储器1230用于存储程序代码和事件帧矩阵，并由处理器1210来控制执行，以执行上述实施例中事件数据处理方法的处理步骤。程序代码中可以包括一个或多个软件模块，这一个或多个软件模块可以为图6实施例中提供的软件模块。
需要说明的是，本实施例可以是通用的物理服务器实现的，例如，ARM服务器或者X86服务器，也可以是基于通用的物理服务器结合NFV技术实现的虚拟机实现的，虚拟机指通过软件模拟的具有完整硬件系统功能的、运行在一个完全隔离环境中的完整计算机系统，本申请不作具体限定。
存储器1230可以包括易失性存储器(Volatile Memory)，例如随机存取存储器(Random Access Memory，RAM)；存储器1230也可以包括非易失性存储器(Non-Volatile Memory)，例如只读存储器(Read-Only Memory，ROM)、快闪存储器(Flash Memory)、硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；存储器1230还可以包括上述种类的组合。存储器1230可以存储有程序代码，具体执行图3实施例中的步骤S310-步骤S350及其可选步骤，这里不再进行赘述。
通信接口1220可以为有线接口(例如以太网接口)，可以为内部接口(例如高速串行计算机扩展总线(Peripheral Component Interconnect express，PCIe)总线接口)、有线接口(例如以太网接口)或无线接口(例如蜂窝网络接口或使用无线局域网接口)，用于与其他设备或模块进行通信。
需要说明的，图6仅仅是本申请实施例的一种可能的实现方式，实际应用中，电子设备1200还可以包括更多或更少的部件，这里不作限制。关于本申请实施例中未示出或未描述的内容，可参见前述图3实施例中的相关阐述，这里不再赘述。
应理解，图6所示的电子设备还可以是至少一个服务器构成的计算机集群，本申请不作具体限定。
本申请实施例还提供一种计算机可读存储介质，计算机可读存储介质中存储有指令，当其在处理器上运行时，图3所示的方法流程得以实现。
本申请实施例还提供一种计算机程序产品，当计算机程序产品在处理器上运行时，图3所示的方法流程得以实现。
上述实施例，可以全部或部分地通过软件、硬件、固件或其他任意组合来实现。当使用软件实现时，上述实施例可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载或执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以为通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line，DSL)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集合的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(Digital Video Disc，DVD)、或者半导体介质。半导体介质可以是SSD。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (16)

  1. 一种事件数据处理方法，其特征在于，包括：
    第一电子设备获取事件相机拍摄的事件数据，其中，所述事件数据包括事件发生的时间戳、像素点坐标和亮度变化情况；
    所述第一电子设备根据预设时间段内的多个事件数据，生成事件帧矩阵，其中，所述事件帧矩阵中的矩阵元素的位置对应像素点坐标，矩阵元素的值表示对应像素点有无事件以及有事件的像素点的事件极性，像素点有事件表示像素点在预设时间段内亮度发生变化，像素点无事件表示像素点在预设时间段内亮度无变化，所述事件极性包括亮度变高、亮度变低。
  2. 根据权利要求1所述的方法，其特征在于，
    所述预设时间段范围包括1至100毫秒。
  3. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    所述第一电子设备根据像素点数量，确定多个预设存储方式分别存储所述事件帧矩阵占用的存储空间大小；
    所述第一电子设备从所述多个预设存储方式中确定存储所述事件帧矩阵占用的存储空间最小的方式；
    所述第一电子设备根据所述存储所述事件帧矩阵占用的存储空间最小的方式，存储所述事件帧矩阵。
  4. 根据权利要求3所述的方法，其特征在于，所述方法还包括：
    第二电子设备获取感兴趣区域RoI；
    所述第二电子设备根据所述RoI和所述存储所述事件帧矩阵的方式，从所述第一电子设备中读取所述RoI内的像素点有无事件或者有事件的像素点的事件极性。
  5. 根据权利要求4所述的方法，其特征在于，
    所述预设存储方式包括：所述第一电子设备存储所述事件帧矩阵中有事件的像素点的坐标和事件极性。
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    所述第二电子设备根据所述RoI，确定所述事件帧矩阵中所述RoI对应的矩阵元素坐标范围，
    所述第二电子设备根据所述矩阵元素坐标范围，从所述第一电子设备中读取所述RoI内有事件的像素点的坐标和事件极性。
  7. 根据权利要求4所述的方法，其特征在于，
    所述预设存储方式包括：所述第一电子设备存储所述事件帧矩阵中每一行以及前面所有行中有事件的像素点的个数，以及所述事件帧矩阵中有事件的像素点的列坐标和事件极性，或者，
    所述第一电子设备存储所述事件帧矩阵中每行有事件的像素点的个数，以及所述事件帧矩阵中有事件的像素点的列坐标和对应的事件极性。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：
    所述第二电子设备根据所述事件帧矩阵，确定与所述RoI有交集的行并读取所述与所述RoI有交集的行中有事件的像素点的个数，
    所述第二电子设备根据所述与所述RoI有交集的行中有事件的像素点的个数，从所述第一电子设备中读取RoI内有事件的像素点的列坐标和事件极性，进而得到所述RoI内有事件的像素点的坐标和事件极性。
  9. 根据权利要求4所述的方法，其特征在于，
    所述预设存储方式包括：所述第一电子设备根据所述事件帧矩阵，将所述事件帧矩阵分为多个子矩阵，存储前一个子矩阵以及前面所有子矩阵总共占用的存储空间大小以及每个子矩阵中有事件的像素点的坐标以及事件极性，或者，
    所述第一电子设备存储每个子矩阵占用的存储空间大小以及每个子矩阵中有事件的像素点的坐标以及事件极性。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    所述第二电子设备根据所述RoI，确定与所述RoI有交集的子矩阵以及所述与所述RoI有交集的子矩阵占用的存储空间大小，
    所述第二电子设备根据所述与所述RoI有交集的子矩阵的存储空间大小，确定存储所述与所述RoI有交集的子矩阵的数据的存储位置，
    所述第二电子设备根据所述存储位置，从所述第一电子设备中读取所述与所述RoI有交集的子矩阵中有事件的像素点的坐标以及事件极性，进而得到所述RoI内有事件的像素点的坐标和事件极性。
  11. 根据权利要求4所述的方法，其特征在于，
    所述预设存储方式包括：所述第一电子设备存储所述事件帧矩阵。
  12. 根据权利要求11所述的方法，其特征在于，所述方法还包括：
    所述第二电子设备根据RoI，确定所述事件帧矩阵中所述RoI对应的矩阵元素坐标范围，
    所述第二电子设备根据所述矩阵元素坐标范围，从所述第一电子设备中读取所述RoI内每个像素点是否有事件以及有事件的像素点的事件极性。
  13. 根据权利要求5至10中任一项所述的方法，其特征在于，还包括；
    所述第二电子设备根据所述RoI内有事件的像素点坐标和事件极性，将没有事件的像素点标记为亮度不变，得到RoI内每个像素点是否有事件以及有事件的像素点的事件极性。
  14. 一种电子设备，其特征在于，包括处理器和存储器，所述存储器用于存储指令，所述处理器用于执行所述指令，当所述处理器执行所述指令时，执行如权利要求1至13中任一项所述的方法。
  15. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机指令，在被电子设备执行时，电子设备执行如权利要求1至13中任一项所述的方法。
  16. 一种计算机可读存储介质，其特征在于，包括指令，当所述指令在电子设备上运行时，使得所述电子设备执行如权利要求1至13中任一项所述的方法。
PCT/CN2023/102176 2022-06-30 2023-06-25 一种事件数据处理方法及相关设备 WO2024001970A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210778563.8A CN117372941A (zh) 2022-06-30 2022-06-30 一种事件数据处理方法及相关设备
CN202210778563.8 2022-06-30

Publications (1)

Publication Number Publication Date
WO2024001970A1 true WO2024001970A1 (zh) 2024-01-04

Family

ID=89383267

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/102176 WO2024001970A1 (zh) 2022-06-30 2023-06-25 一种事件数据处理方法及相关设备

Country Status (2)

Country Link
CN (1) CN117372941A (zh)
WO (1) WO2024001970A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190014258A1 (en) * 2017-07-05 2019-01-10 Intel Corporation Micro-saccadic actuation for an event camera
CN109716392A (zh) * 2018-05-22 2019-05-03 上海芯仑光电科技有限公司 一种光流计算方法及计算设备
CN110428477A (zh) * 2019-06-24 2019-11-08 武汉大学 一种不受速度影响的事件相机的成图方法
CN114584721A (zh) * 2020-12-01 2022-06-03 三星电子株式会社 视觉传感器、图像处理设备及视觉传感器的操作方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190014258A1 (en) * 2017-07-05 2019-01-10 Intel Corporation Micro-saccadic actuation for an event camera
CN109716392A (zh) * 2018-05-22 2019-05-03 上海芯仑光电科技有限公司 一种光流计算方法及计算设备
CN110428477A (zh) * 2019-06-24 2019-11-08 武汉大学 一种不受速度影响的事件相机的成图方法
CN114584721A (zh) * 2020-12-01 2022-06-03 三星电子株式会社 视觉传感器、图像处理设备及视觉传感器的操作方法

Also Published As

Publication number Publication date
CN117372941A (zh) 2024-01-09

Similar Documents

Publication Publication Date Title
US11190715B2 (en) System and method for event camera data processing
US20190364230A1 (en) Method of processing data for dynamic vision sensor, dynamic vision sensor performing the same and electronic device including the same
JP6849101B2 (ja) 高速で漸進的なスーパーボクセルベースの時空間ビデオセグメンテーション法
CN115460414B (zh) 一种基板管理控制芯片的视频压缩方法、系统及相关组件
US10070134B2 (en) Analytics assisted encoding
KR20200011000A (ko) 증강 현실 프리뷰 및 위치 추적을 위한 장치 및 방법
US11862053B2 (en) Display method based on pulse signals, apparatus, electronic device and medium
CN115209145A (zh) 一种视频压缩方法、系统、装置及可读存储介质
WO2023010755A1 (zh) 一种hdr视频转换方法、装置、设备及计算机存储介质
WO2024001970A1 (zh) 一种事件数据处理方法及相关设备
CN112463598A (zh) 一种压测方法、装置、电子设备和可读存储介质
CN114667725A (zh) 利用低功率同步读出的传感器
JP7457878B2 (ja) イベントベースのイメージセンサを動作させるためのデジタルイベント符号化
TWI586144B (zh) 用於視頻分析與編碼之多重串流處理技術
US20100053326A1 (en) Image sensor, the operating method and usage thereof
WO2023015422A1 (zh) 一种图像的处理方法及其装置
CN117979059A (zh) 视频处理方法、装置、电子设备及存储介质
CN116711298A (zh) 一种数据流传输方法及装置
CN115249207A (zh) 视频超分方法、装置、视频超分模型、介质及终端设备
CN116193129A (zh) 基于图像传感器的数据处理方法及装置、图像处理系统
JP2016174309A (ja) 映像データ管理装置及び映像データ記録システム及び映像データ管理方法及び映像データ管理プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23830148

Country of ref document: EP

Kind code of ref document: A1