WO2022226978A1 - Procédé et dispositif de traitement de données - Google Patents

Procédé et dispositif de traitement de données Download PDF

Info

Publication number
WO2022226978A1
WO2022226978A1 PCT/CN2021/091394 CN2021091394W WO2022226978A1 WO 2022226978 A1 WO2022226978 A1 WO 2022226978A1 CN 2021091394 W CN2021091394 W CN 2021091394W WO 2022226978 A1 WO2022226978 A1 WO 2022226978A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
communication bus
register
read
system time
Prior art date
Application number
PCT/CN2021/091394
Other languages
English (en)
Chinese (zh)
Inventor
刘瑛
窦仁峰
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2021/091394 priority Critical patent/WO2022226978A1/fr
Publication of WO2022226978A1 publication Critical patent/WO2022226978A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/16—Handling requests for interconnection or transfer for access to memory bus

Definitions

  • the present application relates to the field of computer technology, and in particular, to a data processing method and device.
  • IMU Inertial Measurement Unit
  • the system time corresponding to the IMU data and the image data can be determined first, and then the synchronization relationship between the image data and the IMU data can be determined based on the system time.
  • the system time corresponding to the first IMU data in the multiple IMU data corresponding to a single image is determined by means of IMU data dotting, and the system time corresponding to other IMU data in the multiple IMU data is determined by software.
  • the embodiments of the present application provide a data processing method and apparatus, which are used to solve the problem of low efficiency in the prior art, which is to determine the system time corresponding to the IMU data by means of software.
  • an embodiment of the present application provides a data processing method, including:
  • the redundant data is replaced with the first system time to generate combined data including the target sensor data and the first system time.
  • an embodiment of the present application provides a data processing apparatus, including:
  • a first interrupt input port configured to receive a first interrupt signal triggered by an external sensor, where the first interrupt signal is used to indicate that the sensor has generated new sensing data
  • a first latch for storing the current first system time in response to the first interrupt signal
  • a first data acquisition circuit for acquiring redundant data and target sensing data read from the sensor in response to the first interrupt signal
  • a replacement circuit for replacing the redundant data with the first system time to generate combined data including the target sensing data and the first system time.
  • the embodiments of the present application provide a data processing method and device, by receiving a first interrupt signal triggered by an external sensor, in response to the first interrupt signal, storing the current first system time and acquiring redundant data read from the sensor and Target sensing data, replacing redundant data with the first system time to generate combined data including the target sensing data and the first system time, realizes the hardware method to determine the system time corresponding to the sensor data without software. , and since the execution efficiency of the hardware method is higher than that of the software method, the efficiency of determining the system time corresponding to the sensor data can be improved.
  • FIG. 1 is a schematic diagram of an application scenario of a data processing method provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a data processing method provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a data processing method provided by another embodiment of the present application.
  • 4A is a schematic diagram of data in a register when no replacement is performed in the process of data buffering according to an embodiment of the present application
  • FIG. 4B is a schematic diagram of data in a register when replacement is performed in the process of data buffering according to an embodiment of the present application
  • Fig. 5A is the schematic diagram of the data relationship of the first communication bus and the register that an embodiment of the application provides;
  • 5B is a schematic diagram of a data relationship between a first communication bus and a register provided by another embodiment of the application.
  • FIG. 6 is a schematic flowchart of a data processing method provided by another embodiment of the present application.
  • FIG. 7A is a schematic diagram of data in a register when no replacement is performed during data reading according to an embodiment of the present application.
  • FIG. 7B is a schematic diagram of data in a register when replacing in the process of data reading according to an embodiment of the present application.
  • 8A is a schematic diagram of a data relationship between a second communication bus and a register according to an embodiment of the application
  • 8B is a schematic diagram of a data relationship between a second communication bus and a register provided by another embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application applied to an ISP chip.
  • the application scenario may include a sensor 11 and a data processing device 12 .
  • the sensor 11 can generate sensing data, and can also output an interrupt signal for indicating that the sensor 11 has generated new sensing data.
  • the data processing device 12 can read the sensor data from the sensor 11 according to the interrupt signal output by the sensor 11, and can also implement the data processing method provided by the embodiment of the present application, so as to realize the determination of the generation of the sensor 11 by hardware.
  • the senor 11 may be any type of sensor whose generated sensing data needs to correspond to the system time.
  • the sensor 11 may include an IMU sensor.
  • the data processing apparatus 12 may specifically be any type of apparatus capable of implementing the data processing method in a hardware manner.
  • the specific implementation of the data processing device 12 can be flexibly selected according to requirements.
  • the data processing device 12 can be applied to an image signal processor (Digital Signal Processor, ISP) chip, that is, an ISP chip can be used.
  • ISP Digital Signal Processor
  • FIG. 2 is a schematic flowchart of a data processing method provided by an embodiment of the present application.
  • the data processing method provided by this embodiment may be implemented by the data processing apparatus 12 in FIG. 1 .
  • the method of this embodiment may include:
  • Step 21 Receive a first interrupt signal triggered by an external sensor, where the first interrupt signal is used to indicate that the sensor has generated new sensing data.
  • the sensor is the sensor 11 in FIG. 1 , the sensor is independent of the data processing device 12 , and when the sensor generates new sensing data, it can trigger a trigger to indicate that the sensor has generated
  • the interrupt signal of the new sensing data ie, the first interrupt signal
  • the first interrupt signal is output through the output port of the sensor.
  • An interrupt input port (hereinafter referred to as the first interrupt input port) of the data processing device 12 can be electrically connected to the output port of the sensor.
  • the first interrupt input port of the data processing device 12 can receive the first interrupt signal.
  • the data processing device 12 can receive a first interrupt signal triggered by an external sensor.
  • the data processing device 12 when the data processing device 12 receives the first interrupt signal, the data processing device 12 can be triggered to respond to the first interrupt signal.
  • Step 22 in response to the first interrupt signal, store the current first system time and acquire redundant data and target sensing data read from the sensor.
  • the first system time stored in response to a certain first interrupt signal is the system time corresponding to the target sensing data read from the sensor in response to the first interrupt signal.
  • a latch in the data processing device 12 can store the current first system time in response to the first interrupt signal.
  • the processing circuit to which the data processing apparatus 12 is applied may include a time generator for generating system time, and in response to the first interrupt signal, the first latch may generate the time. The system time currently generated by the device is latched.
  • a data acquisition circuit (hereinafter referred to as the first data acquisition circuit) in the data processing device 12 can acquire redundant data and target sensing data read from the sensor in response to the first interrupt signal.
  • the new sensing data generated by the sensor may be stored in the register of the sensor, and the processing circuit applied by the data processing device 12 may communicate with the sensor through a communication bus (hereinafter referred to as the first communication bus).
  • the sensor communicates, and the data acquisition circuit can read the register of the sensor through the first communication bus, so that the target sensing data and redundant data in the register of the sensor can communicate through the first communication.
  • the bus is input to the data processing device 12 so that the data post-processing device 12 can obtain redundant data read from the sensors and target sensing data.
  • the target sensor data is the sensor data required by the data processing device 12
  • the redundant data is data not required by the data processing device 12
  • the redundant data may be generated by the sensor
  • the redundant data may also be invalid data not generated by the sensor.
  • the redundant data can be understood as some data (for example, several bytes) read from the register of the sensor, and the size of the redundant data can be related to the system time generated by the time generator.
  • the size of the occupied storage space is related, and the size of the redundant data may be greater than or equal to the size of the storage space occupied by the system time.
  • the target sensory data and the redundant data may be read together.
  • the storage address of the target sensing data in the register of the sensor is 0x20 to 0x30
  • the data of the storage address 0x20 to 0x35 in the register of the sensor can be read, and the data includes the storage address at 0x20 to 0x35.
  • the target sensor data at memory addresses 0x20 to 0x30 and redundant data at memory addresses 0x31 to 0x35, or the sensor's registers at memory addresses 0x1B to 0x30 can be read, including data at memory addresses 0x1B to 0x30. Redundant data at 0x1F and target sensor data at memory addresses 0x20 to 0x30.
  • the target sensory data and the redundant data may be read separately.
  • the storage address of the target sensing data in the register of the sensor is 0x20 to 0x30
  • the data of the storage address 0x20 to 0x30 in the register of the sensor can be read first, and then the The registers of the sensor store data of addresses 0x31-0x35, or, the data of addresses 0x1B to 0x1F stored in the registers of the sensor may be read first, and then the data of addresses 0x20 to 0x30 of the registers of the sensor are read.
  • Step 23 Replace the redundant data with the first system time to generate combined data including the target sensing data and the first system time.
  • the redundant data may be replaced with the first system time by a replacement circuit in the data processing apparatus.
  • the target sensing data and the redundant data are read together, the target sensing data and the redundant data are combined together, and the redundant data can be directly By replacing the data with the first system time, combined data including the target sensor data and the first system time is generated.
  • the target sensor data and the redundant data when the target sensor data and the redundant data are read separately, the target sensor data and the redundant data are not combined first, and the target sensor data can be Combine with the redundant data and then replace the redundant data with the first system time to generate combined data including the target sensor data and the first system time, or by first describing The redundant data is replaced with the first system time and then the target sensor data and the first system time data are combined to generate combined data including the target sensor data and the first system time.
  • the redundant data may be replaced with the first system time by adding an additional processing flow on the basis of the existing data processing flow, for example, after reading the data from the sensor On the basis of buffering the target sensor data and the redundant data to the register of the processing circuit applied by the data processing device 12, the redundant data in the register is replaced by the first system through an additional processing flow time.
  • the redundant data may be replaced with the first system time by modifying the existing data processing flow, thereby facilitating simplified implementation. The following mainly takes the method of modifying the existing data processing flow as an example for specific description.
  • the data processing method by receiving a first interrupt signal triggered by an external sensor, in response to the first interrupt signal, the current first system time is stored and redundant data and target sensing data read from the sensor are acquired , replace the redundant data with the first system time to generate combined data including the target sensor data and the first system time, and realize the hardware method to determine the system time corresponding to the sensor data without software determination, and because the hardware
  • the execution efficiency of the method is higher than that of the software method, so the efficiency of determining the system time corresponding to the sensor data can be improved.
  • FIG. 3 is a schematic flowchart of a data processing method provided by another embodiment of the present application. Based on the embodiment shown in FIG. 2 , this embodiment mainly describes the method of replacing the redundant data with the first system time. An optional implementation.
  • the data processing method in this embodiment is applied to a first processing circuit, and the first processing circuit communicates with the sensor through a first communication bus. As shown in FIG. 3 , the method in this embodiment may include:
  • Step 31 Receive a first interrupt signal triggered by an external sensor, where the first interrupt signal is used to indicate that the sensor has generated new sensing data.
  • step 31 is similar to step 21, and details are not repeated here.
  • Step 32 in response to the first interrupt signal, store the current first system time and acquire redundant data and target sensing data read from the sensor.
  • step 32 is similar to step 22, and details are not repeated here.
  • Step 33 in the process of data buffering for the data signal input by the first communication bus, replace the redundant data with the first system time, so as to generate the target sensor data and the first system time. Combined data for a system time.
  • the timing of replacing the redundant data with the first system time may be any data signal input to the first communication bus during the process of data buffering for the data signal input from the first communication bus.
  • the specific implementation of replacing the redundant data with the first system time in the process of data caching belongs to the protection scope of the present application.
  • the redundant data is not replaced with the first system time in the process of buffering the data signal input from the first communication bus, as shown in FIG. 4A .
  • the target sensing data and the redundant data may be buffered in the register of the first processing circuit. If the redundant data is replaced with the first system time in the process of buffering the data signal input from the first communication bus, then as shown in FIG. 4B , the register of the first processing circuit The combined data including the target sensing data and the first system time may be cached in the .
  • the redundant data can be replaced with the first system time in the process of data buffering for the data signal input by the first communication bus by means of data signal replacement.
  • step 33 may specifically include the following steps A and B.
  • Step A when the first communication bus inputs the data signal corresponding to the target sensor data, buffer the data corresponding to the data signal to a register;
  • Step B when the data signal corresponding to the redundant data is input to the first communication bus, the data corresponding to the data signal is buffered to the register instead of buffering the data corresponding to the target data signal to the register.
  • the target data signal is a data signal corresponding to the first system time.
  • step A and step B may be performed first and then step B.
  • step A the target sensor data can be cached in the register first
  • step B the first system time can be cached in the register, so that the register Combined data including the target sensing data and the first system time is cached in the .
  • a rectangular box may represent 1 byte of data; a rectangular box filled with horizontal stripes represents data input through the first communication bus, wherein byte 0 to byte 13 filled with horizontal stripes represent the For the target sensing data, bytes 14 to 17 filled with horizontal stripes represent the redundant data, and bytes 14'-byte 17' filled with vertical stripes represent the first system time. It can be seen in conjunction with FIG.
  • step A the bytes 0 to 13 filled with the horizontal stripes can be sequentially cached to the register, and by performing step B again, the bytes 14'-byte 17 filled with the vertical stripes can be sequentially ' and then also buffer into the register.
  • step A and step B may be that step B is executed first and then step A is executed.
  • step B the first system time can be cached to the register first
  • step A the target sensor data can be cached in the register, so that the register Combined data including the target sensing data and the first system time is cached in the .
  • a rectangular box may represent 1 byte of data; the rectangular box filled with horizontal stripes represents data input through the first communication bus, wherein byte 0 to byte 3 filled with horizontal stripes represent the Redundant data, bytes 4 to 17 filled with horizontal stripes represent the target sensing data, and bytes 0'-byte 3' filled with vertical stripes represent the first system time. It can be seen in conjunction with FIG.
  • step B the byte 0' to byte 3' filled with vertical stripes can be sequentially cached to the register, and by performing step A again, the byte 4-byte filled with horizontal stripes can be sequentially 17 is then also buffered into the register.
  • FIG. 5A and FIG. 5B represents the data transmission direction on the first communication bus.
  • it may be determined directly based on the data amount of the data corresponding to the data signal that has been input to the first communication bus, whether the first communication bus is currently inputting the data signal corresponding to the target sensor data or the data signal corresponding to the target sensor data.
  • the data signal corresponding to the redundant data may be determined directly based on the data amount of the data corresponding to the data signal that has been input to the first communication bus, whether the first communication bus is currently inputting the data signal corresponding to the target sensor data or the data signal corresponding to the target sensor data.
  • the method provided by the embodiment of the present application may further include: counting the clock edges of the clock signal input by the first communication bus, and the count value can indicate that the target input is the target
  • the data signal corresponding to the sensing data is also the data signal corresponding to the redundant data.
  • the transmission timing of the target sensing data on the first communication bus is earlier than the redundant data; the count value for counting the clock edges of the clock signal input from the first communication bus is smaller than the first
  • the threshold value represents the data signal corresponding to the target sensing data input by the first communication bus, and the count value of the clock edge of the clock signal input by the first communication bus reaches the first threshold value, which represents the first communication bus.
  • the communication bus inputs the redundant corresponding data signals. It can be understood that, the first threshold may be related to the data amount of the target transmission data and the number of bits of the data bus of the first communication bus.
  • the transmission timing of the target sensing data on the first communication bus is later than the redundant data; the count value for counting the clock edges of the clock signal input from the first communication bus is less than the first
  • the two thresholds represent that the redundant corresponding data signal is input to the first communication bus, and the count value that counts the clock edges of the clock signal input to the first communication bus reaches the second threshold to represent the first communication
  • the bus inputs a data signal corresponding to the target sensing data.
  • the second threshold may be related to the data amount of the redundant data and the number of bits of the data bus of the first communication bus.
  • the redundant data may be replaced with the first system time in the buffering stage of buffering the data into the register.
  • step 33 may specifically include: serial-to-parallel conversion to the data signal input by the first communication bus; When the signal is the data signal corresponding to the target sensing data, buffer the data corresponding to the serial-to-parallel converted data signal in the register; and, when the data signal is the data signal corresponding to the redundant signal, Serial-to-parallel conversion is performed on the target data signal, and data corresponding to the serial-to-parallel converted data signal is buffered in the register.
  • SPI Serial Peripheral Interface
  • the redundant data may be replaced with the first system time in a processing stage prior to the buffering stage in which the data is buffered into the register.
  • the redundant data may be replaced with the first system time in a serial-to-parallel conversion stage.
  • the first communication bus includes an SPI bus
  • step 33 may specifically include: when a data signal corresponding to the target sensing data is input to the first communication bus, performing serial-to-parallel conversion on the data signal, When the data signal corresponding to the redundant data is input to the first communication bus, the serial-to-parallel conversion of the data signal is replaced by the serial-to-parallel conversion of the target data signal, and the serial-to-parallel converted data The data corresponding to the signal is buffered into the register.
  • the data processing method by receiving a first interrupt signal triggered by an external sensor, in response to the first interrupt signal, the current first system time is stored and redundant data and target sensing data read from the sensor are acquired , in the process of data buffering of the data signal input by the first communication bus, the redundant data is replaced with the first system time to generate combined data including the target sensor data and the first system time, which realizes the determination by hardware.
  • the system time corresponding to the sensor data.
  • FIG. 6 is a schematic flowchart of a data processing method provided by another embodiment of the present application. Based on the embodiment shown in FIG. 2 , this embodiment mainly describes the method of replacing the redundant data with the first system time. Another optional implementation.
  • the data processing method in this embodiment is applied to a second processing circuit, and the second processing circuit passes through a register and a second communication bus, and the register buffers the target sensing data and the sensor data read from the sensor.
  • the method of this embodiment may include:
  • Step 61 Receive a first interrupt signal triggered by an external sensor, where the first interrupt signal is used to indicate that the sensor has generated new sensing data.
  • step 61 is similar to step 21, and details are not repeated here.
  • Step 62 in response to the first interrupt signal, store the current first system time and acquire redundant data and target sensing data read from the sensor.
  • step 62 is similar to step 22, and details are not repeated here.
  • Step 63 in the process of reading data from the register through the second communication bus, replace the redundant data with the first system time, so as to generate the target sensor data and the target sensor data. Combined data for the first system time.
  • the timing of replacing the redundant data with the first system time may be to read data from the register buffered with the target sensor data and the redundant data through the second communication bus
  • any specific implementation of replacing the redundant data with the first system time in the process of reading data from the register through the second communication bus belongs to the protection scope of the present application. .
  • the redundant data is not replaced with the first system time in the process of reading data from the register through the second communication bus, as shown in the figure.
  • the data transmitted on the second communication bus may be the target sensing data and the redundant data. If the redundant data is replaced with the first system time in the process of reading data from the register through the second communication bus, as shown in FIG. 7B , the second communication bus The transmitted data may be the target sensing data and the first system time.
  • the redundant data may be replaced with the first system time during the process of reading data from the register through the second communication bus by means of a read operation replacement.
  • step 63 may specifically include the following steps C and D.
  • Step C when the data belonging to the target sensing data is about to be read from the register, read the register, and input the read data into the second communication bus;
  • Step D when the data belonging to the redundant data is about to be read from the register, the read operation of the register is replaced by the read operation of the target storage space, and the read data is input into the first step.
  • Two communication buses, the target storage space is used to store the first system time.
  • step C and step D may be to execute step C first and then execute step D.
  • step C the target sensor data buffered in the register can be input into the second communication bus
  • step D the first system time can be input into the second communication bus. Two communication buses, so that the combined data including the target sensing data and the first system time is transmitted on the second communication bus.
  • FIG. 8A a rectangular box can represent data of 1 byte; the rectangular box filled with left diagonal stripes represents the data buffered in the register, wherein, bytes 0 to 13 filled with left diagonal stripes represent the For the target sensing data, bytes 14 to 17 filled with left diagonal stripes represent the redundant data, and bytes 14'-byte 17' filled with right diagonal stripes represent the first system time. It can be seen in conjunction with FIG.
  • step C by first performing step C, the bytes 0 to 13 filled with the left diagonal stripe can be input into the second communication bus in turn, and by performing step D again, the bytes filled with the right diagonal stripe can be sequentially inputted. 14'-byte 17' is then also input to the second communication bus.
  • step C and step D may be to execute step D first and then execute step C.
  • step D the first system time can be input into the second communication bus
  • step C the target sensor data buffered in the register can be input into the second communication bus.
  • Two communication buses so that the combined data including the target sensing data and the first system time is transmitted on the second communication bus.
  • the data buffered in the register is the same as the data on the second communication bus.
  • the relationship between the two can be as shown in FIG. 8B.
  • a rectangular box can represent data of 1 byte; the rectangular box filled with left diagonal stripes represents the data buffered in the register, wherein byte 0 to byte 3 filled with left diagonal stripes represent the Redundant data, bytes 4 to 17 filled with left diagonal stripes represent the target sensing data, and bytes 0'-byte 3' filled with right diagonal stripes represent the first system time. It can be seen in conjunction with FIG.
  • step D the bytes 0' to 3' filled with the right oblique stripes can be sequentially input into the second communication bus, and by performing step C again, the left oblique stripes can be filled in sequence.
  • step C the left oblique stripes can be filled in sequence.
  • step C the left oblique stripes can be filled in sequence.
  • Byte 4 - Byte 17 then also enter the second communication bus.
  • FIG. 8A and FIG. 8B represents the data transmission direction on the second communication bus.
  • the method provided by the embodiment of the present application may further include: counting the read operations on the register, and the count value can indicate that the data to be read from the register belongs to the target sensor data , which is also the data belonging to the redundant data.
  • the read timing of the target sensing data is earlier than the redundant data; the count value counted for the read operation of the register is less than the third threshold, indicating that the read from the register is about to be read.
  • the data of the target sensing data, and the count value counted for the read operation of the register reaching the third threshold indicates that the data belonging to the redundant data is about to be read from the register.
  • the third threshold may be related to the data volume of the target sensing data and the data volume read by the read operation in a single time.
  • the read timing of the target sensing data is later than the redundant data; the count value counted for the read operation of the register is smaller than the fourth threshold, indicating that the register is about to be read from the register belonging to the For the data of the redundant data, when the count value counted for the read operation of the register reaches the fourth threshold, it indicates that the data belonging to the target sensing data is about to be read from the register.
  • the fourth threshold may be related to the data amount of the redundant data and the data amount read by the read operation in a single time.
  • the data processing method by receiving a first interrupt signal triggered by an external sensor, in response to the first interrupt signal, the current first system time is stored and redundant data and target sensing data read from the sensor are acquired , in the process of reading data from the register through the second communication bus, the redundant data is replaced with the first system time, so as to generate combined data including the target sensor data and the first system time, which realizes the determination of transmission by hardware.
  • the system time corresponding to the sensor data.
  • the application scenarios of the embodiments of the present application may further include a photographing apparatus, the photographing apparatus may generate image data, and may also output an interrupt signal for indicating that the photographing apparatus has generated new image data.
  • the photographing device may be, for example, a video camera, a camera, a camera, a mobile phone, and the like.
  • the data processing apparatus may determine the system time corresponding to the video frame generated by the photographing apparatus. Based on this, the following steps E and F may be further included on the basis of the above method embodiments.
  • Step E Receive a second interrupt signal triggered by an external photographing device, where the second interrupt signal is used to indicate that the photographing device has generated new image data.
  • the photographing device is independent of the data processing device 12, and when generating new image data, the photographing device can trigger an interrupt signal (ie, the second interrupt) for indicating that the photographing device has generated new image data signal), and output the second interrupt signal through the output port of the photographing device.
  • Another interrupt input port of the data processing device 12 (hereinafter referred to as the second interrupt input port) can be electrically connected to the output port of the photographing device, and the photographing device outputs the second interrupt through the output port signal, the second interrupt input port of the data processing device 12 can receive the second interrupt signal. In this way, the data processing device 12 can receive the second interrupt signal triggered by the external imaging device.
  • the data processing device 12 when the data processing device 12 receives the second interrupt signal, the data processing device 12 can be triggered to respond to the second interrupt signal.
  • Step F in response to the second interrupt signal, store the current second system time and acquire the video frame read from the photographing device.
  • the second system time stored in response to a certain second interrupt signal is the system time corresponding to the video frame read from the photographing device in response to the second interrupt signal.
  • another latch in the data processing device 12 may store the current second system time in response to the second interrupt signal.
  • the processing circuit to which the data processing device 12 is applied may include a time generator for generating system time, and in response to the second interrupt signal, the second latch may generate the time. The system time currently generated by the device is latched.
  • Another data acquisition circuit in the data processing device 12 (hereinafter referred to as a second data acquisition circuit) can acquire the video frame read from the camera in response to the second interrupt signal. It should be noted that, the present application does not limit the specific manner of the video frame read from the photographing device.
  • the synchronization relationship between the video frame and the target sensing data may also be determined by the data processing device. Based on this, on the basis of the above method embodiments, the method may further include: according to the combined data and the target sensor data.
  • the second system time corresponding to the video frame determines the synchronization relationship between the video frame and the target sensing data. Since the combined data can indicate the correspondence between the target sensing data and the system time, and the system time corresponding to the video frame has been known, it can be determined based on the system time that the video frame and the target transmission Synchronization of sense data. It should be noted that this application does not limit the specific manner of determining the synchronization relationship between video frames and sensor data based on the system time.
  • electronic anti-shake processing may also be performed on the video frame by the data processing device.
  • the method may further include: according to the synchronization relationship, perform an electronic anti-shake processing on the video frame. Perform electronic image stabilization. It should be noted that this application does not limit the specific manner of performing electronic anti-shake processing from the video frame.
  • the ISP chip and the IMU sensor may be included in a movable device provided with the photographing device.
  • the movable device includes one or more of a drone, an unmanned vehicle, a handheld gimbal or a gimbal camera.
  • FIG. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
  • the data processing apparatus can implement the data processing method described in the foregoing embodiment.
  • the data processing device 90 includes:
  • the first interrupt input port 91 is used to receive a first interrupt signal triggered by an external sensor, and the first interrupt signal is used to indicate that the sensor has generated new sensing data;
  • a first latch 92 configured to store the current first system time in response to the first interrupt signal
  • a first data acquisition circuit 93 configured to acquire redundant data and target sensing data read from the sensor in response to the first interrupt signal
  • a replacement circuit 94 is configured to replace the redundant data with the first system time to generate combined data including the target sensing data and the first system time.
  • the data processing apparatus 90 may be applied to a first processing circuit, and the first processing circuit communicates with the sensor through a first communication bus.
  • the replacement circuit 94 includes a first replacement circuit, configured to replace the redundant data with all the redundant data in the process of data buffering for the data signal input from the first communication bus. the first system time.
  • the first replacement circuit is specifically used for:
  • buffering the data corresponding to the data signal in the register is replaced by buffering the data corresponding to the target data signal in the register.
  • the target data signal is the data signal corresponding to the first system time.
  • the first communication bus includes a serial peripheral interface bus; the first replacement circuit is specifically used for:
  • serial-to-parallel conversion is performed on the data signal, and when a data signal corresponding to the redundant data is input to the first communication bus, the The serial-to-parallel conversion of the data signal is replaced by the serial-to-parallel conversion of the target data signal, and the data corresponding to the serial-to-parallel converted data signal is buffered in the register.
  • the device further includes a first counting circuit for counting the clock edges of the clock signal input by the first communication bus, and the count value can indicate that the target transmission is input by the first communication bus.
  • the data signal corresponding to the sense data is also the data signal corresponding to the redundant data.
  • the transmission timing of the target sensing data on the first communication bus is earlier than the redundant data; the count value being smaller than the first threshold indicates that the first communication bus inputs the target sensing data corresponding to the data.
  • the count value For a data signal, when the count value reaches the first threshold, it indicates that the first communication bus inputs a data signal corresponding to the redundant data.
  • the transmission timing of the target sensing data on the first communication bus is later than the redundant data; the count value being smaller than the second threshold indicates that the first communication bus inputs data corresponding to the redundant data signal, the count value reaching the second threshold represents that the first communication bus inputs a data signal corresponding to the target sensing data.
  • the data processing device 90 can be applied to a second processing circuit, the second processing circuit includes a register and a second communication bus, the register buffers the target read from the sensor sensory data and the redundant data.
  • the replacement circuit 94 includes a second replacement circuit, configured to replace the redundant data with a process of reading data from the register through the second communication bus the first system time.
  • the second replacement circuit is specifically used for:
  • the read operation of the register is replaced by the read operation of the target storage space, and the read data is input into the second a communication bus, and the target storage space is used to store the first system time.
  • the device further includes a second counting circuit, configured to count the read operations on the register, and the count value can indicate that the data to be read from the register belongs to the target sensor data. , which is also the data belonging to the redundant data.
  • a second counting circuit configured to count the read operations on the register, and the count value can indicate that the data to be read from the register belongs to the target sensor data. , which is also the data belonging to the redundant data.
  • the read timing of the target sensor data is earlier than the redundant data; the count value is less than the third threshold value indicates that the data belonging to the target sensor data is about to be read from the register, and the count value reaches The third threshold indicates that data belonging to the redundant data is about to be read from the register.
  • the read timing of the target sensing data is later than the redundant data; the count value being less than the fourth threshold indicates that the data belonging to the redundant data is about to be read from the register, and the count value reaches the predetermined threshold.
  • the fourth threshold indicates that data belonging to the target sensor data is about to be read from the register.
  • the senor includes an IMU sensor.
  • the device 90 further includes:
  • a second interrupt input port configured to receive a second interrupt signal triggered by an external photographing device, where the second interrupt signal is used to indicate that the photographing device has generated new image data
  • a second latch configured to store the current second system time in response to the second interrupt signal
  • a second data acquisition circuit configured to acquire the video frame read from the photographing device in response to the second interrupt signal.
  • the apparatus further includes a processor configured to determine a synchronization relationship between the video frame and the target sensing data according to the combined data and the second system time corresponding to the video frame.
  • the processor is further configured to perform electronic anti-shake processing on the video frame according to the synchronization relationship.
  • the data processing apparatus is applied to an ISP chip.
  • the ISP chip and the IMU sensor are included in a movable device provided with the photographing device.
  • the movable device includes one or more of a drone, an unmanned vehicle, a handheld gimbal or a gimbal camera.
  • the data processing apparatus provided in this embodiment can be used to execute the technical solutions of the foregoing method embodiments, and the implementation principles and technical effects thereof are similar to those of the method embodiments, and are not repeated here.
  • the sensor is an IMU sensor
  • the ISP chip performs electronic anti-shake processing on the video frames captured by the shooting device
  • the implementation of the ISP chip may be as shown in FIG. 10 .
  • the IMU interrupt is the aforementioned first interrupt signal
  • the video frame interrupt is the aforementioned second interrupt signal
  • SPI refers to the bus interface of the SPI bus, which is used to read target sensing data and redundant data from the IMU sensor
  • Latch 1 is the aforementioned first latch
  • latch 2 is the aforementioned second latch
  • the processor is used for performing electronic anti-shake processing.
  • the latch 1 latches the current system time (ie, the first system time) and acquires the target sensing data and redundant data read from the IMU sensor through the SPI interface, and the replacement circuit converts the The redundant data is replaced with the first system time to generate combined data including the target sensing data and the first system time, so that the processor can know the system time corresponding to the target sensing data.
  • the latch 2 latches the current system time (ie, the second system time), so that the processor can know the system time corresponding to the video frame.
  • the processor may perform electronic anti-shake processing on the video frame according to the target sensing data and the system time corresponding to the video frame respectively.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Arrangements For Transmission Of Measured Signals (AREA)

Abstract

Procédé et dispositif de traitement de données. Le procédé consiste à : recevoir un premier signal d'interruption déclenché par un capteur externe, le premier signal d'interruption étant utilisé pour indiquer que le capteur a généré de nouvelles données de détection (21) ; en réponse au premier signal d'interruption, stocker le premier temps de système actuel et acquérir des données redondantes lues à partir du capteur et des données de détection cibles (22) ; et remplacer les données redondantes par le premier temps de système pour générer des données combinées comprenant les données de détection cibles et le premier temps de système (23). Selon le procédé, le temps de système correspondant aux données de détection est déterminé dans un mode matériel, et l'efficacité de détermination du temps de système correspondant aux données de capteur peut être améliorée.
PCT/CN2021/091394 2021-04-30 2021-04-30 Procédé et dispositif de traitement de données WO2022226978A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/091394 WO2022226978A1 (fr) 2021-04-30 2021-04-30 Procédé et dispositif de traitement de données

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/091394 WO2022226978A1 (fr) 2021-04-30 2021-04-30 Procédé et dispositif de traitement de données

Publications (1)

Publication Number Publication Date
WO2022226978A1 true WO2022226978A1 (fr) 2022-11-03

Family

ID=83846732

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/091394 WO2022226978A1 (fr) 2021-04-30 2021-04-30 Procédé et dispositif de traitement de données

Country Status (1)

Country Link
WO (1) WO2022226978A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105940390A (zh) * 2013-12-30 2016-09-14 谷歌技术控股有限责任公司 用于同步从设备的多个传感器接收的数据的方法和系统
CN106027909A (zh) * 2016-07-05 2016-10-12 大连海事大学 一种基于微机电惯性传感器与摄像机的船载视频同步采集系统及方法
CN107172320A (zh) * 2017-06-21 2017-09-15 成都理想境界科技有限公司 数据同步方法及摄像设备
US20170289646A1 (en) * 2016-04-01 2017-10-05 Intel Corporation Multi-camera dataset assembly and management with high precision timestamp requirements
CN110139066A (zh) * 2019-03-24 2019-08-16 初速度（苏州）科技有限公司 一种传感器数据的传输系统、方法和装置
CN111813716A (zh) * 2019-11-11 2020-10-23 北京嘀嘀无限科技发展有限公司 多传感器数据同步、电子设备及存储介质
CN112154614A (zh) * 2019-08-29 2020-12-29 上海飞来信息科技有限公司 传感系统、传感设备及其控制方法、可移动平台和存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105940390A (zh) * 2013-12-30 2016-09-14 谷歌技术控股有限责任公司 用于同步从设备的多个传感器接收的数据的方法和系统
US20170289646A1 (en) * 2016-04-01 2017-10-05 Intel Corporation Multi-camera dataset assembly and management with high precision timestamp requirements
CN106027909A (zh) * 2016-07-05 2016-10-12 大连海事大学 一种基于微机电惯性传感器与摄像机的船载视频同步采集系统及方法
CN107172320A (zh) * 2017-06-21 2017-09-15 成都理想境界科技有限公司 数据同步方法及摄像设备
CN110139066A (zh) * 2019-03-24 2019-08-16 初速度（苏州）科技有限公司 一种传感器数据的传输系统、方法和装置
CN112154614A (zh) * 2019-08-29 2020-12-29 上海飞来信息科技有限公司 传感系统、传感设备及其控制方法、可移动平台和存储介质
CN111813716A (zh) * 2019-11-11 2020-10-23 北京嘀嘀无限科技发展有限公司 多传感器数据同步、电子设备及存储介质

Similar Documents

Publication Publication Date Title
US9813783B2 (en) Multi-camera dataset assembly and management with high precision timestamp requirements
CN110035228B (zh) 摄像头防抖系统、方法、电子设备和计算机可读存储介质
CN109756671B (zh) 使用多个相机记录图像的电子装置及其操作方法
JP5536010B2 (ja) 電子カメラ、撮像制御プログラム及び撮像制御方法
JP2001189886A (ja) 撮像装置、情報処理装置、画像処理システム、画像処理方法、及び記憶媒体
US11758284B2 (en) Image processors and image processing methods
CN111882655A (zh) 三维重建的方法、装置、系统、计算机设备和存储介质
JP2008022373A (ja) 改竄検知情報生成装置、撮像装置、改竄検知情報生成方法、プログラムおよび記憶媒体
CN116368814A (zh) 无fov损失的空间对准变换
WO2022226978A1 (fr) Procédé et dispositif de traitement de données
CN111988526B (zh) 移动终端及图像数据处理方法
US11159725B2 (en) Image processing apparatus, image processing method, and recording medium
US8054345B2 (en) Electronic zooming apparatus and digital camera
CN116668837B (zh) 显示缩略图像的方法和电子设备
CN113329170B (zh) 图像抖动校正方法、装置、计算机设备和可读存储介质
JP2018157371A (ja) 撮像装置及び欠陥画素の補正方法
CN109391788B (zh) 用于处理图像信号的系统
JP6763215B2 (ja) 信号処理装置
KR102328151B1 (ko) 제로 레이턴시를 위한 데이터 전송 장치 및 방법
CN112449137A (zh) 基于移动产业处理器接口的数据传输系统及数据传输方法
WO2022246752A1 (fr) Procédé de génération d'une image, dispositif électronique, appareil et support de stockage lisible par ordinateur
JP2008118294A (ja) 撮像装置及びその制御方法
JP6668967B2 (ja) 画像処理装置、撮像装置、画像処理方法及び画像処理プログラム
WO2021102878A1 (fr) Procédé et dispositif permettant de configurer un dispositif de photographie
JP2013211930A (ja) 電子カメラ、撮像制御プログラム及び撮像制御方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21938452

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21938452

Country of ref document: EP

Kind code of ref document: A1