CN116545572A - Original image data acquisition system based on FPGA - Google Patents
Original image data acquisition system based on FPGA Download PDFInfo
- Publication number
- CN116545572A CN116545572A CN202310795516.9A CN202310795516A CN116545572A CN 116545572 A CN116545572 A CN 116545572A CN 202310795516 A CN202310795516 A CN 202310795516A CN 116545572 A CN116545572 A CN 116545572A
- Authority
- CN
- China
- Prior art keywords
- data acquisition
- acquisition unit
- clock
- image data
- master clock
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 23
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000005259 measurement Methods 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 239000013307 optical fiber Substances 0.000 claims description 5
- 238000000034 method Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 108700009949 PTP protocol Proteins 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/665—Control of cameras or camera modules involving internal camera communication with the image sensor, e.g. synchronising or multiplexing SSIS control signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0658—Clock or time synchronisation among packet nodes
- H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
- H04J3/0667—Bidirectional timestamps, e.g. NTP or PTP for compensation of clock drift and for compensation of propagation delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/22—Adaptations for optical transmission
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Synchronisation In Digital Transmission Systems (AREA)
Abstract
The application discloses an original image data acquisition system based on FPGA. Wherein the system comprises: a plurality of original image data acquisition units based on FPGA; the terminals are connected with the original image data acquisition units; wherein, the several original image data acquisition units specifically include: the data receiving module is used for acquiring at least two original image groups shot by the shooting equipment; the data processing module is used for carrying out coding processing on the images and generating a processed final image group; the data sending module is used for sending the final image group to a terminal; wherein, the original image data acquisition unit further includes: a time synchronization module; the terminal controls the time synchronization modules in the original image data acquisition units to realize time synchronization. By the original image data acquisition system, the time-synchronous multichannel original image data can be acquired in real time.
Description
Technical Field
The invention belongs to the field of intelligent driving data acquisition, and particularly relates to an original image data acquisition system based on an FPGA.
Background
With the continuous development of intelligent driving technology, a multichannel data acquisition system plays an increasingly important role in an automatic driving system. The multichannel data acquisition system also comprises acquisition and processing of multichannel video images, and the video image acquisition and processing technology is a key for realizing efficient and accurate data acquisition. The development of video image acquisition and processing techniques is mainly divided into two categories: based on a PC (personal computer) and on the basis of a related specific PCIe (peripheral component interconnect express) acquisition board card, processing a video image through software; and secondly, the video image is acquired and processed by using related integrated hardware such as DSP, MCU, FPGA and the like.
In implementing the prior art, the inventors found that:
in the past, data acquisition generally depends on independent time service of a system clock, although the time service accuracy of independent equipment can be higher (in general, millisecond level is supported), for a multi-channel acquisition system, the single acquisition card is required to have higher accuracy, and uniform time service of the acquisition system is required to be ensured, otherwise, the multi-channel acquisition system has low time synchronization accuracy, the requirement of intelligent driving on the data synchronization accuracy cannot be met, and data acquired by each sensor are inconsistent, so that an automatic driving system makes wrong decisions, and the safety of vehicles and passengers is even endangered.
Therefore, it is necessary to provide a multi-channel video image acquisition and processing system with higher time synchronization, so as to solve the problem that in the prior art, the error decision is made by the autopilot system due to inconsistent data acquired by the sensor caused by low time synchronization precision.
Disclosure of Invention
The embodiment of the application provides an FPGA-based original image data acquisition system which is used for solving the problem that in the prior art, due to low time synchronization precision, data acquired by a sensor are inconsistent, so that an automatic driving system makes an erroneous decision.
The application provides an original image data acquisition system based on FPGA, which comprises:
a plurality of original image data acquisition units based on FPGA;
the terminals are connected with the original image data acquisition units;
wherein, the several original image data acquisition units specifically include:
the data receiving module is used for acquiring at least two original image groups shot by the shooting equipment;
the data processing module is used for carrying out coding processing on the images and generating a processed final image group;
the data sending module is used for sending the final image group to a terminal;
wherein, the original image data acquisition unit further includes: a time synchronization module;
the terminal controls the time synchronization modules in the original image data acquisition units to realize time synchronization.
Further, in a preferred embodiment provided in the present application, the controlling the time synchronization module in the plurality of original image data acquisition units to achieve time synchronization is based on the gPTP protocol, and specifically includes:
the master clock selection module is used for selecting one of the plurality of original image data acquisition units to be a master clock of the data acquisition unit;
the frequency deviation calculation module is used for calculating the frequency deviation between the frequency of the master clock of the data acquisition unit and the frequency deviation between the slave clocks of the data acquisition unit;
the delay time measuring module is used for measuring the delay time between the data acquisition unit master clock and the data acquisition unit slave clock according to the frequency deviation:
the synchronous time module is used for synchronizing the time of the master clock of the data acquisition unit and the slave clock of the data acquisition unit according to the frequency deviation calculation module and the delay time measurement module;
the secondary clock of the data acquisition unit is used as a primary clock selection module to select the rest original image data acquisition units of the primary clock of the data acquisition unit from a plurality of original image data acquisition units.
Further, in a preferred embodiment provided in the present application, the master clock selection module selects according to an Announce message of a plurality of raw image data acquisition units based on FPGA;
the Announce message at least comprises: maximum delay value, bandwidth value, jitter value.
Further, in a preferred embodiment provided in the present application, calculating a frequency deviation between the frequency of the master clock of the data acquisition unit and the slave clock of the data acquisition unit specifically includes:
wherein R represents frequency deviation, T1 represents a time stamp of a first Sync message sent by a main clock of a data acquisition unit, T3 represents a time stamp of a second Sync message sent by the main clock of the data acquisition unit, T2 represents a time stamp of the first Sync message received by the main clock of the data acquisition unit, T4 represents a time stamp of the second Sync message received by the main clock of the data acquisition unit, T1 and T3 are based on the main clock of the data acquisition unit, T2 and T4 are based on a local clock of the main clock of the data acquisition unit, and the main clock of the data acquisition unit also tracks and sends a follow-up message when each Sync message is sent, and the main clock of the data acquisition unit is used for acquiring the time stamp of each Sync message sent by the main clock of the data acquisition unit.
Further, in a preferred embodiment provided in the present application, the delay time between the master clock of the data acquisition unit and the slave clock of the data acquisition unit is measured according to a frequency deviation, which specifically includes:
wherein ,t5 represents the time stamp of the data acquisition unit from the clock to send the PDelay_Req message to the data acquisition unit main clock, T6 represents the time stamp of the data acquisition unit main clock to receive the PDelay_Req message, T7 represents the time stamp of the data acquisition unit main clock to send the PDelay_Resp message to the data acquisition unit from the clock, T8 represents the time stamp of the data acquisition unit from the clock to receive the PDelay_Resp message, and the data acquisition unit main clock is used for receiving the PDelay_Req message simultaneously when the data acquisition unit main clock is used for sending the PDelay_Resp message to the time stamp of the data acquisition unit from the clockThe time stamp T6 is sent to the slave clock of the data acquisition unit, after the master clock of the data acquisition unit sends a PDelay_Resp message to the time stamp of the slave clock of the data acquisition unit, the master clock of the data acquisition unit also sends a PDelay_Resp_Following message to the slave clock of the data acquisition unit, and when the master clock of the data acquisition unit sends a PDelay_Resp_Following message to the slave clock of the data acquisition unit, the master clock of the data acquisition unit sends a PDelay_Resp message to the time stamp T7 of the slave clock of the data acquisition unit to the slave clock of the data acquisition unit.
Further, in a preferred embodiment provided in the present application, according to the frequency deviation calculating module and the delay time measuring module, synchronizing the time of the master clock of the data acquisition unit and the slave clock of the data acquisition unit specifically includes:
wherein Ta is a local time stamp on the master clock of the data acquisition unit, tb is the slave clock of the data acquisition unit,and the delay time is represented, T1 represents the time stamp of the first Sync message sent by the main clock of the data acquisition unit, and T2 represents the time stamp of the first Sync message received by the slave clock of the data acquisition unit.
Further, in a preferred embodiment provided in the present application, the data sending module is configured to send the final image group to a terminal, and specifically includes:
the data transmission module transmits the final image group to the terminal through an RDMA channel;
the data sending module at least comprises one of a 10G SFP optical fiber interface and a 1Gbps Ethernet interface.
The technical scheme provided by the embodiment of the application has at least the following beneficial effects: according to the FPGA-based original image data acquisition system, the original image data of multiple channels with synchronous time can be acquired in real time. In addition, the RDMA channel is used as an image high-speed output interface based on an optical fiber interface, so that the video image acquisition system provided by the invention has the capability of carrying out real-time multichannel synchronous high-speed image output.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
fig. 1 is a schematic structural diagram of an original image data acquisition system based on an FPGA according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of an FPGA-based raw image data acquisition unit according to an embodiment of the present application;
fig. 3 is a schematic diagram of an internal structure of an FPGA-based raw image data acquisition unit according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of implementing multi-channel data time synchronization by using several raw image data acquisition units based on FPGA according to an embodiment of the present application.
Fig. 5 is a schematic diagram of data transmission of an FPGA-based raw image data acquisition unit through an RDMA channel according to an embodiment of the present application.
Fig. 6 is a schematic calculation diagram of a frequency deviation calculation module according to an embodiment of the present application.
Fig. 7 is a schematic calculation diagram of a delay time measurement module according to an embodiment of the present application.
Fig. 8 is a schematic calculation diagram of a synchronous time module according to an embodiment of the present application.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Referring to fig. 1, an FPGA-based original image data acquisition system provided in an embodiment of the present application specifically includes: a plurality of original image data acquisition units based on FPGA; the terminals are connected with the original image data acquisition units; wherein, the several original image data acquisition units specifically include: the data receiving module is used for acquiring at least two original image groups shot by the shooting equipment; the data processing module is used for carrying out coding processing on the images and generating a processed final image group; the data sending module is used for sending the final image group to a terminal; wherein, the original image data acquisition unit further includes: a time synchronization module; the terminal controls the time synchronization modules in the original image data acquisition units to realize time synchronization.
Specifically, the original image data acquisition unit based on the FPGA can be a high-speed data acquisition and parallel control board card based on the FPGA, and the board card uses XCZU5EV-2SFVC784I of Xilinx company as a calculation processing unit. The original image data acquisition unit based on the FPGA comprises: the device comprises a data receiving module, a data processing module, a time synchronization module and a data sending module. As shown in FIG. 2, the data receiving module is connected with the 1-2 channel camera through the FAKRA interface and takes the mode of LVDS differential serial signal transmission as data input, and the serial signal transmitted by the camera is received through the serial-parallel circuit and converted into a parallel signal to be sent to the data processing module of the FPGA. After the data processing module receives the collected original image data, the data can be processed according to the need, for example, the VCU is used for encoding the RAW image into YUV format, the YUV image is encoded into H264 format, and various processes such as ISP processing can be performed on the image. The time synchronization module uses the gPTP protocol (or PTP protocol) to realize microsecond-level time synchronization through a hardware clock, and meets the requirement of intelligent driving multi-sensor data time synchronization. The data sending module comprises 2 10G SFP optical fiber interfaces and 2 1Gbps Ethernet interfaces, supports high-speed transmission of data to NAS storage equipment or other storage equipment through RDMA technology, and is also used for communication with an upper computer or a plurality of acquisition cards.
The terminal connected with the plurality of original image data acquisition units can be understood as NAS storage equipment or other storage equipment and host computer equipment.
It should be further noted that, as shown in fig. 3, an internal structure diagram of an original image data acquisition unit based on an FPGA is shown, and inside the FPGA, there are mainly 3 main modules, namely, a network module ethernet (ethernet), a data codec module VCU, and a data capture processing module capture pipeline. The AXI main line comprises a multiplexer aximux, a Timer module axitimer and a general purpose input/output controller module axigpio. One programmable logic device Zynq, clock-divided IP cores clk_ wiz, axi interconnect.
All data uploading and downloading are carried out through an axi main line, a high-level clock is triggered, a mipi unit in a capture pipeline module takes original data and transmits the original data to a converter unit for transcoding, then short packet data integration writing is carried out, and the writing into the memory has the advantages of avoiding frequent interrupt sending to Soc and avoiding damage to a certain extent. The data in the memory is placed in the axi bus through the zynq core board, the code conversion from RAW data to YUV data is carried out through the VCU, and compared with the traditional software coding, the speed can be greatly improved by utilizing the FPGA to carry out H264 coding.
Furthermore, the time synchronization module in the original image data acquisition units is controlled to realize time synchronization based on the gPTP protocol, and the method specifically comprises the following steps: the master clock selection module is used for selecting one of the plurality of original image data acquisition units to be a master clock of the data acquisition unit; the frequency deviation calculation module is used for calculating the frequency deviation between the frequency of the master clock of the data acquisition unit and the frequency deviation between the slave clocks of the data acquisition unit; the delay time measuring module is used for measuring the delay time between the data acquisition unit master clock and the data acquisition unit slave clock according to the frequency deviation: the time synchronization module is used for synchronizing the time of the master clock of the data acquisition unit and the slave clock of the data acquisition unit according to the frequency deviation calculation module and the delay time measurement module; the secondary clock of the data acquisition unit is used as a primary clock selection module to select the rest original image data acquisition units of the primary clock of the data acquisition unit from a plurality of original image data acquisition units.
Specifically, as shown in fig. 4, the present invention realizes data synchronization based on the gPTP protocol (or PTP protocol). The traditional time synchronization generally depends on independent time service of a system clock, and although the time service accuracy of the independent equipment can be higher (generally supporting millisecond level), for a multi-channel acquisition system, not only is the accuracy required by a single acquisition card higher, but also the uniform time service of the acquisition system needs to be ensured, and the uniform high-accuracy time synchronization is realized. The purpose of gPTP is to synchronize the clocks of the various acquisition nodes (in the present invention, the original image data acquisition units) in the vehicle network. In the system, a gPTP time service device is used as a main clock of a data acquisition unit (namely gPTP Master in fig. 4), and the main clock of the data acquisition unit automatically acquires the current time from a GPS or Beidou as a reference. The remaining data acquisition unit board card is used as a data acquisition unit Slave clock (namely, slave1 and Slave2 … … in fig. 4), and the local clock of the data acquisition unit Slave clock and the clock of the data acquisition unit master clock are synchronized through the gPTP protocol. Switch is used for network devices for packet forwarding.
Further, the master clock selection module selects according to Announce messages of a plurality of original image data acquisition units based on the FPGA; the Announce message at least comprises: maximum delay value, bandwidth value, jitter value.
Specifically, the master clock selection module selects based on the Best Master Clock Algorithm (BMCA). An optimal master clock algorithm (BMCA) selects a data acquisition unit master clock through an Announce message of a plurality of original image data acquisition units based on the FPGA. The Announce message includes: maximum delay value, bandwidth value, jitter value, and other relevant field values.
It can be understood that the vehicle is a closed network, and the situation of total change of the nodes does not occur, so in the vehicle-mounted application, a time service device is used as a main clock of the data acquisition unit.
Further, calculating the frequency deviation between the frequency of the master clock of the data acquisition unit and the frequency of the slave clock of the data acquisition unit specifically includes:
wherein R represents the clock frequency of a data acquisition unit slave clock and the clock frequency of a master clock, T1 represents the time stamp of a first Sync message sent by the data acquisition unit master clock, T3 represents the time stamp of a second Sync message sent by the data acquisition unit master clock, T2 represents the time stamp of the first Sync message received by the data acquisition unit slave clock, T4 represents the time stamp of the second Sync message received by the data acquisition unit slave clock, T1 and T3 are based on the data acquisition unit master clock, T2 and T4 are based on the local clock of the data acquisition unit slave clock, the data acquisition unit master clock also tracks and sends a follow-up message when each Sync message is sent, and the data acquisition unit slave clock is used for acquiring the time stamp of each Sync message sent by the data acquisition unit master clock.
Specifically, the data acquisition unit master clock will continually send Sync and Follow-up, and the data acquisition unit slave clock can measure the time since the Sync message was received, i.e., T2 and T4 in fig. 6, based on the local clock of the data acquisition unit slave clock. After each sync message, the data acquisition unit master clock sends a follow-up message to tell the data acquisition unit slave clock the timestamp of when itself sent the previous sync message, where the local clock of the data acquisition unit master clock is used as a reference. Without a follow-up message, the data acquisition unit cannot actually send the Sync message from the clock, i.e. T1 and T3 in the upper graph. With T1, T3 and T2, T4, the data acquisition unit slave clock is able to calculate the deviation between the own clock frequency and the data acquisition unit master clock.
Further delay time measurement, the delay time between the data acquisition unit master clock and the data acquisition unit slave clock is measured according to the frequency deviation, and the method specifically comprises the following steps:
wherein ,the delay time is represented by T5, the data acquisition unit transmits a PDelay_Req message to a data acquisition unit master clock time stamp, T6, the data acquisition unit master clock receives a time stamp of the PDelay_Req message, T7 represents a time stamp of the data acquisition unit master clock transmitting the PDelay_Resp message to the data acquisition unit slave clock, T8 represents a time stamp of the data acquisition unit slave clock receiving the PDelay_Resp message, when the data acquisition unit master clock transmits the time stamp of the PDelay_Resp message to the data acquisition unit slave clock, the data acquisition unit master clock transmits the time stamp T6 of the data acquisition unit master clock receiving the PDelay_Req message to the data acquisition unit slave clock, and after the data acquisition unit master clock transmits the time stamp of the PDelay_Resp message to the data acquisition unit slave clock, the data acquisition unit master clock also transmits the PDelay_Resp_Foow message to the data acquisition unit slave clock, and the data acquisition unit master clock transmits the PDelay_Resup_Resp message to the data acquisition unit slave clock, and the data acquisition unit master clock transmits the data acquisition unit from the data acquisition unit master clock to the data acquisition unit slave clock.
Specifically, in order to synchronize the own clock with the data acquisition unit master clock, the data acquisition unit slave clock must measure the delay time of the message transfer between itself and the data acquisition unit master clock. The delay measurement uses three gpp messages, namely pdelay_req, pdelay_resp and pdelay_resp_follow. As shown in fig. 7, a specific measurement method is: 1. firstly, a data acquisition unit sends a PDelay_Req message from a clock to request measurement of delay time; when the PDelay_Req message leaves the physical layer, the data acquisition unit obtains a time stamp T5 from the clock by using a local clock; the PDelay_Req message reaches the physical layer of the responder, and the master clock of the data acquisition unit obtains a time stamp T6 by using the local clock; 4. the data acquisition unit master clock generates a PDelay_Resp message and sends the PDelay_Resp message, and the T6 timestamp is brought to the data acquisition unit slave clock; 5. the data acquisition unit slave clock can capture a time stamp T8 when the PDelay_Resp is received by using the local clock; 6. the data acquisition unit master clock then sends a pdelay_resp_followup message to bring the time stamp T7 to the data acquisition unit slave clock. And the timestamp T7 is a timestamp of a data acquisition unit master clock sending a PDelay_Resp message to a data acquisition unit slave clock.
Further time synchronization, the synchronization time module synchronizes the time of the master clock of the data acquisition unit and the slave clock of the data acquisition unit according to the frequency deviation calculation module and the delay time measurement module, and specifically comprises:
wherein Ta is a local time stamp on a main clock of the data acquisition unit, tb is a local time stamp of the data acquisition unit at a certain moment,and the delay time is represented, T1 represents the time stamp of the first Sync message sent by the main clock of the data acquisition unit, and T2 represents the time stamp of the first Sync message received by the slave clock of the data acquisition unit.
Specifically, as shown in fig. 8, after the delay time and the frequency difference are calculated according to the above, the slave clock of the data acquisition unit synchronizes its own clock with the master clock of the data acquisition unit by using sync and follow_up messages continuously sent by the master clock of the data acquisition unit.
Further, the data sending module is configured to send the final image group to a terminal, and specifically includes: the data transmission module transmits the final image group to the terminal through an RDMA channel; the data sending module at least comprises one of a 10G SFP optical fiber interface and a 1Gbps Ethernet interface.
As shown in fig. 5, the data sending module of the original image data acquisition unit based on the FPGA is used for sending the final image group to the terminal. The original image data acquisition unit based on the FPGA specifically is an original image data acquisition card based on the FPGA, and comprises: the device comprises a central processing unit CPU, a Buffer area Buffer1 and a network interface controller Nic. The data storage terminal also includes: the CPU, buffer zone Buffer2 and network interface controller Nic are the above-mentioned FPGA-based original image data acquisition card and data storage terminal, and the network interface controller adopts RDMA technology. The original image data acquisition unit based on the FPGA can directly carry out data transmission between the RDMA technology and the terminal without processing by other processing components.
It can be appreciated that in the process of processing a data packet, the conventional TCP/IP technology needs to occupy a large amount of server resources and memory bus bandwidth through an operating system and other software layers, and data is copied and moved back and forth among a system memory, a processor cache and a network controller cache, which causes a heavy burden to a CPU and a memory of a server. Especially, the serious "mismatch" of network bandwidth, processor speed and memory bandwidth exacerbates the network delay effect. RDMA is a host-offflow technology that enables applications (including storage) to transfer data directly between their memory spaces. The board card collects the original RAW image of the high-resolution camera, the bandwidth of the image is up to 2GB/s per channel, and the high-performance remote direct data access target is achieved by solidifying the RDMA protocol on hardware (namely a network card) and supporting two approaches of Zero-copy and Kernel bypass.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (7)
1. An FPGA-based raw image data acquisition system, comprising:
a plurality of original image data acquisition units based on FPGA;
the terminals are connected with the original image data acquisition units;
wherein, the several original image data acquisition units specifically include:
the data receiving module is used for acquiring at least two original image groups shot by the shooting equipment;
the data processing module is used for carrying out coding processing on the images and generating a processed final image group;
the data sending module is used for sending the final image group to a terminal;
wherein, the original image data acquisition unit further includes: a time synchronization module;
the terminal controls the time synchronization modules in the original image data acquisition units to realize time synchronization.
2. The FPGA-based raw image data acquisition system of claim 1, wherein the controlling the time synchronization module in the plurality of raw image data acquisition units to achieve time synchronization is based on the gPTP protocol, specifically includes:
the master clock selection module is used for selecting one of the plurality of original image data acquisition units to be a master clock of the data acquisition unit;
the frequency deviation calculation module is used for calculating the frequency deviation between the frequency of the master clock of the data acquisition unit and the frequency deviation between the slave clocks of the data acquisition unit;
the delay time measuring module is used for measuring the delay time between the data acquisition unit master clock and the data acquisition unit slave clock according to the frequency deviation:
the synchronous time module is used for synchronizing the time of the master clock of the data acquisition unit and the slave clock of the data acquisition unit according to the frequency deviation calculation module and the delay time measurement module;
the secondary clock of the data acquisition unit is used as a primary clock selection module to select the rest original image data acquisition units of the primary clock of the data acquisition unit from a plurality of original image data acquisition units.
3. The FPGA-based raw image data acquisition system of claim 2, wherein the master clock selection module selects according to an Announce message of a plurality of FPGA-based raw image data acquisition units;
the Announce message at least comprises: maximum delay value, bandwidth value, jitter value.
4. The FPGA-based raw image data acquisition system of claim 2, wherein calculating the frequency deviation between the frequency of the data acquisition unit master clock and the data acquisition unit slave clock, comprises:
；
wherein, R represents a frequency deviation, T1 represents a timestamp of a first Sync message sent by a master clock of a data acquisition unit, T3 represents a timestamp of a second Sync message sent by the master clock of the data acquisition unit, T2 represents a timestamp of the first Sync message received by the slave clock of the data acquisition unit, T4 represents a timestamp T1 and T3 of the second Sync message received by the slave clock of the data acquisition unit, and T2 and T4 are based on the master clock of the data acquisition unit and the local clock of the slave clock of the data acquisition unit, and the master clock of the data acquisition unit also tracks and sends a follow-up message when sending each Sync message, and acquires the timestamp of each Sync message sent by the master clock of the data acquisition unit from the slave clock of the data acquisition unit.
5. The FPGA-based raw image data acquisition system of claim 4, wherein the delay time between the data acquisition unit master clock and the data acquisition unit slave clock is measured according to a frequency deviation, specifically comprising:
；
wherein ,the delay time is represented by T5, T6, T7, T8, T6 and T6 respectively represent that the data acquisition unit master clock transmits the PDelay_Resp message to the time stamp of the data acquisition unit slave clock, the data acquisition unit master clock transmits the time stamp of the PDelay_Resp message to the time stamp of the data acquisition unit slave clock, the time stamp T6 of the data acquisition unit master clock receiving the PDelay_Req message to the time stamp of the data acquisition unit slave clock is transmitted to the data acquisition unit slave clock, the data acquisition unit master clock transmits the time stamp of the PDelay_Resp message to the time stamp of the data acquisition unit slave clock, the data acquisition unit master clock also transmits the PDelay_Resp_Foow message to the data acquisition unit slave clock after the data acquisition unit master clock transmits the time stamp of the PDelay_Resp message to the data acquisition unit slave clock, and the data acquisition unit master clock transmits the PDelay_Resp_Foow_Foow message to the data acquisition unit slave clockAnd sending a PDelay_Resp message to a time stamp T7 of a data acquisition unit slave clock by the data acquisition unit master clock to the data acquisition unit slave clock.
6. The FPGA-based raw image data acquisition system of claim 2, wherein synchronizing the times of the data acquisition unit master clock and data acquisition unit slave clock according to the frequency deviation calculation module and the delay time measurement module, specifically comprises:
；
wherein Ta is a local time stamp on the master clock of the data acquisition unit, tb is the slave clock of the data acquisition unit,and the delay time is represented, T1 represents the time stamp of the first Sync message sent by the main clock of the data acquisition unit, and T2 represents the time stamp of the first Sync message received by the slave clock of the data acquisition unit.
7. The FPGA-based raw image data acquisition system of claim 1, wherein the data transmission module is configured to transmit the final image group to a terminal, and specifically comprises:
the data transmission module transmits the final image group to the terminal through an RDMA channel;
the data sending module at least comprises one of a 10G SFP optical fiber interface and a 1Gbps Ethernet interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310795516.9A CN116545572A (en) | 2023-06-30 | 2023-06-30 | Original image data acquisition system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310795516.9A CN116545572A (en) | 2023-06-30 | 2023-06-30 | Original image data acquisition system based on FPGA |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116545572A true CN116545572A (en) | 2023-08-04 |
Family
ID=87449117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310795516.9A Pending CN116545572A (en) | 2023-06-30 | 2023-06-30 | Original image data acquisition system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116545572A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480350A (en) * | 2010-11-29 | 2012-05-30 | 北京中和威软件有限公司 | Method and device for time synchronization and maintaining based on frequency deviation estimation |
| CN109167667A (en) * | 2018-09-18 | 2019-01-08 | 武汉海晟科讯科技有限公司 | A kind of sensor data acquisition Transmission system and method based on PTP synchronization |
| CN111277804A (en) * | 2020-03-10 | 2020-06-12 | 北京爱笔科技有限公司 | Image processing method and device and multi-camera synchronization system |
| CN114615736A (en) * | 2022-04-18 | 2022-06-10 | 武汉华中天纬测控有限公司 | Time synchronization network device and method |
| CN115865246A (en) * | 2021-09-23 | 2023-03-28 | 北京车和家信息技术有限公司 | Time synchronization device, system and method |
-
2023
- 2023-06-30 CN CN202310795516.9A patent/CN116545572A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480350A (en) * | 2010-11-29 | 2012-05-30 | 北京中和威软件有限公司 | Method and device for time synchronization and maintaining based on frequency deviation estimation |
| CN109167667A (en) * | 2018-09-18 | 2019-01-08 | 武汉海晟科讯科技有限公司 | A kind of sensor data acquisition Transmission system and method based on PTP synchronization |
| CN111277804A (en) * | 2020-03-10 | 2020-06-12 | 北京爱笔科技有限公司 | Image processing method and device and multi-camera synchronization system |
| CN115865246A (en) * | 2021-09-23 | 2023-03-28 | 北京车和家信息技术有限公司 | Time synchronization device, system and method |
| CN114615736A (en) * | 2022-04-18 | 2022-06-10 | 武汉华中天纬测控有限公司 | Time synchronization network device and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105577308B (en) | The method and apparatus of In-vehicle networking time synchronization is provided using redundancy master clock | |
| CN105320632A (en) | Implementation method for high-speed differential bus of autonomous arbitration | |
| JPH0373636A (en) | Data synchronizing transmission system | |
| US8126019B2 (en) | Method for time synchronization in a cyclically operating communication system | |
| WO2020021309A1 (en) | Synchronizing tsn master clocks in wireless networks | |
| WO2015196685A1 (en) | Clock synchronization method and apparatus | |
| JP6449430B2 (en) | Time synchronization method, device and time synchronization server for network device | |
| CN109996325B (en) | Clock synchronization system and method of wireless sensor network | |
| US20070257786A1 (en) | Sequencing multi-source messages for delivery as partial sets to multiple destinations | |
| CN103616591A (en) | Simulation device and method of characters of merging unit of intelligent transformer substation | |
| CN102916758B (en) | Ethernet time synchronism apparatus and the network equipment | |
| WO2012119385A1 (en) | Method, device and system for performing time synchronization on pcie device | |
| CN115296764A (en) | Timestamp confidence level | |
| JP5780157B2 (en) | Computer, parallel computer system, synchronization method, and computer program | |
| CN111010250B (en) | Method and system for realizing high-precision time synchronization | |
| CN116545572A (en) | Original image data acquisition system based on FPGA | |
| US9485083B2 (en) | Method and apparatus for time synchronization between nodes | |
| US20220329336A1 (en) | Full-duplex communication link using TDM | |
| CN112583508A (en) | Clock synchronization communication system and method for rail train | |
| CN112511255B (en) | Time synchronization method and device | |
| CN103236894A (en) | Clock synchronization method and device in seamless redundant network | |
| KR20120051632A (en) | Method for clock synchronization in distributed system having ring topology and apparatus for the same | |
| CN102387155B (en) | Delay-constant PTP (peer to peer) signal protocol conversion system and implementation method thereof | |
| US10122486B2 (en) | Method for clock synchronization between two devices connected by an ethernet wire connection | |
| CN104348692A (en) | Synchronous concurrent inter-board communication method of data acquiring and control equipment based on Ethernet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |