CN116577802A - Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ - Google Patents

Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ Download PDF

Info

Publication number
CN116577802A
CN116577802A CN202310524952.2A CN202310524952A CN116577802A CN 116577802 A CN116577802 A CN 116577802A CN 202310524952 A CN202310524952 A CN 202310524952A CN 116577802 A CN116577802 A CN 116577802A
Authority
CN
China
Prior art keywords
data
trigger signal
duration
zynq
laser radar
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310524952.2A
Other languages
Chinese (zh)
Inventor
谷延锋
唐晨煦
李贤�
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN202310524952.2A priority Critical patent/CN116577802A/en
Publication of CN116577802A publication Critical patent/CN116577802A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1652Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with ranging devices, e.g. LIDAR or RADAR
    • GPHYSICS
    • G04HOROLOGY
    • G04RRADIO-CONTROLLED TIME-PIECES
    • G04R20/00Setting the time according to the time information carried or implied by the radio signal
    • G04R20/02Setting the time according to the time information carried or implied by the radio signal the radio signal being sent by a satellite, e.g. GPS
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ belongs to unmanned aerial vehicle platform remote sensing technical field. The system aims at solving the problems that the multispectral camera and the laser radar carried by the unmanned aerial vehicle cannot be synchronously acquired or the synchronous error is large in the existing system. The invention designs a continuous time synchronous trigger signal generator, when the millisecond timing set in the ZYNQ development board counts every one second or when the inertial navigation board card receives the second pulse rising edge of the satellite GPS signal, the continuous time synchronous trigger signal is output by the continuous time pulse generator, and meanwhile, the millisecond timer is emptied and the cycle is repeated; and recording the difference value between the one-second duration counted by the millisecond timer and the real one-second duration of the UTC time while generating the duration synchronization trigger signal, and obtaining a correction error and correcting the millisecond timer based on the difference value. And carrying out time duration synchronization on the multispectral camera, the laser radar and the inertial navigation board card based on the duration synchronization trigger signal.

Description

Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ
Technical Field
The invention belongs to the technical field of unmanned aerial vehicle platform remote sensing, and relates to an unmanned aerial vehicle platform multi-sensor data acquisition and time synchronization system.
Background
With the rapid development of unmanned aerial vehicle platform remote sensing technology, a laser radar technology and a multispectral remote sensing technology are widely applied to unmanned aerial vehicle platforms respectively. The laser radar technology has the characteristic of rapid and fine acquisition of three-dimensional information of a target space, but has the problem of very limited acquisition capacity of spectrum information. The multispectral remote sensing technology can acquire the spectrum information of the richer spectrum of the target, but has the problems of lower spatial resolution and poorer sensitivity. Although single laser radar technology or multispectral remote sensing technology has outstanding unilateral advantages, three-dimensional-spectrum information of a target space cannot be synchronously acquired, and the requirement of fine application is difficult to meet.
The multispectral laser radar integrated detection technology combines the laser radar technology with the multispectral remote sensing technology, gives full play to the respective advantages of the two technologies, can acquire the spatial three-dimensional information and the spectral information of the target, and has great application potential in various subjects and fields. However, by simply combining the laser radar technology and the multispectral remote sensing technology, the problem that the time synchronization acquisition of the space three-dimensional information and the spectrum information or the time synchronization error is larger cannot be realized, the practical application of the system is seriously affected, and the research of an integrated acquisition storage and time synchronization system based on the laser radar and the multispectral camera is needed currently.
Disclosure of Invention
The invention aims to solve the problems that the laser radar, the multispectral camera and the like carried by the unmanned aerial vehicle cannot be synchronously acquired or the synchronous error is larger in the existing system.
The unmanned aerial vehicle platform multi-sensor data synchronous acquisition system based on ZYNQ comprises a time synchronous subsystem;
the synchronization process of the time synchronization subsystem comprises the following steps:
synchronously stamping the original data of the multispectral camera image and the laser radar with the same time stamp based on the inertial navigation board card, outputting pose data through a serial communication port of the inertial navigation board card to provide position pose information, and outputting the information through a serial port after passing through a conversion circuit; the other serial communication port of the inertial navigation board outputs recommended positioning information;
a duration synchronous trigger signal generator is designed in the ZYNQ development board and is used for outputting a duration synchronous trigger signal; when the millisecond timing set in the ZYNQ development board is full of one second or when the inertial navigation board receives the second pulse rising edge of the satellite GPS signal, the duration synchronization trigger signal generator outputs a high level as the duration synchronization trigger signal, and simultaneously clears the millisecond timer, and the cycle is repeated;
when the inertial navigation board card can normally receive the second pulse from the satellite GPS signal, the rising edge moment of the second pulse of the received GPS signal immediately empties the millisecond timer, and the duration synchronous trigger signal generator outputs a high level, namely a new duration synchronous trigger signal; the method comprises the steps that when an inertial navigation board receives satellite GPS signals and a second pulse output by the inertial navigation board generates a duration synchronous trigger signal, the difference value between one second duration counted by a millisecond timer and one second duration with real UTC time is recorded, the difference value is recorded, the first N difference values at the current moment are subjected to weighted average processing according to weights, and the difference value is recorded as a correction error e; when the inertial navigation board card cannot normally receive the second pulse from the satellite GPS signal, the millisecond timer is emptied when the millisecond timer is full of one second, the millisecond timer is corrected by using correction errors e corresponding to the first N differences of the second pulse of the satellite GPS signal which is not received, and a new duration synchronous trigger signal is output by the duration synchronous trigger signal generator;
and recognizing the newly generated duration synchronization trigger signal as second pulse information in the time synchronization process, and utilizing the duration synchronization trigger signal output by the duration synchronization trigger signal generator to perform time duration synchronization of the multispectral camera and the laser radar.
Further, the system also comprises an acquisition and storage subsystem, wherein the acquisition and storage subsystem comprises a data storage module and a data acquisition module; the data storage module comprises a FLASH memory, an online cache unit, a storage controller and an external storage unit;
the FLASH memory is used for storing an operating system, a file system and related configuration data of the multi-sensor data acquisition and time synchronization system;
the online caching unit is used for storing temporary caches of output data of the multispectral camera and the laser radar, and temporary storage positions of stack allocation and related variables of a ZYNQ processor system of the ZYNQ development board in the running process;
the storage controller controls address signals, data information of the multiple sensors and various command signals in the data storage process, and controls conversion of a data interface between the ZYNQ development board and the multiple sensors and communication of read-write commands; the multi-sensor comprises a multi-spectrum camera, a laser radar and an inertial navigation board card;
the external storage unit is used for offline storing online cache of multi-sensor data and multi-sensor output data, wherein the multi-sensor data comprises multi-spectral images, laser radar original data and pose data;
the data acquisition module is used for acquiring multi-sensor data;
the method comprises the steps that after the laser radar is powered on and works, the collected original data are divided into a data packet and a position packet, the data are transmitted to a ZYNQ development board from a port respectively, and the collected data are read and stored in an external storage unit through a ZYNQ processing system.
Further, the process of performing time duration synchronization of the lidar using the duration synchronization trigger signal output by the duration synchronization trigger signal generator is as follows:
the pose data aligned with the duration synchronous trigger signal and the recommended positioning information signal are stored into an external storage unit through a serial port of the ZYNQ development board processing system; and when the counter arranged in the laser radar receives the duration synchronous trigger signal and the recommended positioning information, emptying the millisecond timer according to the rising edge of the duration synchronous trigger signal, giving the time information of the second part in UTC time in the recommended positioning information to the second counter arranged in the laser radar, updating time data frame information in a laser radar position packet by using the time information including year, month, date and hour, realizing the alignment of the pulse rising edge of the duration synchronous trigger signal and the recommended positioning information and the laser emission time of the laser radar multi-line laser, and transmitting and storing the acquired laser radar original data into an external storage unit through a ZYNQ development board network port.
Further, the triggering process of the laser radar is as follows:
processing a time stamp in an original data frame of the laser radar aiming at the original data acquired by the laser radar, and obtaining specific time recorded in the time stamp of the data frame corresponding to each laser point by the laser device transmitting and receiving the obtained laser point in each transmitting period of the laser radar by interpolation calculation through a single linear interpolation algorithm in combination with a single echo/double echo mode set by the laser radar, so as to obtain specific time corresponding to UTC time provided by a GPS (global positioning system) for each laser point.
Further, the process of performing time duration synchronization of the multispectral camera by using the duration synchronization trigger signal output by the duration synchronization trigger signal generator is as follows:
and aligning the rising edge time of the continuous time synchronous trigger signal with the integral second time of UTC time, sending the continuous time synchronous trigger signal into a ZYNQ development board for level amplitude and cycle duty ratio adjustment, generating a second pulse trigger signal after level amplitude and cycle duty ratio adjustment, providing the second pulse trigger signal after level amplitude and cycle duty ratio adjustment for the multispectral camera to serve as a trigger signal for continuous photographing of the multispectral camera, and controlling the multispectral camera to acquire images.
Further, the level amplitude and cycle duty cycle adjustment process is as follows:
and adjusting the level amplitude and the period duty ratio of the duration synchronous trigger signal to a section with the high level amplitude of 3.3-5V and the duty ratio of 25-50%.
Further, the second pulse trigger signal with the level amplitude and the period duty ratio adjusted is provided for the multispectral camera, the second pulse trigger signal is used for controlling the multispectral camera to continuously shoot and collect the trigger signal of the image, the frequency of the trigger signal is changed through a preset frequency division circuit in the ZYNQ development board, and the sampling frequency is adjusted.
Further, laser radar original data acquired after the laser radar is powered on is transmitted to the ZYNQ development board through a user datagram protocol.
Further, in the process that data acquired after the laser radar is powered on and works is transmitted to the ZYNQ development board from the port based on the user datagram protocol, firstly, space initialization is carried out on a data acquisition part, a new space is opened up, and an empty new user datagram protocol data frame area is generated; then applying for the ports corresponding to the two sockets respectively, and respectively realizing the receiving and collecting tasks of the data packet and the position packet;
copying the received data to an online caching unit, traversing all linked lists of a data packet structure body, copying the data pointed by a pointer corresponding to the data packet structure body to a user datagram protocol data frame, and recording the length information received by the total data volume; then, data judgment operation is carried out: and identifying a user datagram protocol data header mark, extracting related information, realizing information conversion operation from a. Pcap format to a. Txt format, and finally writing the obtained information in the. Txt format into an external storage unit to finish the functions of receiving, converting and storing data.
Further, the system also comprises a multispectral camera and a laser radar which are respectively used for collecting multispectral images and laser radar original data and providing the multispectral images and the laser radar original data for the time synchronization subsystem and the collecting and storing subsystem.
The beneficial effects are that:
the ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system can be used for completing acquisition and storage of multispectral images, laser radar original data and inertial navigation board card pose data, time synchronization of the laser radar and the inertial navigation board card pose data and time synchronization of the laser radar and the multispectral camera, and can realize three-dimensional remote sensing detection with multidimensional space-time consistency. Advantages of the invention may be represented by the following:
(a) Realize the integration collection storage of multisensor data: the system designed by the invention performs integrated acquisition and storage on multispectral images acquired by the multispectral camera, the original point cloud acquired by the laser radar and pose data provided by the inertial navigation system, is connected with interfaces provided by the sensors through ZYNQ and is processed and stored in an external storage unit, and can meet the integrated acquisition of multi-sensor data in a multi-application scene.
(b) Microsecond time synchronization of multi-sensor data is achieved: the system designed by the invention aims at the high-precision time synchronization processing of data of a plurality of sensors such as a multispectral camera, a laser radar, an inertial navigation board card and the like in space remote sensing detection, can realize the space-time consistency of the data, meets the requirements on higher-precision space detection and three-dimensional reconstruction of ground object multidimensional information, fills the defect of related research in the field, and develops a set of usable unmanned plane platform multidimensional information detection system.
(c) The expansibility and the adjustability are good: since interfaces towards more sensors are reserved, more sensors, such as hyperspectral sensors, are added if needed. Furthermore, the multispectral image acquisition of different sampling frequencies can be realized by setting the frequency of the external trigger pulse signal provided for the multispectral camera, and the multispectral image acquisition device has better adjustability.
Drawings
Fig. 1 is an overall block diagram of a ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system
Fig. 2 is a test result of laser radar data and inertial navigation board pose data.
FIG. 3 is an oscilloscope observation of a pulse-per-second signal and a GPRMC signal provided by an inertial navigation system.
Fig. 4 shows the falling edge width of the second pulse signal and the trigger signal of the spectrum camera.
Fig. 5 is a schematic diagram of laser radar pulse per second locking.
Fig. 6 is a flowchart of generation of a duration synchronization trigger signal.
Fig. 7 is a flow chart of a millisecond timer real-time weighted error update algorithm.
FIG. 8 is a view of VLP-16 lidar entity.
Fig. 9 is a real view of the ZYNQ development board.
Fig. 10 is a diagram of a north cloud bynav A1 GNSS board card connection entity.
Detailed Description
The invention combines the advantages of the laser radar for acquiring the three-dimensional information of the target space and the multispectral camera for acquiring the rich spectral information, combines the two sensors together on the unmanned plane platform to play respective advantages, and realizes synchronous acquisition and storage of the three-dimensional information and the spectral information of the detection target space. The pulse-per-second signal provided by the inertial navigation board card is combined with pose information provided by the inertial navigation system to process the plurality of sensors, so that the time synchronization of microsecond level is kept while the laser radar, the multispectral camera and the inertial navigation system collect data, the space-time consistency of the data collection of the plurality of sensors is achieved, and more accurate data is provided for the multi-sensor remote sensing detection of the unmanned aerial vehicle platform. The following detailed description is made in connection with specific embodiments.
The first embodiment is as follows: the present embodiment will be described with reference to figure 1,
the embodiment is an unmanned aerial vehicle platform multi-sensor data synchronous acquisition system based on ZYNQ, which comprises a ZYNQ development board, a multispectral camera, a laser radar, an inertial navigation board card, an acquisition and storage subsystem and a time synchronization subsystem;
the acquisition and storage subsystem and the time synchronization subsystem are realized based on a ZYNQ development board and an inertial navigation board card;
(1) The acquisition and storage subsystem comprises a data storage module and a data acquisition module;
and (1.1) a data storage module which comprises a FLASH memory, an online caching unit, a storage controller and an external storage unit.
The FLASH memory is used for storing an operating system, a file system and related configuration data of the multi-sensor data acquisition and time synchronization system;
the online caching unit is used for storing temporary caches of output data of the multispectral camera and the laser radar, and temporary storage positions of stack allocation and related variables of a ZYNQ processor system (PS end) of the ZYNQ development board in the running process;
the memory controller controls address signals, data information and various command signals in the data storage process of the multi-sensor (multi-spectrum camera, laser radar and inertial navigation board card), and completes conversion of data interfaces between the core development board and the sensors and communication of read-write commands.
The external storage unit is a nonvolatile storage medium and is used for offline storage of acquired data, and an SD storage card is selected as the external storage unit and is used for storing multi-sensor data, wherein the multi-sensor data comprises multispectral images, laser radar original data and pose data.
The data acquisition module (1.2) is as follows:
the laser radar of the embodiment adopts a VLP-16 laser radar, a set of laser radar original data acquisition module based on ZYNQ-7000 is developed aiming at the characteristics and transmission mode of VLP-16 laser radar original data, a User Datagram Protocol (UDP) data transmission network protocol based on LWIP (lightweight TCP/IP protocol stack) is established, information acquired after the laser radar is powered on and works is divided into a data packet and a position packet, the data are transmitted to a ZYNQ development board through network cables from a port 2368 and a port 8308 respectively, and the acquired data are read through a ZYNQ-processing system and stored in an external storage unit. Data from the lidar is transmitted to the ZYNQ development board via the network cable via the user datagram protocol and stored in an external storage unit.
Based on the establishment of the underlying protocol, the data acquisition module is further perfected. The key to collecting data from the lidar via the user datagram protocol is the relative setting of the user datagram protocol receive callback function. Firstly, space initialization is carried out on a data acquisition part, a new space is opened up, the space initialization is carried out, and an empty new user datagram protocol data frame area is generated. Socket application and port connection are then performed: the two sockets are applied to correspond to ports 2368 and 8308 respectively, and the task of receiving and collecting the data packet and the position packet is realized respectively. And copying the received data to an online caching unit, traversing all linked lists of the data packet structure body, copying the data pointed by the pointer corresponding to the data packet structure body to a user datagram protocol data frame, and recording the length information received by the total data volume. Then, data judgment operation is carried out: and identifying a user datagram protocol data header mark, extracting related information, realizing information conversion operation from a. Pcap format to a. Txt format, and finally writing the obtained information in the. Txt format into an external storage unit to finish the functions of receiving, converting and storing data.
In some embodiments, the overall structure of the pcap file may be: file header-data packet header 1-data packet header 2-data packet 2; the data is composed of: the 1248 bytes are 42+12×100) +4+2, wherein the first 42 bytes are a file header and a data packet header, 12×100 denotes data content containing 12 groups of 100 bytes, each data content contains 2 bytes of start flag FFEE, 2 bytes of azimuth, 96 bytes of distance and reflectivity information, and the last 4+2 denotes 6 bytes of time stamp and echo mode.
The data acquisition and storage task related test is carried out on the data acquisition and storage system based on the ZYNQ-7000 development board and the VLP-16 laser radar, and the following results can be obtained:
observing the state of real-time data acquisition from the ZYNQ development board by using a serial port debugging assistant; the data acquisition and storage are realized by setting related programs, the user datagram protocol information is returned to the serial port and the information is fed back in real time through the PC serial port debugging assistant, so that the data acquisition and storage real-time state monitoring is completed, as shown in fig. 2, after the laser radar original data are subjected to data acquisition for three continuous hours, the system still operates normally, the continuous working time requirement of field data acquisition is met, and the state monitoring and the acquired data checking results of the acquisition and storage subsystem for three continuous hours are also shown in fig. 2.
(2) Time synchronization subsystem: based on an inertial navigation board card, the same time stamp is stamped on a multispectral camera image and a laser radar point cloud, position and posture information (satellite GPS and IMU) is provided by outputting position and posture data through one serial communication port of the inertial navigation board card, and the position and posture information is output through the serial port after passing through a conversion circuit; the other serial communication port of the inertial navigation board outputs recommended positioning information (GPRMC);
the standard time service system based on the satellite GPS has the advantages of high time service precision, high accuracy, safety, reliability and the like, a plurality of sensors are involved in the multi-sensor data acquisition and time synchronization system, the acquisition frequency of the laser radar raw data is as high as tens to hundreds of KHz, the data refreshing frequency of the inertial navigation board card is 200-500Hz, and the exposure frequency of the spectrum camera is about 1Hz. According to the inertial navigation board card, the north cloud bynav A1 GNSS board card is adopted, the same time stamp is synchronously stamped on the multispectral camera, the laser radar, the satellite GPS and the IMU, the north cloud bynav A1 GNSS board card is utilized to output a second pulse signal of the satellite GPS and a GPRMC signal to trigger and control the laser emission time of the laser radar device, 16-line laser is emitted according to the sequence set in the laser radar when the second pulse rising edge arrives, the specific emission time of the laser radar point cloud can be determined according to time information, and therefore time synchronous control of the laser radar and the inertial navigation board card on an unmanned aerial vehicle platform is achieved, and the first step of time-space consistency of multi-sensor data acquisition is achieved.
COM3 of the North cloud bynav A1 GNSS board card outputs satellite GPS and IMU data to provide POS information, an output interface is RS232, and the information is output through a serial port after passing through an RS232-TTL conversion circuit; outputting GPRMC information, second pulse and GND through a TXD serial port of COM1 of the North cloud bynav A1 GNSS board card; the rising edge of the second pulse signal is aligned with the receiving time of the satellite GPS and the IMU information, so that the accuracy of synchronous data can be ensured.
In outdoor environment experiments, the phenomenon that satellite GPS information and second pulse information are lost is found, and a duration synchronous trigger signal generator is arranged in the ZYNQ7010 and is used for outputting a duration synchronous trigger signal; by setting the millisecond timer count set in the ZYNQ development board, a short high level is output by the duration synchronization trigger signal generator to serve as a duration synchronization trigger signal every time when the time is up to one second or when the north cloud bynav A1 GNSS board card receives the second pulse rising edge of the satellite GPS signal, and the millisecond timer is cleared at the same time, so that the cycle is repeated.
When the signal is good and the second pulse from the satellite GPS signal can be normally received, the rising edge of the second pulse of the satellite GPS signal is receivedThe millisecond timer is immediately emptied, the duration synchronous trigger signal generator outputs a high level, namely a duration synchronous trigger signal, and the error between the newly generated duration synchronous trigger signal and the second pulse received from the satellite GPS signal can be accurate to the nanosecond level, so that the data frequency of the multispectral camera, the laser radar and the inertial navigation board card related by the system can be completely ignored. While normally receiving a second pulse generation duration synchronization trigger signal from a satellite GPS signal, ZYNQ7010 records the difference between the one-second duration counted by an internally set millisecond timer and the one-second duration with the UTC time being true. The difference has random error characteristics, and according to the data verification of a large number of repeated experiments, the random error distribution is found to be more approximate to normal distribution along with the increase of the statistics times. The invention records the difference value and processes the weighted average, and provides a real-time weighted error updating algorithm which corrects the millisecond timer. Using as a reference the difference of N seconds before the pulse of seconds of the satellite GPS signal not received (e 1 ,e 2 ,…e N ) The time when no second pulse is received is different, the difference data adopted is also different, and meanwhile, the N differences are not simply averaged, but the closer to the second pulse when no satellite GPS signal is received, the larger the weight of the error correction value is. Corrected errorWherein a is n The weight is calculated by adopting an exponential weighting algorithm, and the calculation formula is a n =(1-β)β N-n . The value of the beta is set to be 0.9 according to experiments and analysis, which means that the effect of the difference value of the adjacent time is larger on the result, and the real-time performance of the algorithm on error correction is shown. The millisecond timer corrected by the scheme can further reduce the interference of random errors and further improve the accuracy.
When the second pulse from the satellite GPS signal can not be normally received, the millisecond timer is emptied when the millisecond timer is full of one second, the obtained corrected error is utilized to correct the millisecond timer, and then a duration synchronization trigger signal generator generates a new duration synchronization trigger signal, so that the new duration synchronization trigger signal is closer to the second pulse from the satellite GPS signal than the second pulse which is obtained by the correction, and has higher precision to meet the time synchronization requirement of the whole system.
After the operation is finished, the newly generated duration synchronization trigger signal is regarded as second pulse information provided in the time synchronization process, and the second pulse information is output for subsequent operation, namely, the second pulse (second pulse of a satellite GPS) in subsequent laser radar and multispectral camera synchronization is the duration synchronization trigger signal.
Time synchronization process of laser radar and inertial navigation board card:
the outputted GPRMC information, duration synchronous trigger signal information and GND are respectively endowed to GPS RECEIVE, GPS PULSE and GND interfaces of the VLP-16 laser radar through the patch cord, and the interface is used for controlling the laser emission time of the laser radar by aligning the time rising edge of the laser radar emission sequence, so that when each duration synchronous trigger signal rising edge is received, the 16 lines sequentially emit laser signals in sequence to realize second PULSE locking. The satellite GPS and IMU data (POS information) aligned with the duration synchronous trigger signal and the GPRMC signal are stored in an external storage unit through a serial port of a ZYNQ7010 board card PS end, during operation, a counter built in a VLP-16 laser radar is arranged, when the duration synchronous trigger signal and the GPRMC information are received, a millisecond timer is emptied according to the rising edge of the duration synchronous trigger signal every time, time information of a second part in UTC time in the GPRMC information is endowed to the second counter built in the laser radar, more time information including years, months, dates, hours and the like is used for updating time data frame information in a-16 position packet, accuracy of the time information recorded in original data of the VLP-16 laser radar is guaranteed, the pulse rising edge of the duration synchronous trigger signal and the VLP-16 laser radar is aligned with the laser emission time of the VLP-16 laser radar 16 line laser, and the acquired original data of the VLP-16 laser radar is stored in the external storage unit through the ZYNQ7010 board card port, and time synchronization of the GPS radar and the original data and the satellite IMU data is processed.
The time synchronization system developed based on the ZYNQ-7000 core development board and the North cloud bynav A1 GNSS board card performs time synchronization related test, and the following results can be obtained:
the output is measured by an oscilloscope, and the output sequence, time node, period and signal duration of the second pulse signal and the GPRMC signal can be observed as shown in fig. 3. Since the GPRMC signal is a pulse-in-second signal, unlike a pulse-in-second signal, a certain processing time is required in the receiving process, and thus the pulse-in-second signal is delayed after the observation time of the oscilloscope.
Time synchronization process of multispectral camera and inertial navigation board card:
because the data acquisition frequency of the multispectral camera is about 1Hz, the photographing mode of the multispectral camera is divided into two modes of setting an internal timer to trigger photographing and setting an external trigger pulse signal to trigger photographing. The photographing mode of the external trigger pulse signals is selected, the trigger signals are designed according to the external trigger logic of the multispectral camera, the corresponding high-level duration time of the trigger signals is set according to the time interval from the time when the camera receives the trigger signals to the time when exposure starts, and the self-contained positioning and gesture determining module of the multispectral camera records the position and gesture information of the exposure moment of the camera without additional endowing related information. The inertial navigation board provides two data interfaces for the multispectral camera, the two data interfaces are GND ground wires and external trigger signals with the frequency of 1Hz respectively, the former ensures that no potential difference exists between the multispectral camera, the laser radar and the GND poles of the power supply part of the inertial navigation board, the normal working operation of the sensor is maintained, and the latter enables the multispectral camera to expose and photograph at the moment when the rising edge of the continuous time synchronous trigger signals arrives, so that the time synchronization of the multispectral camera and the inertial navigation board is realized, and the time synchronization of the multispectral camera and the laser radar is further realized, and the time-space consistency of the data acquisition of the multispectral sensor is truly realized.
The north cloud bynav A1 GNSS board card receives satellite GPS information provided by a navigation system, generates a duration synchronization trigger signal through the satellite GPS information, and aligns rising edge time of the duration synchronization trigger signal with UTC time integer second time; sending the duration synchronous trigger signal into a ZYNQ development board to adjust the level amplitude and the period duty ratio, and generating a second pulse trigger signal after adjusting the level amplitude and the period duty ratio;
level amplitude and period duty cycle adjustment: the level amplitude and the period duty ratio of a second pulse signal contained in a satellite GPS are adjusted to a section with the high level amplitude of 3.3-5 v, and the duty ratio is 25% -50%;
the second pulse trigger signal with the level amplitude and the period duty ratio being adjusted is provided for the multispectral camera and is used as the trigger signal for continuous photographing of the multispectral camera, so that the photographing frequency of one second can be realized, and the frequency of the trigger signal can be changed from one second to one fifth second, two seconds and even lower through an externally-added frequency dividing circuit and a frequency dividing circuit preset in a ZYNQ development board, so that synchronous trigger control of data acquisition of different photographing frequencies is met. The data obtained by photographing the multispectral camera are stored in an external storage unit of the multispectral camera, and an acquisition and storage system is not required to be designed independently.
And in the triggering process of the multispectral camera and the laser radar, the falling edge of the pulse-per-second signal and the trigger signal of the multispectral camera generated by the pulse-per-second signal are observed independently, so that the result shown in fig. 4 can be obtained, and fig. 4 shows the falling edge of the pulse-per-second signal provided by the inertial navigation board and the external trigger signal provided to the multispectral camera. The falling edge can be found to be about 23ns, which is much less than the multispectral camera exposure to light for a time on the order of nearly seconds, so that the accuracy of the time synchronization for the time synchronization subsystem is far enough to meet the accuracy requirements of the multispectral camera,
the external trigger signal provided for the multispectral camera is a square wave signal with the frequency of 1Hz and the duty ratio of 50%, and the rising edge of the signal is aligned with the rising edge of the second pulse signal and then is used for triggering the multispectral camera to take a picture, so that the high-precision time synchronization of the system is ensured.
Processing the time stamp in the original data frame of the laser radar according to the data obtained by the laser radar at high speed and periodically, and obtaining the specific time recorded in the time stamp of the data frame corresponding to each laser point by the 16-wire laser in each transmitting period of the VLP-16 laser according to the single-echo/double-echo mode set by the laser radar by combining a single-linear interpolation algorithm, so as to obtain the specific time of UTC time provided by each laser point corresponding to a satellite GPS;
the system designed by the invention is processed by the software debugging and hardware connection interface, is convenient to use, simple to operate, wide in application range and multiple in application scene, can meet the multi-dimensional data space acquisition processing tasks of various requirements, has high space-time consistency of the obtained data of a plurality of sensors, and is convenient for the processing of subsequent data and the related operations of space point cloud generation, three-dimensional reconstruction and the like.
Examples
The specific implementation mode of the ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system is described as follows:
(1) Laser radar raw data acquisition
And carrying out data acquisition on the laser radar through the LWIP-user datagram protocol network protocol built by the ZYNQ development board. The address 192.168.1.201, ports 2368 and 8308 of the laser radar respectively send the data packet and the position packet of the laser radar to the ZYNQ development board, the address of the ZYNQ development board is 192.168.1.10, and then the data processing is carried out by the processing system, and the collected laser radar original data is stored in the SD memory card (external memory unit).
(2) North cloud bynav A1 GNSS board card output data
The north cloud bynav A1 GNSS board card outputs 4 groups of signals, outputs a ground signal through serial ports COM1-GND, outputs a GPRMC signal through serial ports COM1-TX, outputs satellite GPS and IMU through serial ports COM3-TX, and outputs a second pulse signal through pins No. 23 to serve as a trigger signal and position and posture information of multi-sensor time synchronization.
(3) Laser radar and second pulse rising edge alignment locking
After the duration synchronous trigger signal is given to the laser radar, through parameter setting of the VLP-16 laser radar, accurate alignment of laser emission time of the laser radar 16 lines and the rising edge of the second pulse is achieved, after alignment of the laser emission time of the VLP-16 laser radar 16 lines and the rising edge of the second pulse and the pulse of GPRMC information, the 16 lines of lasers are ensured to emit sequentially according to the sequence of the marks 1-16, and the emission time of the laser with the number 1 is just the rising edge of the second pulse, so that locking of the laser radar emission time is achieved.
A schematic diagram of laser radar pulse per second locking is shown in fig. 5.
(4) Multi-sensor time synchronization connection
The second pulse signal output by the north cloud bynav A1 GNSS board card is directly given to the multispectral camera to serve as an external trigger signal; GND, second PULSE and GPRMC signals are respectively connected to GND, GPS-PULSE and GPS-RECEIVE interfaces of the VLP-16 laser radar to serve as time synchronization connection signals of the laser radar; the satellite GPS and IMU signals are output to a UART serial port of the ZYNQ-processing system by RS232, and are stored into an external storage unit after ZYNQ processing to serve as position and posture information after time synchronization alignment, so that subsequent point cloud processing is facilitated.
A multi-sensor time synchronization flow chart is shown in fig. 6.
A flow chart of the millisecond timer real-time weighted error update algorithm is shown in fig. 7
The physical diagram connection of the system is shown in fig. 8 to 10, and comprises physical connection of a laser radar part, a ZYNQ development board part and a North cloud bynav A1 GNSS board card part of the whole system.
The present invention is capable of other and further embodiments and its several details are capable of modification and variation in light of the present invention, as will be apparent to those skilled in the art, without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (10)

1. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system is characterized by comprising a time synchronization subsystem; the synchronization process of the time synchronization subsystem comprises the following steps:
synchronously stamping the original data of the multispectral camera image and the laser radar with the same time stamp based on the inertial navigation board card, outputting pose data through a serial communication port of the inertial navigation board card to provide position pose information, and outputting the information through a serial port after passing through a conversion circuit; the other serial communication port of the inertial navigation board outputs recommended positioning information;
a duration synchronous trigger signal generator is designed in the ZYNQ development board and is used for outputting a duration synchronous trigger signal; when the millisecond timing set in the ZYNQ development board is full of one second or when the inertial navigation board receives the second pulse rising edge of the satellite GPS signal, the duration synchronization trigger signal generator outputs a high level as the duration synchronization trigger signal, and simultaneously clears the millisecond timer, and the cycle is repeated;
when the inertial navigation board card can normally receive the second pulse from the satellite GPS signal, the rising edge moment of the second pulse of the received GPS signal immediately empties the millisecond timer, and the duration synchronous trigger signal generator outputs a high level, namely a new duration synchronous trigger signal; the method comprises the steps that when an inertial navigation board receives satellite GPS signals and a second pulse output by the inertial navigation board generates a duration synchronous trigger signal, the difference value between one second duration counted by a millisecond timer and one second duration with real UTC time is recorded, the difference value is recorded, the first N difference values at the current moment are subjected to weighted average processing according to weights, and the difference value is recorded as a correction error e; when the inertial navigation board card cannot normally receive the second pulse from the satellite GPS signal, the millisecond timer is emptied when the millisecond timer is full of one second, the millisecond timer is corrected by using correction errors e corresponding to the first N differences of the second pulse of the satellite GPS signal which is not received, and a new duration synchronous trigger signal is output by the duration synchronous trigger signal generator;
and recognizing the newly generated duration synchronization trigger signal as second pulse information in the time synchronization process, and utilizing the duration synchronization trigger signal output by the duration synchronization trigger signal generator to perform time duration synchronization of the multispectral camera and the laser radar.
2. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system of claim 1, further comprising an acquisition storage subsystem comprising a data storage module and a data acquisition module; the data storage module comprises a FLASH memory, an online cache unit, a storage controller and an external storage unit;
the FLASH memory is used for storing an operating system, a file system and related configuration data of the multi-sensor data acquisition and time synchronization system;
the online caching unit is used for storing temporary caches of output data of the multispectral camera and the laser radar, and temporary storage positions of stack allocation and related variables of a ZYNQ processor system of the ZYNQ development board in the running process;
the storage controller controls address signals, data information of the multiple sensors and various command signals in the data storage process, and controls conversion of a data interface between the ZYNQ development board and the multiple sensors and communication of read-write commands; the multi-sensor comprises a multi-spectrum camera, a laser radar and an inertial navigation board card;
the external storage unit is used for offline storing online cache of multi-sensor data and multi-sensor output data, wherein the multi-sensor data comprises multi-spectral images, laser radar original data and pose data;
the data acquisition module is used for acquiring multi-sensor data;
the method comprises the steps that after the laser radar is powered on and works, the collected original data are divided into a data packet and a position packet, the data are transmitted to a ZYNQ development board from a port respectively, and the collected data are read and stored in an external storage unit through a ZYNQ processing system.
3. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronization acquisition system of claim 2, wherein the process of time duration synchronization of the lidar using the duration synchronization trigger signal output by the duration synchronization trigger signal generator is as follows:
the pose data aligned with the duration synchronous trigger signal and the recommended positioning information signal are stored into an external storage unit through a serial port of the ZYNQ development board processing system; and when the counter arranged in the laser radar receives the duration synchronous trigger signal and the recommended positioning information, emptying the millisecond timer according to the rising edge of the duration synchronous trigger signal, giving the time information of the second part in UTC time in the recommended positioning information to the second counter arranged in the laser radar, updating time data frame information in a laser radar position packet by using the time information including year, month, date and hour, realizing the alignment of the pulse rising edge of the duration synchronous trigger signal and the recommended positioning information and the laser emission time of the laser radar multi-line laser, and transmitting and storing the acquired laser radar original data into an external storage unit through a ZYNQ development board network port.
4. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system of claim 3, wherein the triggering process of the lidar is as follows:
processing a time stamp in an original data frame of the laser radar aiming at the original data acquired by the laser radar, and obtaining specific time recorded in the time stamp of the data frame corresponding to each laser point by the laser device transmitting and receiving the obtained laser point in each transmitting period of the laser radar by interpolation calculation through a single linear interpolation algorithm in combination with a single echo/double echo mode set by the laser radar, so as to obtain specific time corresponding to UTC time provided by a GPS (global positioning system) for each laser point.
5. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronization acquisition system of claim 4, wherein the process of time duration synchronization of the multispectral camera using the duration synchronization trigger signal output by the duration synchronization trigger signal generator is as follows:
and aligning the rising edge time of the continuous time synchronous trigger signal with the integral second time of UTC time, sending the continuous time synchronous trigger signal into a ZYNQ development board for level amplitude and cycle duty ratio adjustment, generating a second pulse trigger signal after level amplitude and cycle duty ratio adjustment, providing the second pulse trigger signal after level amplitude and cycle duty ratio adjustment for the multispectral camera to serve as a trigger signal for continuous photographing of the multispectral camera, and controlling the multispectral camera to acquire images.
6. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system of claim 5, wherein the level amplitude and period duty cycle adjustment process is as follows:
and adjusting the level amplitude and the period duty ratio of the duration synchronous trigger signal to a section with the high level amplitude of 3.3-5V and the duty ratio of 25-50%.
7. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system according to claim 6, wherein the second pulse trigger signal after level amplitude adjustment and cycle duty ratio is provided for the multispectral camera, the second pulse trigger signal is used for controlling the multispectral camera to continuously shoot and acquire the trigger signal of the image, and the frequency of the trigger signal is changed through a preset frequency dividing circuit in the ZYNQ development board, so that the sampling frequency is adjusted.
8. The ZYNQ-based unmanned aerial vehicle platform multi-sensor data synchronous acquisition system of claim 7, wherein laser radar raw data acquired after power-on work of the laser radar is transmitted to the ZYNQ development board through a user datagram protocol.
9. The system for synchronously collecting the data of the multiple sensors of the unmanned aerial vehicle platform based on the ZYNQ, which is disclosed in claim 8, is characterized in that in the process that the data collected after the laser radar is powered on and works is transmitted to the ZYNQ development board from a port based on a user datagram protocol, firstly, space initialization is carried out on a data collection part, a new space is opened up, and an empty new user datagram protocol data frame area is generated; then applying for the ports corresponding to the two sockets respectively, and respectively realizing the receiving and collecting tasks of the data packet and the position packet;
copying the received data to an online caching unit, traversing all linked lists of a data packet structure body, copying the data pointed by a pointer corresponding to the data packet structure body to a user datagram protocol data frame, and recording the length information received by the total data volume; then, data judgment operation is carried out: and identifying a user datagram protocol data header mark, extracting related information, realizing information conversion operation from a. Pcap format to a. Txt format, and finally writing the obtained information in the. Txt format into an external storage unit to finish the functions of receiving, converting and storing data.
10. The ZYNQ-based unmanned aerial vehicle platform multisensor data synchronization acquisition system of any one of claims 1 to 9, further comprising a multispectral camera, a lidar for acquiring multispectral images, lidar raw data, respectively, and providing to the time synchronization subsystem and the acquisition storage subsystem.
CN202310524952.2A 2023-05-10 2023-05-10 Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ Pending CN116577802A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310524952.2A CN116577802A (en) 2023-05-10 2023-05-10 Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310524952.2A CN116577802A (en) 2023-05-10 2023-05-10 Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ

Publications (1)

Publication Number Publication Date
CN116577802A true CN116577802A (en) 2023-08-11

Family

ID=87544745

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310524952.2A Pending CN116577802A (en) 2023-05-10 2023-05-10 Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ

Country Status (1)

Country Link
CN (1) CN116577802A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117782227A (en) * 2024-02-26 2024-03-29 中国铁路设计集团有限公司 Multisource aerial remote sensing data acquisition device, system and control method
CN117782227B (en) * 2024-02-26 2024-05-10 中国铁路设计集团有限公司 Multisource aerial remote sensing data acquisition device, system and control method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117782227A (en) * 2024-02-26 2024-03-29 中国铁路设计集团有限公司 Multisource aerial remote sensing data acquisition device, system and control method
CN117782227B (en) * 2024-02-26 2024-05-10 中国铁路设计集团有限公司 Multisource aerial remote sensing data acquisition device, system and control method

Similar Documents

Publication Publication Date Title
CN107807365A (en) Small-sized digital photography there-dimensional laser scanning device for the unmanned airborne vehicle in low latitude
CN104764442B (en) Method and device for determining exposure time of aerial photogrammetric camera in light-small unmanned aerial vehicle
US7885143B2 (en) Seismic acquisition system
US20090079960A1 (en) Integrated still image, motion video and speed measurement system
CN109405969A (en) A kind of airborne DOAS spectrometer imaging and control circuit
US10708572B2 (en) Photogrammetric system and photogrammetric method
CN108303043B (en) Multi-sensor information fusion plant leaf area index detection method and system
CN112383675B (en) Time synchronization method and device and terminal equipment
CN211180212U (en) Mobile terminal for collecting map data and map collecting system
CN112861660B (en) Laser radar array and camera synchronization device, method, equipment and storage medium
CN206832361U (en) A kind of unmanned plane snap formula hyperspectral remote sensing system
CN107402069A (en) Unmanned plane snap formula hyperspectral remote sensing system and uniformity radiation correction method
CN110632577B (en) Time code demodulation processing circuit and method
CN108562279A (en) A kind of unmanned plane mapping method
Fuchs et al. Conceptual design of the CZMIL data acquisition system (DAS): integrating a new bathymetric lidar with a commercial spectrometer and metric camera for coastal mapping applications
CN114754769B (en) Data synchronization time service system and method for laser radar and inertial sensor
WO2020113358A1 (en) Systems and methods for synchronizing vehicle sensors and devices
CN104155664A (en) Satellite borne receiver timing functional test system and method
CN111896474A (en) BRDF data acquisition and data synchronization method based on aerial unmanned aerial vehicle
CN109752714B (en) Rotary radar micro-variation monitoring data processing method and radar system
CN107679260A (en) Precision of being united when on a kind of stationary orbit remote sensing satellite star determines method
CN116577802A (en) Unmanned aerial vehicle platform multisensor data synchronous acquisition system based on ZYNQ
CN108663024A (en) A kind of high rail satellite image localization method and system based on posture auxiliary information
CN112995524A (en) High-precision acquisition vehicle, and photo exposure information generation system, method and synchronization device thereof
CN113503965B (en) Method for realizing synchronous imaging of satellite-borne imaging spectrometer by using satellite platform second pulse

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination