CN108737809B

CN108737809B - Remote synchronous image acquisition method

Info

Publication number: CN108737809B
Application number: CN201810567759.6A
Authority: CN
Inventors: 王京梅; 房晓瑜; 董岳; 晏子杰
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2018-06-05
Filing date: 2018-06-05
Publication date: 2020-10-23
Anticipated expiration: 2038-06-05
Also published as: CN108737809A

Abstract

The invention belongs to the field of image acquisition, and provides a remote synchronous image acquisition method which is applied to single-point, multi-point and multi-angle real-time image acquisition scenes aiming at the problems of transmission distance, time precision, algorithm complexity, application flexibility and the like in the prior art. The invention adopts an AS6802 clock synchronization protocol based on the packet network technology, can realize the clock synchronization among all the acquisition terminals in a fault-tolerant way, interactively transmits three types of messages by respectively configuring different devices in the network AS a synchronous controller (SM), a compression Controller (CM) and a Synchronous Client (SC), maintains a global time reference, and achieves the aim of synchronously triggering to generate acquisition signals. The invention can accurately control the synchronous triggering time of each image acquisition terminal, realizes the purposes of processing, exchanging and transmitting data quickly and efficiently, and has flexible configuration and high time synchronization precision.

Description

Remote synchronous image acquisition method

Technical Field

The invention belongs to the field of image acquisition, relates to Ethernet communication and data acquisition, and particularly relates to a flexible and efficient remote synchronous image acquisition method.

Background

In the field of stereo and multi-view imaging, most modern computer vision techniques require the acquisition of a single frame image to be synchronized, with temporal accuracy controlled on the order of milliseconds. The acquisition of synchronized images and the data processing of images of multiple acquisition points sometimes require one or more computer platforms, and the connection buses (firewire, USB, etc.) between the computer and the cameras cannot meet the data transmission and processing requirements of multiple high-speed cameras. The method for overcoming the limitation is to use a packet switching system, each host machine processes one or more high-speed cameras, and all terminals are connected by adopting an Ethernet interface protocol, so that the aims of flexible networking and high-efficiency image information processing are fulfilled. For example, machine vision three-dimensional measurement is a technology for capturing a frame of image information of the same object from different angles by using a plurality of high-speed cameras and then transmitting the frame of image information to a computer for analysis and processing, and is widely applied to the production and living fields of portrait feature recognition, detection of the size, the shape and the surface defects of micro devices, exploration of railway tunnel shapes and the like. In order to avoid image blurring, especially in the case of a dynamic object as a capture object, theoretically, multiple cameras need to be triggered at the same time, and then the images are collected and transmitted back. Although a synchronous acquisition technology that a plurality of cameras continuously acquire a plurality of frames of images and time alignment is carried out on a processing end by means of a reference object is adopted at present, the algorithm of the processing end is complex, and when an application scene is changed, a target reference object needs to be recalibrated, so that the application flexibility is greatly reduced.

The traditional image transmission modes include USB transmission, digital private line transmission, packet network transmission and the like. The USB transmission technology has the advantages of high transmission speed and simple integration, but has short transmission distance, poor synchronization performance and limited number of accessed devices, and is only suitable for high-speed near-end transmission; the digital private line transmission technology has the advantages of high transmission speed, high safety, low delay, high cost and low application flexibility; the packet network transmission technology has long transmission distance, no limit on the number of accessed devices in theory, mature technology and low cost. However, the packet network transmission technology has low real-time performance and high delay, and the real-time performance and the delay must be improved and reduced by a clock synchronization protocol; the currently widely applied network clock synchronization protocol is NTP and PTP, the NTP protocol takes international standard time UTC as an accurate time source and is used for time synchronization between a distributed time server and a client, the precision of the NTP protocol can reach 100us at most in a local area network, and the precision of the NTP protocol can only reach 1ms-50ms in other networks; the PTP protocol is a precision clock synchronization protocol standard of a network measurement and control system, an optimal master clock is dynamically elected by adopting a master-slave clock synchronization method, the time precision error of the calibration can be controlled in microsecond level, but the algorithm is complex, the resource occupation is high, the master clock is seriously relied on, and the problem of single-point failure is easily caused.

Based on the problems of the prior art in the aspects of transmission distance, time precision, algorithm complexity, application flexibility and the like, the invention provides a remote synchronous image acquisition method which is applied to single-point, multi-point and multi-angle real-time image acquisition scenes.

Disclosure of Invention

The invention aims to provide a remote synchronous image acquisition method aiming at the defects of low time precision, complex algorithm and low application flexibility in the prior art. The invention can accurately control the synchronous triggering time of each image acquisition terminal, realizes the purposes of processing, exchanging and transmitting data quickly and efficiently, and has flexible configuration and high time synchronization precision.

In order to achieve the purpose, the invention adopts the technical scheme that:

a remote synchronous image acquisition method is characterized in that the image acquisition method is realized based on a packet network formed by a plurality of acquisition terminals and processing terminals, wherein each acquisition terminal is formed by an image acquisition module and a clock synchronization module, and the plurality of acquisition terminals are connected with the processing terminals through an AS6802 protocol switch; the method specifically comprises the following steps:

step 1, each acquisition terminal, each processing terminal and each switch exchange messages according to an AS6802 protocol, finish clock synchronization among multiple points in a packet network and generate a synchronization mark signal; after that, the asynchronous group detection function is executed periodically, and when the number of nodes which are not in the synchronous state in the packet network is greater than a preset threshold value, the clock synchronization algorithm is executed again;

step 2, the processing terminal detects the synchronous mark signal, if the packet network is in a synchronous state, the processing terminal simultaneously sends a control message to each acquisition terminal; if the packet network is not in a synchronous state, firstly executing a clock synchronization algorithm, and sending a control message after waiting that each node is in the synchronous state; the control message comprises image acquisition time, message type and destination address;

step 3, each acquisition terminal receives and analyzes the control message forwarded by the switch, registers acquisition time, and compares the acquisition time with local time maintained by a local clock module; when the two values are the same, triggering an image acquisition module to realize synchronous acquisition of a frame of image and caching image data to the local;

and 4, transmitting the image data acquired by each acquisition terminal to a processing terminal by the synchronous Ethernet configuration platform in the packet network.

The invention provides a remote synchronous image acquisition method.A clock synchronization scheme adopts an AS6802 clock synchronization protocol based on a packet network technology, can realize clock synchronization among acquisition terminals in a fault-tolerant manner, interactively transmits three types of messages by respectively configuring different devices in a network into a synchronous controller (SM), a compression Controller (CM) and a Synchronous Client (SC), and maintains a global time reference to achieve the aim of synchronously triggering and generating acquisition signals.

In conclusion, the beneficial effects of the invention are as follows:

1. the method and the device can accurately control the time point of image acquisition, keep clock synchronization of all acquisition terminals in the cluster, and reduce the influence of transmission delay on the trigger signal.

2. By adopting an AS6802 clock synchronization mechanism, an upper network protocol is not required to be involved, a synchronization algorithm is realized by depending on hardware, and the precision can reach dozens of ns.

3. The method is applied to a distributed network, can flexibly network, improve the transmission rate, reasonably utilize the bandwidth and has low cost.

4. The FPGA is used as a main control chip of the image acquisition terminal, so that the operation efficiency is high and the transmission rate is high.

Drawings

Fig. 1 is a block diagram of the overall structure of an image acquisition module and a clock synchronization module in the embodiment of the present invention.

Fig. 2 is an example diagram of networking of an image acquisition terminal and a processing terminal in the embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The embodiments described by referring to the drawings are exemplary only for the purpose of illustrating the invention and are not to be construed as limiting the invention.

The embodiment provides a remote synchronous image acquisition system, AS shown in fig. 2, which is a packet network composed of three image acquisition terminals, a processing terminal and a switch supporting an AS6802 protocol, and is applied to a three-dimensional imaging scene; the image acquisition terminal (101), the image acquisition terminal (102) and the image acquisition terminal (103) are composed of an image acquisition module and a clock synchronization module, and form a packet network together with the processing terminal and the switch; the image acquisition terminal, the processing terminal and the switch are mutually independent, and flexible networking can be realized according to specific application scenes. Specifically, the structural block diagram of each image acquisition terminal is shown in fig. 1, and the image acquisition module comprises a CCD sensor, a level conversion circuit, a filtering module, a cache FIFO, an SDRAM control circuit, an SDRAM, a ping-pong operation module, an MAC controller and a configuration module, and mainly completes the functions of image acquisition and packing transmission; the clock synchronization module comprises a local clock module, an AS6802 synchronization algorithm module, a frame receiving and sending module and a receiving and sending FIFO module, and mainly completes the clock synchronization algorithm related to the AS6802 protocol and the receiving and sending functions of PCF frames; the processing terminal mainly completes the functions of sending control signals, synchronizing clocks and processing transmission data. The control circuit of the image acquisition terminal is designed by an FPGA chip. According to different application scenes, the network structure between the synchronous image acquisition terminal and the processing terminal can be flexible and changeable to form network topologies such as star type and tree type.

The remote synchronous image acquisition method based on the remote synchronous image acquisition system specifically comprises the following steps:

(1) the system is initialized and powered on, a CC1 signal (falling edge triggers to read data) of the CCD sensor is set to be high level, the CCD image sensor has no data transmission, and the FPGA configures a mode register of the CCD sensor through a SerTc serial signal; according to the AS6802 clock synchronization protocol, three image capturing terminals (101), (102), (103) and one processing terminal (104) are configured AS SM, and a switch (201) is configured AS CM.

(2) And after the system time node configuration is completed, automatically executing a clock synchronization algorithm. And the image acquisition terminal and the processing terminal send CS frames to the switch to request synchronization. And the exchanger receives the sent CS frame, detects whether the state is idle or not, and informs the state of the exchanger to all the image acquisition terminals and the processing terminals. If the exchanger is idle, the image acquisition terminal and the processing terminal send CA frames to the exchanger to inform the exchanger to start synchronization, the exchanger also sends a CA frame to the image acquisition terminal and the processing terminal in a response mode, the handshake mechanism is established, and the synchronization mechanism is started. The three image acquisition terminals and the processing terminal send PCF frames to the switch in an integration period, and the switch executes a curing function after receiving the PCF frames, so as to determine the relative sequence of sending time points among the frames. And then the exchanger executes a compression function to obtain a time fault-tolerant average value among the image acquisition terminal, the processing terminal and the exchanger, and corrects a local clock value of the exchanger. After the local clock of the switch is calibrated, the PCF frame is sent to the image acquisition terminal and the processing terminal which are configured as SM, and the image acquisition terminal and the processing terminal execute the curing function and the compression function to calibrate the local clock, thus completing the process of clock synchronization once. After the clock synchronization of each terminal is completed, a high level flag signal syn _ flag is respectively generated, and each acquisition point sends a message signal of a user-defined data segment to the processing terminal to inform the completion of the synchronization. And each synchronized clock node executes an asynchronous group detection function in each integration period, and when the number of unsynchronized clock nodes is greater than a preset threshold value, a syn _ flag signal is set to be at a low level, and each clock node executes a clock synchronization algorithm again.

(3) Assuming that in the network topology shown in fig. 2, the processing terminal (104) needs to acquire images of three image acquisition terminals (101), (102), and (103) at the same time t, the processing terminal first detects whether the syn _ flag signal is in a high level state. If the syn _ flag signal is at a high level, sequentially sending control messages to each image acquisition terminal in a self-defined message form through a gigabit Ethernet port, namely storing the self-defined message identification type and an image acquisition time point t in a fixed data byte bit of the message; and if the syn _ flag signal is at a low level, starting a synchronization algorithm, repeating the step (1) until the clock synchronization among the nodes is achieved, and then sending a control message.

(4) The image acquisition terminal monitors a GMII interface between the MAC and the PHY, when the type of the message is detected to be the message with the acquisition time, a counter is defined, the address pointer points to the acquisition time information bit through the counter, and the time information is stored in a register and is compared with the counter defined by the local clock module. When the acquisition time point is the same as the local clock and the syn _ flag signal is at a high level, a control signal CC1 of the CCD sensor is set to be at a low level, so that a falling edge is generated to trigger the acquisition of one frame of image. The CCD sensor and the signal conversion circuit are arranged outside the FPGA chip and are connected with the FPGA through a Camera link interface. The signal conversion circuit comprises a TTL to LVDS circuit and an LVDS to TTL circuit. The CCD image sensor generates a 28-bit data signal (including a 4-bit video control signal and a 24-bit image data signal) and a clock signal. Signals are transmitted between the FPGA and the CCD sensor in a differential mode, the number of transmission lines can be saved, and interference among the signals is reduced.

(5) The collected image data is filtered by the filter module to extract effective pixel points, and the effective pixel points are sent to the cache FIFO. The FIFO is composed of a dual-port RAM and a peripheral control circuit, the full signal and the empty signal control read-write enable, and the read-write enable controls the change of address bits. The reading clock frequency of the buffer FIFO is higher than that of the writing clock, so that the process that one frame of image is uninterruptedly written into the FIFO from the CCD sensor, read out by the FIFO and written into the SDRAM is realized. The SDRAM is internally provided with 4 banks, and each bank is independent. And gating bank0 in the SDRAM through an address line, operating a write operation state machine in the SDRAM control circuit, sequentially performing operations of charging, refreshing, mode register configuration, writing data and the like, and storing the image in one frame. When the CCD sensor has finished transmitting one frame of image, the CC1 signal is set high to wait for the next falling edge trigger signal. After a frame of image is stored in a local cache SDRAM, each image acquisition terminal sends a self-defined message to a processing terminal to inform the processing terminal that the processing terminal can carry out transmission work.

(6) The configuration module is connected with the MAC controller, the clock synchronization module and the Camera Link bus, and adjusts the data transmission starting time, the transmission data length and the transmission period by receiving the configuration information sent by the synchronous Ethernet configuration platform, so that the contention-free, damage-free and end-to-end data transmission is realized. If the configuration module of the image acquisition terminal (101) detects the configuration information and requires the configuration information to transmit an image, the ping-pong operation module connected with the SDRAM output end starts to work. The ping-pong operation module consists of two dual-port rams with a depth of 1024 and a bit width of 16 bits, which are named ramA and ramB. Firstly, setting write enable of the ramA to be effective, setting read enable to be ineffective, storing one row of pixel data read out from the SDRAM into the ramA, and indicating the storage number by the address bit of the ramA. And when the ramA stores one row of pixel points, setting the write enable signal of the ramA to be invalid, and sending a transmission request signal to the MAC controller. And meanwhile, setting the write enable signal of the ramB to be effective, setting the read enable signal to be ineffective, and storing the next row of pixel points into the ramB. And when the SDRAM writes data into the ramB, the MAC controller starts a sending state machine, prepares packet header data, sends a response signal to the ramA and sets a read enable signal of the ramA to be effective. And pixel data in the ramA is filled into a data field of an Ethernet frame by controlling the sequential logic between the interfaces, so that the image data transmission based on the Ethernet is realized. Similarly, when the SDRAM writes the third row of pixel data into ramA, the MAC controller packetizes and transmits the data in ramB. The gigabit Ethernet transmission clock is higher than the working clock of the SDRAM, and the pixel data in the ramA and ramB can be alternately packed and transmitted, so that the transmission time is saved, and the working efficiency is improved.

(7) The data transmitted by the image acquisition terminal through the Ethernet is forwarded by the switch and then reaches the processing terminal corresponding to the destination address. And the processing terminal performs CRC check on the received data frame and observes whether the image frame loss and the error code phenomenon occur or not. And if the image frames appear, discarding all the currently transmitted image frames, generating error signals, and informing all the time nodes in the cluster in a message broadcasting mode. And after receiving the message containing the error signal, each image acquisition terminal terminates all current transmission tasks, restarts a sending task according to the configuration information and completes the transmission work of the remote synchronous image. If the image frame loss and error code phenomena do not occur, the received image frame is cached to local storage until all time nodes controlled by the image frame are completely transmitted, and then a processing algorithm is executed on a plurality of frames of images which are triggered to be acquired by a plurality of acquisition points at the same time and are transmitted in a time-sharing and segmented manner, so that the modeling project of the three-dimensional image is realized.

While the invention has been described with reference to specific embodiments, any feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise; all of the disclosed features, or all of the method or process steps, may be combined in any combination, except mutually exclusive features and/or steps.

Claims

1. A remote synchronous image acquisition method is characterized in that the image acquisition method is realized based on a packet network formed by a plurality of acquisition terminals and processing terminals, wherein each acquisition terminal is formed by an image acquisition module and a clock synchronization module, and the plurality of acquisition terminals are connected with the processing terminals through an AS6802 protocol switch; the method specifically comprises the following steps:

step 3, each acquisition terminal receives and analyzes the control message forwarded by the switch, registers acquisition time, and compares the acquisition time with local time maintained by a local clock module in the clock synchronization module; when the two values are the same, triggering an image acquisition module to realize synchronous acquisition of a frame of image and caching image data to the local;