CN108806243B

CN108806243B - Traffic flow information acquisition terminal based on Zynq-7000

Info

Publication number: CN108806243B
Application number: CN201810371782.8A
Authority: CN
Inventors: 陆生礼; 庞伟; 范雪梅; 泮雯雯; 武瑞利
Original assignee: Southeast University Wuxi Institute Of Integrated Circuit Technology; Southeast University
Current assignee: Southeast University Wuxi Institute Of Integrated Circuit Technology; Southeast University
Priority date: 2018-04-24
Filing date: 2018-04-24
Publication date: 2020-09-29
Anticipated expiration: 2038-04-24
Also published as: CN108806243A

Abstract

The invention discloses a traffic flow information acquisition terminal based on Zynq-7000, belonging to the technical field of traffic control system signal devices. The terminal takes a Zynq-7000 chip as a carrier, a framework comprising a video image acquisition sensor, an external memory module and an HDMI display is built, an AXI4 bus is used for interconnecting the PS module and the PL module, an IP core for accelerating the calculation of a convolutional neural network is designed, a communication framework for driving the AXI4-VDMA IP core and the AXI4-DMA IP core by an MCU realizes the real-time data interaction of the PS module and the PL module, the functions of video image acquisition, storage, target detection, flow statistics, display output and the like are integrated on a single chip, the integration level is high, and the digital image processing and data transmission with high speed and low delay can meet the real-time requirement of traffic flow statistics.

Description

Traffic flow information acquisition terminal based on Zynq-7000

Technical Field

The invention discloses a traffic flow information acquisition terminal based on Zynq-7000, belonging to the technical field of traffic control system signal devices.

Background

With the rapid development of social economy, the owned quantity of urban motor vehicles is rapidly increased, the road traffic flow is greatly improved, the overload running condition is increasingly prominent, the phenomena of traffic vehicle congestion, environmental deterioration, frequent accidents and the like are increasingly serious, and the traffic flow monitoring becomes the urban management problem concerned by people. Therefore, in urban road traffic management, real-time target detection, traffic flow statistics, management and scheduling on complex road conditions through a more effective technology is urgently needed, so that control of an Intelligent Traffic System (ITS) is realized, and urban traffic environment quality is improved.

At present, the common traffic flow detection technologies include: induction coil detection techniques, magnetometer detection techniques, ultrasonic detection techniques, passive infrared detector techniques, radar detection techniques, computer video detection techniques, and the like. However, no matter the wave detection method or the magnetic detection method is used, the detection result is seriously influenced by road environment factors, comprehensive and accurate traffic information cannot be provided, and the method can only be used as an auxiliary means of an intelligent traffic monitoring system. The road traffic flow detection technology based on the computer video has the advantages of small damage to urban road environment, flexible detector installation, low installation and maintenance cost, higher application value and wider application prospect. The traditional computer video image detection processing technology is that a field video signal acquired by a camera is transmitted to an information Processor (PC), and after digital processing, target detection and identification are carried out, and the target is converted into traffic flow information data to be transmitted to a control command center. However, the conventional information processor employs a CPU or even a GPU, and its application in video traffic flow information processing still has many defects, such as: the image information processor has large volume and high price; special data communication channels such as cables or optical fibers and the like must be laid for data transmission between the camera and the information processor, so that the installation complexity is improved while high cost and expenditure are caused; the increase of the detection units makes the network layout and wiring more and more complex, the system reliability is reduced, the maintenance difficulty is increased, and the problems of data transmission delay, distortion and even loss can be caused, so that the system accuracy and real-time performance are reduced.

Therefore, the invention adopts a ZYNQ-7Z20 series development board integrating ARM and FPGA for development, combines the convenient operability of ARM and the high-parallelism computing capability of FPGA, adopts a high-speed data communication interface, and meets the requirements of an advanced traffic management system and an intelligent vehicle road system under the local offline state with limited hardware resources by adopting a YOLO (You Only Look one) real-time rapid target detection algorithm based on a convolutional neural network model, thereby realizing a traffic flow information acquisition terminal with lossless image transmission quality, strong real-time performance, strong expandability and good compatibility.

Disclosure of Invention

The invention aims to provide a traffic flow information acquisition terminal based on Zynq-7000 aiming at the defects of the background technology, realizes the traffic flow statistics based on computer vision, and solves the technical problems of high cost, low real-time performance and poor expandability of video traffic flow statistics.

The invention adopts the following technical scheme for realizing the aim of the invention:

a traffic flow information acquisition terminal based on Zynq-7000 mainly comprises a video image acquisition sensor, a Zynq-7000 chip, an HDMI display and an external memory, wherein the Zynq-7000 chip integrates a PL (programmable Logic) module and a PS (Processing System) module, the video image acquisition sensor is used for acquiring road condition video information and sending the road condition video information to the PL module through an MIPI (mobile industry processor interface), the PL module is used for receiving, operating and outputting video image information, the PS module is used for data interaction control and traffic flow statistics, and the HDMI display is used for displaying processed video images.

The PL module mainly comprises: the Video input module comprises an OV _ Sensor IP module and a Video in To AXI4-Stream IP module, the OV _ Sensor IP module generates a 24-bit data Stream and line synchronization and field synchronization signals from received Video data, and the Video in To AXI4-Stream IP module converts the 24-bit data into a data format meeting an AXI4-Stream interface protocol, so that single-frame image information data acquisition and storage are realized.

Further, the convolutional neural network accelerator reads the preprocessed image In the external storage module FIFO (First In, First Out) buffer area to perform traffic target detection and identification operation, and returns the operation result to the external storage module FIFO buffer area.

Further, the Video output module comprises an AXI4-Stream To Video out IP module, converts the processed Video image data into a corresponding HDMI output format and outputs the format To the HDMI display, and generates a timing signal required by the output of the HDMI display by using a VTC.

Further, the PL module also includes an xlontact IP module that combines the DMA and VDMA two independent interrupt signals together for connection to the interrupt signal interface of the ZYNQ IP module of the PS module.

Furthermore, the external storage module adopts three buffer areas connected end to store data, and ping-pong reading and writing are carried out.

Further, the PS module comprises an ARM CotrexA9 dual-hardmac MCU, single-frame image information is acquired from the external storage module, the image is stored in the external storage module after being preprocessed, target position and type information are acquired according to the PL module operation result, target picture frame processing and traffic flow statistics are conducted on the image, and the image after the picture frame is returned to the external storage module.

Further, the MCU controls the working state of the convolutional neural network accelerator through the 10-bit flag bit of the GPIO, and the method comprises the following steps: receiving data and starting operation, judging whether convolution kernel data or characteristic diagram data and operation rules are received, judging whether relu operation is needed or not, and whether stride value is 1 or 2, and returning data group number after operation is finished.

Meanwhile, in the specific communication architecture of the traffic flow information acquisition terminal, the PS module and the PL module are connected through an AXI4 bus, so that data communication among the modules is realized.

Further, the MCU in the PS module drives the AXI4-VDMA IP core and is connected to the AXI 4-HP 0 interface of the PS end through the AXI 4-interconnection IP, the MCU drives the AXI4-DMA IP core and is connected to the AXI 4-HP 1 interface of the PS end through the AXI 4-interconnection 1 IP, and data interactive transmission of the PS module and the PL module is achieved.

Further, the video input module stores the preprocessed image to the external storage module through the AXI4-VDMA interface, and the video output module also obtains the video image data subjected to frame processing in the external storage module through the AXI4-VDMA interface.

Furthermore, the convolutional neural network accelerator reads data information to be subjected to convolutional operation in the external storage module through the AXI4-DMA interface, performs specified operation, and returns an operation result to the external storage module through the AXI4-DMA interface.

In the traffic flow information acquisition terminal based on Zynq-7000, the traffic target detection operation is based on an optimized YOLO method, a network parameter training set is a PASCAL VOC data set, the algorithm divides a picture to be detected into different grid areas, then frame prediction and probability of each area are obtained through operation, weight is distributed according to the probability value, finally, a threshold value is set, and a target detection result with the probability value larger than the threshold value is output.

Furthermore, the traffic flow statistical method based on double virtual detection lines is used for traffic flow statistical operation, in order to avoid missed detection and false detection of traffic targets and improve accuracy of flow statistics, two parallel virtual detection lines are arranged on the middle lower portion of a video image frame and are perpendicular to the direction of traffic flow, the length of each detection line is the length of each image, and the width between the two detection lines is adjusted according to road conditions and image acquisition angles.

By adopting the technical scheme, the invention has the following beneficial effects:

(1) according to the method, a Zynq-7000 chip is used as a carrier, an AXI4 bus is used for interconnecting a PS module and a PL module, an IP core for accelerating convolution neural network calculation is designed, a communication framework for driving the AXI4-VDMA IP core and the AXI4-DMA IP core by an MCU is adopted to realize real-time data interaction of the PS module and the PL module, functions of video image acquisition, storage, target detection, flow statistics, display output and the like are integrated on a single chip, and digital image processing and data transmission with high integration level, high speed and low delay can meet the real-time requirement of traffic flow statistics;

(2) the identification of different types of targets is realized through an optimized target detection mode, the identification precision is improved, a LINUX operating system is not required to be transplanted, the compatibility requirement of traffic road condition management can be met, and the expandability is strong;

(3) after testing and adding a convolutional neural network accelerator, the target detection operation time of a single image is shortened by 0.5 second, and the video display output verification can be completely synchronous with the real-time road condition;

(4) the Zynq-7000 chip with high cost performance is used, the cost of a single chip is 1/5 of a common GPU, the power consumption is low, only 5V power supply voltage is needed, and meanwhile, the operation speed can reach 25 times of that of a general CPU.

Drawings

Fig. 1 is a schematic diagram of the overall structural framework of the present invention.

Fig. 2 is a schematic diagram of a data communication architecture of the present invention.

Fig. 3 is a flow chart of the PS module control function of the present invention.

Detailed Description

The technical scheme of the invention is explained in detail in the following with reference to the attached drawings.

Fig. 1 is a schematic diagram of an overall structural framework of the present invention, the structural framework uses a Zynq-7000 chip of saint (Xilinx) with an ARM + FPGA functional architecture as a carrier, and an MIZ702N development board is provided with an LVDS interface and an HDMI interface, which support a variety of extensible devices. The video image acquisition sensor is connected with MIZ702N through an LDVS interface, and the HDMI display is connected with MIZ702N through an HDMI interface, so that acquisition input and display output of image data are realized.

The functional structure design implementation principle and the data communication principle of the traffic flow information collecting terminal disclosed in the present application are described below with reference to the data communication architecture shown in fig. 2 and the PS control module functional flowchart shown in fig. 3.

The specific functional structure design implementation principle and process are as follows:

1. an OV5640 camera is used as a video image acquisition sensor and is correspondingly configured to acquire a road condition information image data stream;

2. the Video image acquisition Sensor sends the obtained RGB565 format data To an OV _ Sensor IP module of the Video input module, the OV _ Sensor IP module decodes the RGB565 format data To obtain 32-bit wide data, and the Video in To AXI4-Stream IP module converts the 32-bit wide data into a data format meeting an AXI4_ Stream bus interface protocol and sends the data format To an AXI4 bus;

3. the MCU drives the AXI4-VDMA IP core to send the VDMA image data stream to an external storage module DDR3 memory;

4. the method comprises the following steps that an MCU (microprogrammed control unit) preprocesses image data and then drives an AXI4-DMA IP (Internet protocol) core to send a DMA data stream to a convolutional neural network accelerator to carry out corresponding specified operation, the MCU sends flag bit data through a GPIO (general purpose input/output), the 1 st bit from low to high is a GPIO enabling flag bit, the 2 nd and 3 rd bits are used for marking AXI4-DMA IP core to send convolution core or feature map data, the 4 th bit is an operation enabling flag bit, the 5 th, 6 th and 7 th bits are used for indicating the number of network layers, the 8 th bit is used for indicating whether relu is carried out, and the 9 th bit;

5. the convolutional neural network accelerator receives image data in the DDR3 memory, meanwhile, convolution operation is carried out according to corresponding GPIO values to detect a target, and the convolution operation result is returned to the DDR3 memory through an AXI4-DMA IP core;

6. the MCU reads the operation result of the convolutional neural network accelerator, identifies the target in the image to be detected according to the convolution result, screens out the target larger than a set threshold value as a detection result according to the confidence value of the detected target, selects a detection result with the highest confidence as a target identification area, and carries out classification, identification and calculation on the target identification area to obtain the size and the type of a block diagram of the final detection target;

7. the MCU carries out picture frame processing on the acquired original image data according To the size of a block diagram of a target, and outputs the image data To an AXI4-Stream To Video out IP module of a Video output module through an AXI4-VDMAIP core and an AXI4 bus, and the MCU carries out traffic flow statistics according To the type of the target;

8. the AXI4-Stream To Video out module carries out format conversion on the data, generates a final HDMI signal according To the time sequence sent by the VTC module, and outputs digital image information conforming To an HDMI interface protocol To an HDMI display.

The data communication architecture of the invention is shown in fig. 2, a PS module, a PL module and an external storage module are connected through an AXI4 bus to realize high-speed data communication among the modules, and an interface signal output by an AXI4-VDMA IP core is connected to an AXI4_ HP0 interface at a PS end through an AXI4-Interconnect IP core; the AXI4-DMA interface signal is connected to an AXI4_ HP1 interface at the PS end through an AXI4-Interconnect1 IP, and all drive control is realized by the MCU.

The specific data communication principle and process are as follows:

1. the video input module mainly comprises: the system comprises an OV _ Sensor IP module, a Video in To AXI4_ Stream IP module, an AXI4-VDMA IP module and an AXI4-Interconnect IP module, wherein the OV _ Sensor IP module sends received road condition information image data To the Video in To AXI4_ Stream IP module for format conversion, and the data subjected To format conversion is stored in an external storage module through an AXI4 bus and an AXI4-VDMA interface;

2. an input port and an output port of the convolutional neural network accelerator both adopt an AXI4_ Stream protocol and are connected with an AXI4-DMA IP module input port and an output port, the AXI4-DMA IP module is connected with an AXI4-Interconnect1 IP module, an AXI4-Interconnect1 IP core is accessed through an AXI4-DMA IP core, a DDR3 memory is accessed through an AXI4_ HP1 interface to read data information to be subjected to convolutional operation in an external storage module, then specified operation is carried out, and an operation result is returned to the external storage module through an AXI4-DMA IP core;

3. the MCU is connected with an AXI4_ period IP module (interface parameter configuration IP module) through an AXI4_ GPIO signal interface, and the AXI4_ period IP module carries out corresponding configuration on an AXI 4-interconnection IP module, an AXI4-VDMA IP module, an AXI4-DMA IP module and a GPIO, such as: the number of slave equipment numbers mounted by the AXI4_ Interconnect IP module, the working modes of the AXI4-VDMAIP module and the AXI4_ DMA IP module, the register address of GPIO and the like;

4. the xlontact IP module (signal merging IP module) combines 2 independent interrupt signals of an AXI4_ DMA IP module and an AXI4_ VDMA IP module together and connects the combined signals to an IRQ interface of a ZYNQ IP module in the PS module;

5. the video output module mainly comprises: AXI4_ Stream To Video out IP module, Hdmi _ displayIP module; video image data after frame processing in an external storage module is obtained through a multiplexing AXI4-VDMA IP module and an AXI 4-interconnection IP module, an AXI4_ Stream To Video out IP module performs data format conversion on the Video image data after frame processing To obtain a Video Stream and outputs the Video Stream To an Hdmi _ display IP module, and the Hdmi _ display IP module performs Video image coding on the Video Stream and displays the Video Stream on an HDMI display.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A traffic flow information acquisition terminal based on Zynq-7000 is characterized by comprising: a video image acquisition sensor for acquiring traffic road condition image data stream, a Zynq-7000 chip for processing the traffic road condition image data stream to detect a target and count the traffic flow, an external memory module for caching data in the process of processing the traffic road condition image data stream by the Zynq-7000 chip, and an HDMI display for displaying the target image detected by the Zynq-7000 chip,

the Zynq-7000 chip integrates:

the video input module converts the traffic road condition image data stream uploaded by the video image acquisition sensor into an AXI4 data stream and then sends the AXI4 data stream to an AXI4 bus,

the AXI4-VDMA IP core converts the AXI4 data stream read from the AXI4 bus into a VDMA data stream, buffers the VDMA data stream in the external memory module, transmits the traffic road condition image which is read from the external memory module and processed by the picture frame to the video output module,

the MCU is used for preprocessing a VDMA data stream read from the external memory module, driving the AXI4-VDMA IP core to feed back the preprocessed image data to the external memory module, driving the AXI4-DMA IP core to read a convolution result output by the convolution neural network accelerator from the external memory, screening according to a target detection result finally output by the convolution neural network accelerator to determine a target identification area, classifying prediction frames in the target identification area to determine the size and the category of a target block diagram, carrying out frame processing on the acquired traffic road condition image according to the size of the target block diagram and counting the traffic flow, and caching the traffic road condition image processed by the frame in the external memory module,

AXI4-DMA IP core, converting the preprocessed image data read from the external memory into DMA data stream, transmitting the DMA data stream to the convolutional neural network accelerator, transmitting the convolutional result output by the convolutional neural network accelerator to the external memory module,

the convolution neural network accelerator outputs a convolution result after performing convolution calculation on the received DMA data stream,

and the video output module is used for converting the traffic road condition images processed by the picture frames into digital image information conforming to the HDMI interface protocol and outputting the digital image information according to the time sequence sent by the VTC IP core.

2. The Zynq-7000 based traffic flow information collection terminal of claim 1, wherein the Zynq-7000 chip further comprises a signal merging IP core, and the signal merging IP core combines the interrupt signal of AXI4-VDMA IP core and the interrupt signal of AXI4-DMA IP core and sends the combined interrupt signal to the IRQ interface of the MCU.

3. The Zynq-7000 based traffic flow information collection terminal of claim 1, wherein the Zynq-7000 chip further comprises an interface parameter configuration IP core, and the interface parameter configuration IP core configures the operation modes of the AXI4-VDMA IP core and the AXI4_ DMA IP core.

4. The Zynq-7000-based traffic flow information acquisition terminal as claimed in claim 3, wherein the GPIO interface of the MCU sends flag bit data to the interface parameter configuration IP core, the flag bit data comprising: the convolutional neural network accelerator comprises a convolutional neural network accelerator enabling signal, a sending data type, an operation enabling signal, the number of layers of convolutional layers, an activating signal of the convolutional layers and step length information of a sliding convolutional window.

5. The Zynq-7000-based traffic flow information acquisition terminal according to claim 1, wherein the external memory module is composed of three Ping-Pong read-write FIFO buffer areas connected end to end.

6. The Zynq-7000-based traffic flow information collection terminal of any one of claims 1 to 5, wherein the convolutional neural network accelerator detects the target based on a Yolo algorithm.

7. The Zynq-7000-based traffic flow information collection terminal of any one of claims 1 to 5, wherein the MCU counts the traffic flow including but not limited to pedestrian, automotive and non-automotive traffic flow based on the traffic flow statistical method of double virtual detection lines.