CN117092627A - Side-scan sonar intelligent imaging hardware system and method based on lifting+FPGA - Google Patents
Side-scan sonar intelligent imaging hardware system and method based on lifting+FPGA Download PDFInfo
- Publication number
- CN117092627A CN117092627A CN202311255749.6A CN202311255749A CN117092627A CN 117092627 A CN117092627 A CN 117092627A CN 202311255749 A CN202311255749 A CN 202311255749A CN 117092627 A CN117092627 A CN 117092627A
- Authority
- CN
- China
- Prior art keywords
- module
- sonar
- fpga
- lifting
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 87
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims abstract description 6
- 238000001914 filtration Methods 0.000 claims description 32
- 238000004891 communication Methods 0.000 claims description 17
- 230000000630 rising effect Effects 0.000 claims description 16
- 238000010586 diagram Methods 0.000 claims description 14
- 230000001133 acceleration Effects 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 9
- NVMMUAUTQCWYHD-ABHRYQDASA-N Asp-Val-Pro-Pro Chemical compound OC(=O)C[C@H](N)C(=O)N[C@@H](C(C)C)C(=O)N1CCC[C@H]1C(=O)N1[C@H](C(O)=O)CCC1 NVMMUAUTQCWYHD-ABHRYQDASA-N 0.000 claims description 8
- 230000003321 amplification Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 8
- 238000013135 deep learning Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 238000003709 image segmentation Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 6
- 239000013078 crystal Substances 0.000 claims description 4
- 230000002093 peripheral effect Effects 0.000 claims description 4
- 238000000227 grinding Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims description 2
- 230000003993 interaction Effects 0.000 claims description 2
- 230000010354 integration Effects 0.000 abstract description 5
- 230000005540 biological transmission Effects 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000007792 addition Methods 0.000 description 3
- 102100030148 Integrator complex subunit 8 Human genes 0.000 description 2
- 101710092891 Integrator complex subunit 8 Proteins 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 206010013647 Drowning Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000002592 echocardiography Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000002513 implantation Methods 0.000 description 1
- 238000011068 loading method Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/52—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/88—Sonar systems specially adapted for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
Abstract
A side scan sonar intelligent imaging hardware system and method based on a lifting+FPGA, the system comprises a lifting intelligent image processing subsystem and a sonar signal acquisition and processing subsystem, a double-core architecture of lifting 310B processing and FPGA is adopted to construct the lifting intelligent image processing subsystem taking lifting 310B processing as a core, and the system is responsible for command control, sonar signal analysis, real-time imaging and sonar image processing; and the sonar signal acquisition and processing subsystem taking the FPGA as a core is responsible for acquisition, transmission and conversion of sonar analog signals and preprocessing of digital signals. The invention effectively solves the technical problems that the side-scan sonar system is difficult to deploy an AI algorithm, has low intelligence, low automation degree, low integration level and large high-speed signal data volume, and is difficult to image in real time, and simultaneously meets the requirements of side-scan sonar products with small volume and low power consumption.
Description
Technical Field
The invention belongs to the technical field of underwater acoustic mapping equipment, and particularly relates to a side scan sonar intelligent imaging hardware system and method based on a lifting processor and an FPGA.
Background
The side-scan sonar system is an imaging sonar, can draw an underwater environment, adopts an active sonar working mode, actively transmits sound wave signals, and detects underwater topography or obstacles according to information such as time delay, frequency, phase and the like contained in a reflected echo; the scanning array of the underwater acoustic imaging device has a large volume, and the emitting beam is a sector, so that the underwater acoustic imaging device can effectively create a large-area underwater acoustic image. Side-scan sonar typically works in the water by way of ROV or towed on the tail and sides of the hull, with transducers on both sides transmitting fan beam sonar signals when sailing. The side-scan sonar system is controlled by upper computer software and used for imaging in the upper computer software, and has wide and important application in the fields of ocean drafting, bridge detection, drowning personnel rescue and the like.
Most of the existing side scan sonar systems adopt a DSP chip, an FPGA chip or an SoC chip based on ARM+FPGA architecture as a unique processor. The DSP and the FPGA have stronger digital signal processing and parallel computing capabilities, and the ARM fills advantages in transaction management and system application program control. The side scan sonar products at home and abroad mostly adopt self-contained general purpose computers to run upper computers, and large-scale software such as Klein company is run under Windows operating systemSonar Processing System upper computer processing system and HiMAX SSS side scan sonar upper computer software of Zhonghaida corporation have low integration level, lack of application of underwater target detection, image noise reduction and the like which need to be combined with deep learning, and lack of high-performance hardware support in the fields of underwater rescue, underwater object identification, sonar image optimization and the like.
The Atlas 200I A2 acceleration module is embedded with a lifting 310B processor, which is a further optimization and upgrade of the lifting 310 processor, the processor is a system on chip SoC (System on Chip), adopts a Taishan V200M four-core CPU based on ARM, has a main frequency of 1.8GHz, and integrates AI calculation, pictures and visionThe functions of frequency encoding and decoding, ISP, graphic output and the like are utilized, the highest of the single-Core low-power consumption GPU Mali G52 and DVPP module can support 1080P/30 frame video decoding of 40 paths and 1080P/30 frame video encoding of 20 paths, and the Davinci V300AI Core based on the self-grinding Davinci architecture analyzes and infers and calculates various types of data such as images, videos and the like, so that compared with the architecture of a lifting 310 processor, the system has the advantages of upgrading the architecture, improving the service performance by 20 percent, providing the multiplication and addition computing capability of up to 10TFLOPS FP16 (20 TOPS INTS) and having unique advantages in the fields of video processing, image recognition, machine learning, reasoning computing and the like. The module has small volume, low power consumption, has more abundant and easily-expanded interfaces than the rising 310 processor, reduces expansion cost, such as media interfaces HDMI2.0, MIPI-DSI and 8Lane high-speed interfaces, supports the configuration of high-speed buses such as PCIe 3.0x4, SGMII, USB3.0, SATA and the like, and also provides abundant I 2 C. The CAN, SPI and other low-speed interfaces are suitable for processing low-speed signals of sensors, system debugging and the like, and are combined with a complete stack software and hardware development platform for a rising series processor, so that the system is convenient and reliable to develop, low in cost and short in period.
The FPGA chip has the following advantages: 1) The device has rich universal pins and various triggers, is convenient to develop software and hardware, has short period and flexible hardware programmability; 2) The integrated circuit interface module can be mutually compatible with integrated circuits or peripherals with different protocol levels such as other TTL, LVDS and the like, and has stronger compatibility due to the fact that input and output standards are customized; 3) The method is suitable for high-speed signal processing, and the abundant high-speed interfaces enable the method to have unique advantages in the aspects of data extraction filtering, processing of large data volume of high-speed AD and the like; 4) The method has strong parallel computing capability, is suitable for computation intensive tasks, repeatedly distributes data computing resources by utilizing a concurrency technology, segments the tasks by utilizing a pipelining technology, simultaneously processes the segmented tasks, reduces time delay and improves instantaneity; 5) The system has lower power consumption than a CPU, a GPU and the like under unit calculation power, and is lower and convenient to integrate in a circuit board.
Disclosure of Invention
In order to overcome the defects of the existing side-scan sonar technology, the invention provides a side-scan sonar intelligent imaging hardware system and method based on a rising processor and an FPGA dual core.
The intelligent imaging hardware system of the side scan sonar based on the rising processor and the FPGA is characterized in that the rising 310B processor adopts a high-integration SoC, integrates CPU, GPU, AI, picture and video coding and decoding capabilities, has rich and easily-expanded interfaces, is provided with various accelerators, strengthens machine learning and reasoning computing capabilities, improves computing power under unit power consumption, adapts to complex multi-scene requirements, and meets the development trend of implanting various embedded devices in future AI technology; the FPGA has strong parallel computing capability, has unique advantages in synchronous acquisition and processing of large-data-volume sonar signals, and meets the requirements of instantaneity and low delay of side-scan sonar waterfall image imaging; the system effectively solves the technical problems that the side-scan sonar system is difficult to deploy an AI algorithm, is intelligent, low in automation degree and integration level, has large high-speed signal data quantity and is difficult to image in real time, and simultaneously meets the requirements of side-scan sonar products with small volume and low power consumption.
In order to solve the technical problems, the invention provides the following technical scheme:
side-scan sonar intelligent imaging hardware system based on lifting+FPGA, comprising: the intelligent imaging system comprises a lifting intelligent image processing subsystem and a sonar signal acquisition and processing subsystem, wherein the lifting intelligent image processing subsystem analyzes sonar waveform data uploaded by the sonar acquisition and processing subsystem to form imaging data, then carries out data storage and encoding and decoding of a sonar image, presents a side-scan sonar waterfall diagram on a display screen in real time, and synchronously realizes image processing functions related to deep learning, such as image segmentation and target detection; the sonar signal acquisition and processing subsystem is responsible for acquiring acoustic-electric signals transmitted by the underwater acoustic transducer and sending the processed signals to the lifting intelligent image processing subsystem.
Further, the intelligent imaging processing subsystem comprises:
the Atlas 200I A2 acceleration module comprises an Atlas 200I A2 minimum system based on a lifting 310B processor, an SD card and a debugging interface, is responsible for data interaction among an FPGA module, a gigabit network communication module and a lifting intelligent image processing subsystem, is used for analyzing and encoding and decoding imaging data uploaded by a sonar signal acquisition and processing subsystem, is converted into image data or video streams through an on-chip GPU and a DVPP module of the lifting 310B processor, and realizes real-time application of image segmentation based on an MP-Former (mask-piloted Transformer) network model or target detection based on a CenterNet-Key network by utilizing high calculation force of an on-chip Davinci architecture AI Core of the lifting 310B processor;
the PCIE3.0 high-speed bus interface comprises 4 paths of PCIe transceiving channels and is used for connecting SSD, and storing imaging data or exporting data in real time for playback;
the DSI interface is used for providing a visual interface of the rising system, transmitting imaging data to the display screen and displaying the side scan sonar waterfall map in real time;
the SD card is used for storing parameters and starting information of the rising system;
the debugging interface is used for accessing the Linux operating system in the rising processor and carrying out configuration of system parameters and protocol drivers; the gigabit network communication module comprises a PHY chip and an RJ45 network port with a transformer, and is used for sending control commands to the FPGA module by the elevation processor, wherein the control commands comprise PWM signal pulse width and interval of a driving transmitting plate, TVG curve mode, system starting and resetting information and data uploaded by a sonar signal acquisition and processing subsystem;
the USB3.0 interface is used for expanding peripheral equipment, comprises a network card and an human body input device, and can upload data to the cloud end and control a system and an upper computer by utilizing the human body input device.
The lifting intelligent image processing subsystem completes the receiving, analyzing, storing and encoding and decoding of imaging data uploaded by the sonar collecting and processing subsystem, the real-time display of a side-scan sonar waterfall diagram, the deployment and realization of an AI algorithm and the command control of the sonar signal collecting and processing subsystem.
Still further, sonar signal acquisition and processing subsystem include:
the amplifying and filtering module is used for amplifying and bandpass filtering the multipath tiny sonar analog signals and extracting sine signals with specific frequency and bandwidth;
the controllable gain module is used for performing gain compensation on the signal output by the amplifying and filtering module;
the DA chip is used for providing an accurate gain control signal for the controllable gain module;
the high-resolution AD chip is used for converting the multipath analog signals into digital signals and then transmitting the digital signals into the FPGA module;
JTAG interface, is used for the debugging of the sonar signal acquisition and processing subsystem;
the PWM interface is used for receiving pulse signals sent by the FPGA module so as to control the pulse period, pulse width and frequency of the transmitting voltage of the external transmitting plate of the system;
the RS232 interface is used for connecting the gesture sensor, and transmitting gesture information data acquired by the gesture sensor to the FPGA module for processing through an RS232 serial port;
the gigabit network communication module comprises a PHY chip and an RJ45 network port with a transformer and is used for transmitting the data processed by the FPGA module to the lifting intelligent image processing subsystem;
the FPGA module comprises an FPGA chip, a DDR, a crystal oscillator, an SD card module and a JTAG module, and is used for sending control signals to the PWM interface, the DA chip and the high-resolution AD chip, processing signals uploaded by the high-resolution AD chip and the attitude sensor, and transmitting data to the lifting intelligent image processing subsystem through the gigabit network communication module.
The sonar signal acquisition and processing subsystem completes the control of the transmitted signal, the acquisition, amplification, filtering and analog-to-digital conversion of the received signal, and the extraction and down-sampling and azimuth matched filtering of the digital signal, so that the imaging data are finally obtained.
The intelligent imaging of the side-scan sonar based on the intelligent imaging hardware system of the side-scan sonar based on the lifting+FPGA comprises the following steps:
(1) the FPGA module sends pulse signals through the PWM interface and controls an external transmitting plate of the system to drive the underwater acoustic transducer to transmit acoustic signals; (2) the underwater sound transducer array converts sonar echo into an electric signal, the electric signal is subjected to preliminary processing by the amplifying and filtering module and the controllable gain module, and then the electric signal is converted into a digital signal by the high-resolution AD module and enters the FPGA module; (3) the FPGA module combines the gesture data and the multi-beam imaging algorithm to process the gesture data into imaging data, and transmits the imaging data to the Atlas 200I A2 acceleration module through the gigabit network communication module; (4) the lifting 310B processor in the Atlas 200I A2 acceleration module is internally provided with four cores of TaishanV200M, the main frequency is 1.8GHz, the computing and storage resources in the lifting processor are allocated, the processes including upper computer control software in the system are scheduled, the lifting processor analyzes signals processed by the FPGA through an imaging algorithm, the imaging data are subjected to encoding and decoding operation by utilizing the built-in Mali G52 single-core GPU and the DVPP module, a video stream is formed at the imaging frame rate of 3fps, the video stream is transmitted to a display screen through a DSI interface for real-time imaging display, and meanwhile, the imaging data are synchronously stored in the SSD through the PCIE 3.0x4 high-speed bus provided by the lifting processor, so that data backup and subsequent playback on the display screen are realized; (5) the 10TFLOPS FP16 (20 TOPS INT8) high-power AI Core based on the Davinci architecture Davinci V300 in the lifting 310B processor is utilized to analyze the real-time imaging side-scan sonar waterfall map, so that the application of related algorithms such as target detection and image segmentation based on deep learning is realized.
The underwater sound transducer array in the step (2) converts the sonar echo into an electric signal, the electric signal is primarily processed by an amplifying and filtering module and a controllable gain module, and then the electric signal is converted into a digital signal by a high-resolution AD module and then enters an FPGA module, and the method specifically comprises the following steps: (21) The primary gain circuit performs preliminary amplification on 10 paths of signals input by the underwater acoustic transducer; (22) Filtering by using a Chebyshev band-pass filter to obtain the relation between the amplitude and the frequency of the filtered signal:(23) The FPGA gives an accurate voltage value through the DA chip, so that the gain compensation circuit is controlled to compensate the amplitude of the filtered signal, and the relation between the voltage and the FPGA control signal is as follows: />Wherein N is the number of DA chip bits; (24) The gain compensation circuit outputs voltage V according to the DA chip out Adjust the signal amplitude, gain and V out The relation of (2) is: gain (d)B)＝40V out +10, where V out Input in differential form; (25) The signals enter a high-resolution AD chip to carry out analog-to-digital conversion, and data generated at a sampling rate of 2Mbps are transmitted into an FPGA through an LVDS interface; (26) The FPGA performs preprocessing on the digital signals through the steps of band-pass filtering, quadrature demodulation, low-pass filtering, decimating and downsampling and matched filtering.
The invention takes the lifting 310B processor as an image processing unit, has the functions of CPU, AI calculation, picture and video encoding and decoding, ISP, graphic output and the like, performs system process scheduling and resource allocation by integrating a CPU with a high-performance Core, analyzes and encodes and decodes imaging data by utilizing a GPU Mali G52 and a DVPP module, performs matrix calculation on a complex neural network based on Davinci V300AI Core of a self-grinding Davinci architecture, performs image reasoning calculation by combining FCN and YOLO network structures after self-high-power implantation training, and realizes mass image data transmission by rich high-speed interfaces, so that the lifting 310B processor can realize efficient system control, image processing and data transmission under the conditions of low power consumption, small volume and high computing power.
The intelligent side-scan sonar imaging hardware system is designed by a dual-core architecture of the rising and the FPGA, and has the following beneficial effects:
(1) The computing power of the system is improved, the intelligent degree and the integration level are improved, an AI algorithm can be deployed, the integrated side-scan sonar system has the practical functions based on a deep learning algorithm such as necessary target detection, image recognition and the like in underwater rescue or hydrologic mapping, and meanwhile, the general upper computer hardware platform is completely replaced by a form of a rising processor, a display screen and rich interfaces;
(2) The real-time performance of the system imaging is improved, the time delay is reduced, the system can rapidly image under high-precision mass data, and the requirements of small-volume and low-power-consumption side-scan sonar products are met.
Drawings
FIG. 1 is a schematic diagram of a side scan sonar intelligent imaging hardware system of a lifting+FPGA in an embodiment of the invention.
FIG. 2 is a schematic diagram of a lifting intelligent image processing subsystem according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of a sonar signal acquisition and processing subsystem according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating the internal architecture of a processor of a boot 310B according to an embodiment of the present invention.
FIG. 5 is a flow chart of the signal acquisition and preprocessing of the sonar signal acquisition and processing subsystem according to an embodiment of the present invention.
Detailed Description
In order to describe the present invention more specifically, the following description of the technical solution in the present embodiment will be given in full in connection with schematic diagrams 1 to 5 in the present embodiment.
Example 1
FIG. 1 is a schematic diagram of a side scan sonar intelligent imaging hardware system of a lifting+FPGA according to an embodiment of the present invention, where the system specifically includes: a lifting intelligent image processing subsystem 100 and a sonar signal acquisition and processing subsystem 200, wherein the lifting intelligent image processing subsystem 100 comprises: the DSI interface 101 can transmit image and video streams to a display screen to provide a visual interface of a rising system and display a side scan sonar waterfall diagram in real time; a USB3.0 interface 103 for expanding peripherals including a network card, an ergonomic input device; the PCIe3.0 high-speed bus interface 104 includes 4 PCIe transceiver channels, which are used to connect to the SSD, store imaging data in real time or export data for playback; a debug interface 105 for accessing the internal system of the elevator and performing configuration of system parameters and protocol drivers; the PWM interface 201 is configured to receive a pulse signal sent by the FPGA module, so as to control a transmitting voltage of an external transmitting board of the system; JTAG interface 202, which is used for debugging sonar signal acquisition and processing subsystem 200; the RS232 interface 203 is used for connecting with a gesture sensor and returning system gesture data to the FPGA module for processing; the intelligent image processing subsystem 100 and the sonar signal acquisition and processing subsystem 200 are connected with each other through the gigabit network communication module 102, so that the function of command control of the intelligent image processing subsystem 100 for issuing signals such as system startup and TVG mode to the sonar signal acquisition and processing subsystem 200 through upper computer software and the function of data transmission of the intelligent image processing subsystem 100 after the data preprocessed by the FPGA module are transmitted by the sonar signal acquisition and processing subsystem 200.
FIG. 2 is a schematic diagram of a smart image processing subsystem 100 according to an embodiment of the present invention, the subsystem specifically comprising: atlas 200i A2 acceleration module 106 based on a rising 310B processor, DSI interface 101, gigabit network communication module 102, USB3.0 interface 103, pcie3.0 high-speed bus interface 104, debug interface 105, display 107, SSD module 108, SD card 109, wireless network card 110, and ergonomic input device 111. The system start program is burnt into the SD card 109 and inserted into the card slot of the rising intelligent image processing subsystem 100, so as to power up the system; entering a system through a debugging interface 105 to carry out loading of upper computer software or adjustment of system parameters and driving; the Atlas 200I A2 acceleration module 106 receives imaging data uploaded by the gigabit network communication module 102, stores the imaging data to the SSD module 108 through the PCIE3.0 high-speed bus interface 104, or transmits the imaging data to the cloud end through the wireless network card 110 externally connected with the USB3.0 interface 103, and simultaneously transmits an image processed by the Atlas 200I A2 acceleration module 106 to the display screen 107 through the DSI interface 101 for real-time display; the control of the host computer in system operation operates through an ergonomic input device 111 externally connected to the USB3.0 interface 103.
Fig. 3 is a schematic structural diagram of a sonar signal acquisition and processing subsystem 200 according to an embodiment of the present invention, where the subsystem specifically includes: FPGA chip 204, PWM interface 201, JTAG interface 202, RS232 interface 203, amplification filter module 205, controllable gain module 206, DA chip 207, high resolution AD chip 208, crystal oscillator 209, DDR210, SD card 211, and gigabit network communication module 102. The FPGA chip 204 loads a digital signal processing program through the JTAG interface 202, is powered on and started after being inserted into an SD card with system starting parameters, and the crystal oscillator 209 starts to vibrate, so that the program starts to run; the FPGA chip 204 controls the PWM interface 201 to drive the transmitting plate to transmit sound pressure pulses through the transmitting transducer, and the signal echoes enter the sonar signal acquisition and processing subsystem 200 through the underwater sound transducer receiving array and then enter the FPGA chip 204 through the LVDS interface after being processed by the amplifying and filtering module 205, the controllable gain module 206 and the high-resolution AD chip 208 in sequence; during sampling, the FPGA chip 204 controls the DA chip 207 to give different level signals to the controllable gain module 206 through the I2C so as to perform different gain compensation; the FPGA chip 204 performs signal processing in combination with the gesture data returned by the RS232 interface 203, mass data in the calculation process is temporarily stored in the DDR210, and imaging data obtained after the preprocessing is sent to the elevation intelligent image processing subsystem 100 through the gigabit network communication module 102.
FIG. 4 is a diagram illustrating the internal architecture of a processor of a boost 310B in the boost intelligent image processing subsystem 100 according to an embodiment of the present invention, the processor mainly comprises: the AI computing unit comprises an AI Core based on DaVinciV300, a memory access unit LSU, a Vector processing unit Vector, a Scalar processing unit scaler and a Cache/Buffer of a Cache structure, and is responsible for completing matrix multiplication and addition computation of a neural network in sonar image processing, and providing high-efficiency computing resources for segmentation and target detection of a side-scan sonar waterfall diagram; the CPU comprises four TAISHANV200M cores based on ARM architecture, is responsible for distributing computing resources and storage resources of an AI computing unit and a storage structure, runs a lifting Linux operating system and schedules the process of the side-scan sonar upper computer; the GPU is responsible for real-time rendering of the side-scan sonar waterfall map by adopting a Mail G52 single-core image processing unit; ISP and DVPP, namely an image signal processing module and a digital vision preprocessing module, are responsible for encoding and decoding sonar image data; interfaces, including high-speed bus interface, media bus interface, DDR interface, are responsible for the interconnection communication of the boost processor and sonar signal acquisition and processing subsystem 200, external I/O devices, and external memory.
Fig. 5 is a flow chart of signal acquisition and preprocessing of a sonar signal acquisition and processing subsystem according to an embodiment of the present invention, where the signal acquisition and preprocessing steps are sequentially:
primary amplification, signal amplification is performed by using an in-phase amplification circuit,gain is
Chebyshev band-pass filter, which has the characteristics of ripple fluctuation such as frequency response amplitude on pass band or stop band, fast out-of-band attenuation and small circuit volume, and can obtain the filterRelation of signal amplitude to frequency:screening out signals with the transmitting frequency as the center;
program-controlled gain compensation, wherein the gain intensity of a TVG chip compensation signal is utilized, the FPGA gives an accurate voltage value through the DA chip, and the relation between the voltage and the FPGA control signal is as follows:wherein N is the number of DA chip bits, V out Directly controlling TVG chip to perform gain compensation, gain and V out The relation of (2) is: gain (dB) =40v out +10, where V out Input in differential form;
the analog/digital conversion is carried out, the high-precision AD chip samples the signals output by the TVG chip on the basis of meeting the Nyquist sampling theorem, and multiple paths of analog signals are converted into digital signals;
the FPGA receives the differential digital signals uploaded by the AD chip through an LVDS channel;
band-pass filtering, the filtering effect of the hardware circuit is enhanced, and signals with corresponding bandwidths are extracted from the mixed signals;
quadrature demodulation, realizing the movement of frequency bands, separating real signals and virtual signals, and facilitating the subsequent signal processing;
low-pass filtering to remove noise of high-frequency components after quadrature demodulation and keep the screened signal envelope;
extracting and downsampling, namely extracting signals by taking n data as a period, realizing n times of downsampling, reducing calculated amount, improving sonar imaging speed and improving system operation efficiency;
and matched filtering, namely matching a signal consistent with the transmitted Chirp pulse from the down-sampled signal.
The invention effectively solves the technical problems that the side-scan sonar system is difficult to deploy an AI algorithm, has low intelligence, low automation degree, low integration level and large high-speed signal data volume, and is difficult to image in real time, and simultaneously meets the requirements of side-scan sonar products with small volume and low power consumption.
Example 2
The intelligent imaging of the side-scan sonar based on the intelligent imaging hardware system of the side-scan sonar based on the lifting+FPGA comprises the following steps:
(1) the FPGA module sends pulse signals through the PWM interface and controls an external transmitting plate of the system to drive the underwater acoustic transducer to transmit acoustic signals; (2) the underwater sound transducer array converts sonar echo into an electric signal, the electric signal is subjected to preliminary processing by the amplifying and filtering module and the controllable gain module, and then the electric signal is converted into a digital signal by the high-resolution AD module and enters the FPGA module; (3) the FPGA module combines the gesture data and the multi-beam imaging algorithm to process the gesture data into imaging data, and transmits the imaging data to the Atlas 200I A2 acceleration module through the gigabit network communication module; (4) the lifting 310B processor in the Atlas 200I A2 acceleration module is internally provided with four cores of TaishanV200M, the main frequency is 1.8GHz, the computing and storage resources in the lifting processor are allocated, the processes including upper computer control software in the system are scheduled, the lifting processor analyzes signals processed by the FPGA through an imaging algorithm, the imaging data are subjected to encoding and decoding operation by utilizing the built-in Mali G52 single-core GPU and the DVPP module, a video stream is formed at the imaging frame rate of 3fps, the video stream is transmitted to a display screen through a DSI interface for real-time imaging display, and meanwhile, the imaging data are synchronously stored in the SSD through the PCIE 3.0x4 high-speed bus provided by the lifting processor, so that data backup and subsequent playback on the display screen are realized; (5) the 10TFLOPS FP16 (20 TOPS INT8) high-power AI Core based on the Davinci architecture Davinci V300 in the lifting 310B processor is utilized to analyze the real-time imaging side-scan sonar waterfall map, so that the application of related algorithms such as target detection and image segmentation based on deep learning is realized.
The underwater sound transducer array in the step (2) converts the sonar echo into an electric signal, the electric signal is primarily processed by an amplifying and filtering module and a controllable gain module, and then the electric signal is converted into a digital signal by a high-resolution AD module and then enters an FPGA module, and the method specifically comprises the following steps: (21) The primary gain circuit performs preliminary amplification on 10 paths of signals input by the underwater acoustic transducer; (22) Filtering by using a Chebyshev band-pass filter to obtain the relation between the amplitude and the frequency of the filtered signal:(23) The FPGA gives an accurate voltage value through the DA chip, so that the gain compensation circuit is controlled to compensate the amplitude of the filtered signal, and the relation between the voltage and the FPGA control signal is as follows: />Wherein N is the number of DA chip bits; (24) The gain compensation circuit outputs voltage V according to the DA chip out Adjust the signal amplitude, gain and V out The relation of (2) is: gain (dB) =40v out +10, where V out Input in differential form; (25) The signals enter a high-resolution AD chip to carry out analog-to-digital conversion, and data generated at a sampling rate of 2Mbps are transmitted into an FPGA through an LVDS interface; (26) The FPGA performs preprocessing on the digital signals through the steps of band-pass filtering, quadrature demodulation, low-pass filtering, decimating and downsampling and matched filtering.
The foregoing detailed description of the preferred embodiments and the advantages of the invention has been presented in conjunction with the drawings, and the foregoing detailed description is merely illustrative of the invention and not intended to limit the invention to the particular embodiments and the invention is intended to cover modifications, additions, combinations and equivalents of the invention as well as all modifications and variations as fall within the true scope of the invention.
Claims (5)
1. Side-scan sonar intelligent imaging hardware system based on lifting+FPGA, which is characterized in that the device comprises: the intelligent imaging system comprises a lifting intelligent image processing subsystem and a sonar signal acquisition and processing subsystem, wherein the lifting intelligent image processing subsystem analyzes sonar waveform data uploaded by the sonar acquisition and processing subsystem to form imaging data, then carries out data storage and encoding and decoding of a sonar image, presents a side-scan sonar waterfall diagram on a display screen in real time, and synchronously realizes image processing functions related to deep learning, such as image segmentation and target detection; the sonar signal acquisition and processing subsystem is responsible for acquiring acoustic-electric signals transmitted by the underwater acoustic transducer and sending the processed signals to the lifting intelligent image processing subsystem.
2. A lifting + FPGA-based side scan sonar intelligent imaging hardware system as defined in claim 1, wherein said lifting intelligent image processing subsystem comprises:
the Atlas 200IA2 acceleration module comprises an Atlas 200IA2 minimum system based on a lifting 310B processor, an SD card and a debugging interface, is responsible for data interaction among an FPGA module, a gigabit network communication module and a lifting intelligent image processing subsystem, is used for analyzing and encoding and decoding imaging data uploaded by a sonar signal acquisition and processing subsystem, is converted into image data or video streams through an on-chip GPU and a DVPP module of the lifting 310B processor, and realizes real-time application of image segmentation based on an MP-Former (mask-piloted Transformer) network model or target detection based on a CenterNet-Keypoint network by utilizing the high calculation power of an on-chip Davinci architecture AI Core of the lifting 310B processor;
the PCIE3.0 high-speed bus interface comprises 4 paths of PCIe transceiving channels and is used for connecting SSD, and storing imaging data or exporting data in real time for playback;
the DSI interface is used for providing a visual interface of the rising system and transmitting the side scan sonar imaging data to the display screen for real-time display;
the SD card is used for storing parameters and starting information of the rising system;
the debugging interface is used for accessing the Linux operating system in the rising processor and carrying out configuration of system parameters and protocol drivers;
the gigabit network communication module comprises a PHY chip and an RJ45 network port with a transformer, and is used for sending control commands to the FPGA module by the elevation processor, wherein the control commands comprise PWM signal pulse width and interval of a driving transmitting plate, TVG curve mode, system starting and resetting information and data uploaded by a sonar signal acquisition and processing subsystem;
the USB3.0 interface is used for expanding peripheral equipment, and comprises a network card and an ergonomic input device.
3. A lift+fpga-based intelligent imaging hardware system for side scan sonar as defined in claim 1 or 2, wherein said sonar signal acquisition and processing subsystem comprises:
the amplifying and filtering module is used for amplifying and bandpass filtering the multipath tiny sonar analog signals and extracting sine signals with specific frequency and bandwidth;
the controllable gain module is used for performing gain compensation on the signal output by the amplifying and filtering module;
the DA chip is used for providing an accurate gain control signal for the controllable gain module;
the high-resolution AD chip is used for converting the multipath analog signals into digital signals and then transmitting the digital signals into the FPGA module;
JTAG interface, is used for the debugging of the sonar signal acquisition and processing subsystem;
the PWM interface is used for receiving the pulse signals sent by the FPGA module and controlling the transmitting voltage of the external transmitting plate of the system;
the RS232 interface is used for connecting the gesture sensor, and transmitting gesture information data acquired by the gesture sensor to the FPGA module for processing through an RS232 serial port;
the gigabit network communication module comprises a PHY chip and an RJ45 network port with a transformer and is used for transmitting the data processed by the FPGA module to the lifting intelligent image processing subsystem;
the FPGA module comprises an FPGA chip, a DDR, a crystal oscillator, an SD card module and a JTAG module, and is used for sending control signals to the PWM interface, the DA chip and the high-resolution AD chip, processing signals uploaded by the high-resolution AD chip and the attitude sensor, and transmitting data to the lifting intelligent image processing subsystem through the gigabit network communication module.
4. The intelligent imaging method of the side-scan sonar of the intelligent imaging hardware system of the side-scan sonar based on the lifting+FPGA as claimed in claim 1, which is characterized by comprising the following steps:
(1) the FPGA module sends pulse signals through the PWM interface and controls an external transmitting plate of the system to drive the underwater acoustic transducer to transmit acoustic signals;
(2) the underwater sound transducer array converts sonar echo into an electric signal, the electric signal is subjected to preliminary processing by the amplifying and filtering module and the controllable gain module, and then the electric signal is converted into a digital signal by the high-resolution AD module and enters the FPGA module;
(3) the FPGA module combines the gesture data and the multi-beam imaging algorithm to process the gesture data into imaging data, and transmits the imaging data to the Atlas 200IA2 acceleration module through the gigabit network communication module;
(4) the lifting 310B processor in the Atlas 200I A2 acceleration module is internally provided with four cores of TaishanV200M, the main frequency is 1.8GHz, the computing and storage resources in the lifting processor are allocated, the processes including upper computer control software in the system are scheduled, the lifting processor analyzes signals processed by the FPGA through an imaging algorithm, the imaging data are subjected to encoding and decoding operation by utilizing the built-in MaliG52 single-core GPU and DVPP module, a video stream is formed at an imaging frame rate of 3fps, the video stream is transmitted to a display screen through a DSI interface for real-time imaging display, and meanwhile, the imaging data are synchronously stored in the SSD through the PCIE 3.0x4 high-speed bus provided by the lifting processor, so that data backup and subsequent playback on the display screen are realized;
(5) the elevation 310B processor is utilized to analyze the real-time imaging side scan sonar waterfall pattern based on the 10TFLOPS FP16 (20 TOPSINT 8) high-computation AICore of the self-grinding Davinci architecture Davinci V300, so that the application of related algorithms such as target detection, image segmentation and the like based on deep learning is realized.
5. The intelligent imaging method of the side-scan sonar according to claim 4, wherein the underwater sound transducer array in the step (2) converts the sonar echo into an electric signal, the electric signal is initially processed by an amplifying and filtering module and a controllable gain module, and then the electric signal is converted into a digital signal by a high-resolution AD module and then enters an FPGA module, and the method specifically comprises the following steps:
(21) The primary gain circuit performs preliminary amplification on 10 paths of signals input by the underwater acoustic transducer;
(22) Filtering by using a Chebyshev band-pass filter to obtain the relation between the amplitude and the frequency of the filtered signal:
(23) FPGA (field programmable gate array) generalThe DA chip is used for giving an accurate voltage value, so that the gain compensation circuit is controlled to compensate the amplitude of the filtered signal, and the relation between the voltage and the FPGA control signal is as follows:wherein N is the number of DA chip bits;
(24) The gain compensation circuit outputs voltage V according to the DA chip out Adjust the signal amplitude, gain and V out The relation of (2) is: gain (dB) =40v out +10, where V out Input in differential form;
(25) The signals enter a high-resolution AD chip to carry out analog-to-digital conversion, and data generated at a sampling rate of 2Mbps are transmitted into an FPGA through an LVDS interface;
(26) The FPGA performs preprocessing on the digital signals through the steps of band-pass filtering, quadrature demodulation, low-pass filtering, decimating and downsampling and matched filtering.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311255749.6A CN117092627A (en) | 2023-09-27 | 2023-09-27 | Side-scan sonar intelligent imaging hardware system and method based on lifting+FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311255749.6A CN117092627A (en) | 2023-09-27 | 2023-09-27 | Side-scan sonar intelligent imaging hardware system and method based on lifting+FPGA |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117092627A true CN117092627A (en) | 2023-11-21 |
Family
ID=88781872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311255749.6A Pending CN117092627A (en) | 2023-09-27 | 2023-09-27 | Side-scan sonar intelligent imaging hardware system and method based on lifting+FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117092627A (en) |
-
2023
- 2023-09-27 CN CN202311255749.6A patent/CN117092627A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107817216B (en) | Automatic optical detection system based on CPU + GPU + FPGA architecture | |
| CN108806243B (en) | Traffic flow information acquisition terminal based on Zynq-7000 | |
| CN105631798A (en) | Low-power consumption portable real-time image target detecting and tracking system and method thereof | |
| CN1267826C (en) | On-chip system processor for multimedia | |
| CN102202171B (en) | Embedded high-speed multi-channel image acquisition and storage system | |
| US20130226001A1 (en) | Method and apparatus for performing ultrasound imaging | |
| EP3117774B1 (en) | Software-based ultrasound imaging system | |
| AU2008202891B2 (en) | Radiation imaging system | |
| CN205486304U (en) | Portable realtime graphic object detection of low -power consumption and tracking means | |
| CN110648273B (en) | Real-time image processing apparatus | |
| CN103780819B (en) | Intelligent industrial camera | |
| CN103268238A (en) | Method for achieving drive of universal serial bus (USB) camera on basis of ReWorks operation system | |
| CN102881159B (en) | Embedded double-DSP (digital signal processing) information data processing device and method | |
| CN111090603A (en) | LVDS-to-USB 3.0 adapter | |
| CN112810553A (en) | Data visualization system and method based on self-adaptive platform automobile open system architecture | |
| CN105068066B (en) | A kind of embedded fixed and floating multibeam echosounding sonar signal collection and processing platform | |
| CN117092627A (en) | Side-scan sonar intelligent imaging hardware system and method based on lifting+FPGA | |
| CN103140049B (en) | A kind of chip mounter quick positioning system based on FPGA and method | |
| CN209897174U (en) | Video image processing device based on FPGA | |
| CN203396956U (en) | Multi-sensor detection and information integration system | |
| JP2001187056A (en) | Adaptive scanning technique for ultrasonic color flow | |
| CN209895383U (en) | High-speed transmission device for digital image big data | |
| CN112433966B (en) | Multi-interface architecture based on Windows platform and implementation method | |
| CN213092147U (en) | Multichannel signal acquisition device based on FPGA and STM32 | |
| CN103565476A (en) | Medical ultrasound whole-frame image transmission system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |