CN110647491A - Distributed data acquisition and storage system and method for in-pipeline detection - Google Patents
Distributed data acquisition and storage system and method for in-pipeline detection Download PDFInfo
- Publication number
- CN110647491A CN110647491A CN201911049625.6A CN201911049625A CN110647491A CN 110647491 A CN110647491 A CN 110647491A CN 201911049625 A CN201911049625 A CN 201911049625A CN 110647491 A CN110647491 A CN 110647491A
- Authority
- CN
- China
- Prior art keywords
- data
- data acquisition
- acquisition
- storage system
- main controller
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 15
- 230000002093 peripheral effect Effects 0.000 claims abstract description 38
- 230000006854 communication Effects 0.000 claims abstract description 10
- 238000004891 communication Methods 0.000 claims abstract description 9
- 238000013480 data collection Methods 0.000 claims description 9
- 238000007689 inspection Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000007547 defect Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 3
- 239000000523 sample Substances 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 239000013078 crystal Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
Abstract
The invention relates to a distributed data acquisition and storage system and a method for in-pipeline detection, wherein the system comprises a main controller, a plurality of data collectors and an upper computer, the main controller is connected to the plurality of distributed data collectors through RS485, the data collectors are provided with a plurality of peripheral interfaces, the peripheral interfaces are used for connecting detection peripherals, the main controller is in communication connection with the upper computer, and the main controller comprises: a main processor: sending an acquisition instruction to a data acquisition unit and receiving acquisition data fed back by the data acquisition unit; SATA hard disk: storing the collected data fed back by the data collector; caching: the data cache is used for caching data before the data are stored and collected in the SATA hard disk; the SATA hard disk and the cache are both connected to the main processor. Compared with the prior art, the multipoint data acquisition and storage device can realize the acquisition and storage of multipoint data and can meet the requirements of high transmission speed and real-time property.
Description
Technical Field
The present invention relates to a data acquisition and storage system and method for in-pipeline detection, and more particularly, to a distributed data acquisition and storage system and method for in-pipeline detection.
Background
The detection in the pipeline is realized by detecting the geometric deformation, the metal corrosion defects of the inner wall and the outer wall of the pipeline and the like under the pushing of medium pressure difference in the pipeline by an inner detector, and positioning the positions of the defects in the pipeline. In order to adapt to the working conditions that the working pressure of a medium in a pipeline is large and the running speed of an inner detector is high, the inner detector in the pipeline is required to reduce sampling time and improve the sampling rate so as to ensure the measurement precision. The distribution randomness of the defects requires that the probes of the inner detector are arrayed along the circumferential direction of the pipe wall, the density is high, and multiple channels are sampled simultaneously. In addition, because the oil and gas pipeline has long transmission mileage, the operation time of the internal detector is long, the data point acquisition amount is large, and high-speed data transmission and storage are needed.
At present, data acquisition of a detector in a pipeline is generally single-point acquisition, and the summary integration of a plurality of pipeline detection data cannot be realized. Meanwhile, the data storage of the in-pipeline detector is mainly to transmit and store the data to the storage unit through the PCI-e bus, and this method is limited by the bandwidth of each transmission interface, and cannot increase the measurement speed to a high level. In addition, these storage methods generally transmit data to the storage unit based on the embedded operating system, so that the real-time performance of the system cannot be satisfied due to the random delay caused by task scheduling and the like of the operating system.
Disclosure of Invention
The present invention is directed to overcoming the above-mentioned drawbacks of the prior art, and providing a distributed data acquisition and storage system and method for in-pipeline inspection.
The purpose of the invention can be realized by the following technical scheme:
the utility model provides a distributed data acquisition storage system for detecting in pipeline, this system includes main control unit, a plurality of data collection station and host computer, main control unit be connected to a plurality of data collection station that are the distributing type through RS485, data collection station be equipped with a plurality of peripheral hardware interfaces, peripheral hardware interface be used for connecting the detection peripheral hardware, main control unit communication connection the host computer, main control unit include:
a main processor: sending an acquisition instruction to a data acquisition unit and receiving acquisition data fed back by the data acquisition unit;
SATA hard disk: storing the collected data fed back by the data collector;
caching: the data cache is used for caching data before the data are stored and collected in the SATA hard disk;
the SATA hard disk and the cache are both connected to the main processor.
Preferably, the data collector comprises:
an acquisition processor: the host processor and the peripheral interface are connected and used for receiving a data acquisition command of the host processor and transmitting the data acquisition command to the detection peripheral and simultaneously feeding back data acquired by the detection peripheral to the host processor;
RS485 transceiver: a plurality of the processors are used for acquiring the communication connection between the processors and the main processor and the peripheral interfaces.
Preferably, the main processor is a Kintex-7 XC7K70T FPGA.
Preferably, the cache comprises two pieces of DDR2 SDRAM.
Preferably, the main controller is connected with the upper computer through a USB 3.0.
Preferably, the upper computer comprises a PC.
Preferably, the acquisition processor is Spartan-6XC6SLX45 FGPA.
Preferably, the RS485 transceiver comprises a MAX3485 chip.
A distributed data acquisition and storage method for in-pipeline detection is based on the distributed data acquisition and storage system, and comprises the following steps:
(1) the main controller issues an acquisition instruction to the data acquisition unit;
(2) the data acquisition unit receives the acquired data of the detection peripheral equipment through the peripheral interface and sends the acquired data to the main controller;
(3) the main controller caches the acquired data in a cache;
(4) and the main controller stores the acquired data in the cache to the SATA hard disk.
Compared with the prior art, the invention has the following advantages:
(1) the data acquisition and storage system is in a distributed acquisition form, and can realize acquisition and storage of multipoint data and integration and summarization of the data;
(2) according to the invention, the SATA hard disk is arranged, and the bandwidth of the SATA2.0 interface can reach 300MB/s, so that the requirements of high speed and real-time data transmission of the detection positioning system in the pipeline can be met;
(3) because the main controller needs to receive data sent by a plurality of data collectors, and each data collector is connected with a plurality of peripherals, in order to avoid bus collision caused by overlarge data receiving amount of the main controller and insufficient storage speed of the SATA hard disk, the invention adopts two pieces of DDR2 SDRAM to perform data caching, the two pieces of SDRAM are connected in parallel to control a bus and an address bus, the total storage capacity reaches 2Gbit, the read-write width is 16bit, enough caching can be provided for batch data transmission, and the stability of data storage is ensured;
(4) the main controller is connected with the upper computer through the USB3.0, the USB3.0 transmission speed is high and reaches 300MB/s, and the acquired data can be guided into the upper computer in time for off-line calculation.
Drawings
FIG. 1 is a block diagram of a distributed data acquisition and storage system according to the present invention;
FIG. 2 is a schematic diagram of a main processor according to the present invention;
FIG. 3 is a schematic structural diagram of a data collector of the present invention;
FIG. 4 is a block diagram of a process for the master controller to send data;
FIG. 5 is a block diagram of a process for receiving data by the host controller;
in the figure, 1 is a main controller, 2 is a data collector, 3 is an upper computer, 4 is a peripheral interface, 11 is a main processor, 12 is a SATA hard disk, 13 is a cache, 14 is a LED display lamp, 15 is an active crystal oscillator, 16 is a power module, 17 is an RS485 module, 18 is a USB3.0 communication module, 21 is an acquisition processor, and 22 is an RS485 transceiver.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. Note that the following description of the embodiments is merely a substantial example, and the present invention is not intended to be limited to the application or the use thereof, and is not limited to the following embodiments.
Examples
As shown in fig. 1, a distributed data acquisition and storage system, this system includes main control unit 1, a plurality of data collection station 2 and host computer 3, data collection station 2 be distributed and be connected to main control unit 1 through RS485 module 17, data collection station 2 be connected with a plurality of peripheral hardware interfaces 4 respectively, peripheral hardware interfaces 4 be used for connecting the detection peripheral hardware, main control unit 1 passes through USB3.0 module 18 and connects host computer 3, host computer 3 is the PC in this embodiment, wherein RS485 module 17 adopts the MAX3485 chip.
As shown in fig. 2, the main controller 1 includes:
the main processor 11: receiving a configuration instruction sent by an upper computer 3, transmitting data stored in an SATA hard disk to the upper computer, issuing an acquisition instruction to a data acquisition unit 2 and storing the acquired data fed back by the data acquisition unit 2, wherein a main processor 11 is a Kintex-7 XC7K70T FPGA;
SATA hard disk 12: the data acquisition unit 2 is used for storing the acquired data fed back by the data acquisition unit, and the SATA hard disk 12 is connected to the main processor 11;
and (4) caching 13: the data cache is used for performing data cache before storing the collected data in the SATA hard disk 12, the cache 13 comprises two pieces of DDR2 SDRAM, and the cache 13 is connected to the main processor 11;
LED display lamp 14: for displaying the operating state of the main controller 1;
active crystal oscillator 15: providing a clock to the main processor 11;
the power supply module 16: power is supplied to the main processor 11.
And an RS485 module 17: a plurality of RS485 modules 17 are provided for the communication connection between the main processor 11 and the acquisition processor 21, and each RS485 module includes a MAX3485 chip.
Specifically, the method comprises the following steps: the core chip of the main controller 1 selects a Kintex-7 XC7K70T FPGA, application layer software runs on the FPGA, and the kernel chip is responsible for receiving a configuration instruction of an upper computer, transmitting data stored in an SATA hard disk to the upper computer, and issuing a signal acquisition instruction to the data acquisition unit 2 for execution according to a format. Meanwhile, the FPGA is connected with the 512G SATA hard disk 12 through a GTP high-speed transceiving interface, and the SATA interface is used for driving the solid state hard disk to store through the SATA-HC IP Core. The bandwidth of the SATA2.0 interface is very high and can reach 300MB/s, so that the speed requirement of the system can be met. Because the main controller 1 needs to receive data sent by a plurality of data collectors 2, in order to avoid bus collision caused by overlarge data receiving amount of the main controller 1 and insufficient storage speed of the SATA hard disk 12, the system adopts two pieces of DDR2 SDRAM MT47H64M16HR buffer memory 13 to perform data buffer memory, the two pieces of SDRAM control the bus and the address bus in parallel, the total storage capacity reaches 2Gbit, the read-write width is 16 bits, and enough buffer memory can be provided for probe batch transmission. The FPGA is connected with the USB3.0 controller CYUSB3014 through a GPIF II interface, so that the main controller 1 can communicate with the upper computer 3 through the USB3.0 interface at the speed of 300MB/s, and data are guided into a computer to be resolved offline.
As shown in fig. 3, the data collector 2 includes:
the acquisition processor 21: the main controller 1 and the peripheral interface 4 are connected, and are used for transmitting a received data acquisition instruction of the main processor 11 to the detection peripheral, and feeding back data acquired by the detection peripheral to the main processor 11, wherein the acquisition processor 21 is Spartan-6XC6SLX45 FGPA;
RS485 transceiver 22: a plurality of the RS485 transceivers 22 are arranged and used for acquiring the communication connection between the processor 21 and the main processor 11 as well as the peripheral interface 4, and each RS485 transceiver comprises a MAX3485 chip.
Specifically, the core chip of the data collector 2 is Spartan-6XC6SLX45 FGPA. And the data acquisition unit 2 responds to the data acquisition instruction sent by the main controller 1 and transmits the data acquisition instruction to the peripheral module for execution. And simultaneously, feeding back the data and the state acquired by the peripheral module to the main controller 1. In 8 MAX3485 chips designed in the data acquisition unit 2, the MAX3485 chip realizes signal transmission in a differential mode, and the defects of interference, noise and the like in the signal communication process are effectively overcome. Wherein, 7 MAX3485 are responsible for connecting with 7 peripheral interface modules 4, and the 7 peripheral interface modules 4 are 7 probes. The 1 MAX3485 is connected with the main controller 1 and is responsible for transmitting the collected information of all the peripheral interface 4 modules to the main controller 1.
The invention relates to a collecting and storing method of a distributed data collecting and storing system, which comprises the following steps:
(1) the main controller 1 issues an acquisition instruction to the data acquisition unit 2;
(2) the data acquisition unit 2 receives the acquired data of the detection peripheral equipment through the peripheral interface 4 and sends the acquired data to the main processor 11;
(3) the main controller 1 caches the acquired data in a cache 13;
(4) the main controller 1 stores the collected data in the buffer 13 to the SATA hard disk 12.
The method specifically comprises the following steps:
(1) the master controller 1 transmits data
As shown in fig. 4, it is a block diagram of a flow of sending data by the main controller 1, and the main controller 1 and the data collector 2 communicate with each other by using a handshake mechanism. The main controller 1 continuously inquires the ready signal of the data collector 2, if the data collector 2 is ready, the main controller 1 will output an instruction through the register bridge to transmit the configuration instruction of the data collector 2 to the data collector 2. After receiving the instruction, the data collector 2 performs CRC error code check, performs data timeout check by using a watchdog timer, and sends a check mark to the main controller 1. The main controller 1 sets or resets the response bit according to the check mark to judge whether the data acquisition unit 2 correctly receives the instruction sent by the main controller 1.
(2) The master controller 1 receives data
As shown in fig. 5, which is a block diagram of a flow of receiving data by the main controller 1, the data collector 2 outputs an instruction to transmit the collected data to the main controller 1 through a pipeline. The output data is transmitted in a block structure, with 1 block equal to 1024 bytes. After the data collector 2 prepares a block of data, the ready data bit is set so that the main controller 1 can inquire and receive the data. After the main controller 1 receives the data, the CRC error code check and the data overtime check are carried out, the check mark is sent to the data receiver, the data receiver sets or resets the response signal according to the check mark, and if the response is successful, the main controller sends the data and receives the next group of sampling data from the data collector 2.
The above embodiments are merely examples and do not limit the scope of the present invention. These embodiments may be implemented in other various manners, and various omissions, substitutions, and changes may be made without departing from the technical spirit of the present invention.
Claims (9)
1. The utility model provides a distributed data acquisition storage system for detecting in pipeline, a serial communication port, this system includes main control unit (1), a plurality of data collection station (2) and host computer (3), main control unit (1) be connected to a plurality of data collection station (2) that are the distributing type through RS485, data collection station (2) be equipped with a plurality of peripheral hardware interfaces (4), peripheral hardware interface (4) be used for connecting and detect the peripheral hardware, main control unit (1) communication connection host computer (3), main control unit (1) include:
main processor (11): sending an acquisition instruction to the data acquisition unit (2) and receiving acquisition data fed back by the data acquisition unit (2);
SATA hard disk (12): the collected data fed back by the data collector (2) is stored;
cache (13): the data cache is used for caching data before the collected data is stored in the SATA hard disk (12);
the SATA hard disk (12) and the cache (13) are both connected to the main processor (11).
2. A distributed data acquisition and storage system for in-pipe inspection according to claim 1, wherein the data collector (2) comprises:
acquisition processor (21): the peripheral interface is connected with the main processor (11) and the peripheral interface (4) and is used for receiving a data acquisition instruction of the main processor (11) and transmitting the data acquisition instruction to the detection peripheral and simultaneously feeding back data acquired by the detection peripheral to the main processor (11);
RS485 transceiver (22): a plurality of the communication interfaces are provided for the acquisition processor (21) to communicate with the main processor (11) and the peripheral interface (4).
3. The distributed data acquisition and storage system for in-pipe inspection as claimed in claim 1, wherein said host processor (11) is a Kintex-7 XC7K70T FPGA.
4. The distributed data acquisition and storage system for in-pipe inspection as recited in claim 1, wherein the cache (13) comprises two DDR2 SDRAM.
5. The distributed data acquisition and storage system for in-pipeline detection according to claim 1, wherein the main controller (1) is connected with the upper computer (3) through a USB 3.0.
6. The distributed data acquisition and storage system for in-pipeline inspection according to claim 1, wherein the upper computer (3) comprises a PC.
7. The distributed data acquisition and storage system for in-pipe inspection as claimed in claim 2, wherein the acquisition processor (21) is Spartan-6XC6SLX45 FGPA.
8. The distributed data acquisition and storage system for in-pipe inspection as recited in claim 2, wherein said RS485 transceiver (22) comprises a MAX3485 chip.
9. A data acquisition and storage method for in-pipeline detection, which is based on the distributed data acquisition and storage system of any one of claims 1 to 8, and comprises the following steps:
(1) the main controller issues an acquisition instruction to the data acquisition unit;
(2) the data acquisition unit receives the acquired data of the detection peripheral equipment through the peripheral interface and sends the acquired data to the main controller;
(3) the main controller caches the acquired data in a cache;
(4) and the main controller stores the acquired data in the cache to the SATA hard disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911049625.6A CN110647491A (en) | 2019-10-31 | 2019-10-31 | Distributed data acquisition and storage system and method for in-pipeline detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911049625.6A CN110647491A (en) | 2019-10-31 | 2019-10-31 | Distributed data acquisition and storage system and method for in-pipeline detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110647491A true CN110647491A (en) | 2020-01-03 |
Family
ID=69013998
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911049625.6A Pending CN110647491A (en) | 2019-10-31 | 2019-10-31 | Distributed data acquisition and storage system and method for in-pipeline detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110647491A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117342A (en) * | 2011-01-21 | 2011-07-06 | 中国科学院上海技术物理研究所 | Peripheral component interconnect (PCI) Express bus-based multiband infrared image real-time acquisition system and method |
| CN103593315A (en) * | 2013-11-20 | 2014-02-19 | 中国船舶重工集团公司第七二四研究所 | Direct multi-hard-disk high-speed parallel reading and writing method based on FPGA |
| CN104102601A (en) * | 2014-07-16 | 2014-10-15 | 西安电子科技大学 | FPGA (field programmable gate array) based image data acquisition and storage system |
| CN106027992A (en) * | 2016-07-12 | 2016-10-12 | 池州职业技术学院 | High-speed data acquisition system |
| CN109491276A (en) * | 2017-09-11 | 2019-03-19 | 清华大学 | A kind of oil-gas pipeline internal detector data receiver and storage device |
-
2019
- 2019-10-31 CN CN201911049625.6A patent/CN110647491A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117342A (en) * | 2011-01-21 | 2011-07-06 | 中国科学院上海技术物理研究所 | Peripheral component interconnect (PCI) Express bus-based multiband infrared image real-time acquisition system and method |
| CN103593315A (en) * | 2013-11-20 | 2014-02-19 | 中国船舶重工集团公司第七二四研究所 | Direct multi-hard-disk high-speed parallel reading and writing method based on FPGA |
| CN104102601A (en) * | 2014-07-16 | 2014-10-15 | 西安电子科技大学 | FPGA (field programmable gate array) based image data acquisition and storage system |
| CN106027992A (en) * | 2016-07-12 | 2016-10-12 | 池州职业技术学院 | High-speed data acquisition system |
| CN109491276A (en) * | 2017-09-11 | 2019-03-19 | 清华大学 | A kind of oil-gas pipeline internal detector data receiver and storage device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102495132B (en) | Multi-channel data acquisition device for submarine pipeline magnetic flux leakage internal detector | |
| CN203673475U (en) | Serial port server | |
| CN103218338B (en) | The real-time many DSP debug system of a kind of signal processor system | |
| CN201583945U (en) | Serial communication system for multiple singlechips based on FPGA | |
| CN101887401A (en) | Apparatus for acquiring and storing high speed data in real time | |
| CN201306610Y (en) | Simple oil delivery pipeline temperature measuring device | |
| CN111090603B (en) | LVDS-to-USB 3.0 adapter | |
| CN103593315A (en) | Direct multi-hard-disk high-speed parallel reading and writing method based on FPGA | |
| CN112783818A (en) | Online upgrading method and system for multi-core embedded system | |
| WO2019047395A1 (en) | Oil and gas pipeline internal detector data receiving and storage apparatus | |
| CN201622113U (en) | Intelligent vibration monitor | |
| CN104361143A (en) | Portable data acquisition card and method thereof | |
| CN103856364A (en) | Bus signal monitoring device and method | |
| CN104615386A (en) | Off-core cache device | |
| US8909862B2 (en) | Processing out of order transactions for mirrored subsystems using a cache to track write operations | |
| CN210428439U (en) | Distributed data acquisition and storage system for in-pipeline detection | |
| CN106908136A (en) | A kind of signal acquisition processing circuit of distributed optical fiber vibration sensing system | |
| CN114138297A (en) | ZYNQ-based FPGA radar data debugging system and debugging method | |
| CN110647491A (en) | Distributed data acquisition and storage system and method for in-pipeline detection | |
| CN104123246A (en) | Interface expansion device and serial attached SCSI expander | |
| CN104317747A (en) | Data caching and sending device and method of network receiver | |
| CN202583967U (en) | LBE (Location Based Entertainment) bus monoboard computer based on processor MPC5200B | |
| CN115221070B (en) | NVMe disk-based system-on-chip diagnosis method | |
| CN100423039C (en) | Integral automatic integrating testing system | |
| CN207991687U (en) | Vibration analyzer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |