CN116929451B - Pipeline three-dimensional visual management system based on big data - Google Patents
Pipeline three-dimensional visual management system based on big data Download PDFInfo
- Publication number
- CN116929451B CN116929451B CN202310917633.8A CN202310917633A CN116929451B CN 116929451 B CN116929451 B CN 116929451B CN 202310917633 A CN202310917633 A CN 202310917633A CN 116929451 B CN116929451 B CN 116929451B
- Authority
- CN
- China
- Prior art keywords
- filtering
- monitoring data
- monitoring
- data
- pipeline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 9
- 238000001914 filtration Methods 0.000 claims abstract description 248
- 238000012544 monitoring process Methods 0.000 claims abstract description 244
- 238000012545 processing Methods 0.000 claims abstract description 59
- 238000004891 communication Methods 0.000 claims abstract description 42
- 238000004364 calculation method Methods 0.000 claims abstract description 25
- 238000012800 visualization Methods 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims description 26
- 238000001514 detection method Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 9
- 230000002123 temporal effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
- G06F2218/06—Denoising by applying a scale-space analysis, e.g. using wavelet analysis
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention belongs to the field of pipeline management, and discloses a pipeline three-dimensional visual management system based on big data, which comprises a monitoring sensor and an upper computer, wherein the upper computer comprises a communication module, a filtering module and a visual module; the monitoring sensor is used for obtaining monitoring data of the pipeline; the monitoring sensor carries out filtering processing on the obtained monitoring data according to a preset first filtering processing algorithm, and sends the filtered monitoring data to the communication module or directly sends the monitoring data to the communication module; the communication module is used for forwarding data sent by the monitoring sensor; the filtering module is used for carrying out filtering processing on the monitoring data which are not subjected to the filtering processing by adopting a second filtering processing algorithm; the second filter processing algorithm has a temporal complexity greater than that of the first filter processing algorithm; the visualization module is used for displaying the filtered monitoring data in a pre-established three-dimensional model of the pipeline. The invention can reduce the calculation capability requirement of the upper computer while ensuring the filtering effect.
Description
Technical Field
The invention relates to the field of pipeline management, in particular to a pipeline three-dimensional visual management system based on big data.
Background
The three-dimensional model is built on the pipeline, and then the data obtained by the sensor are displayed at the corresponding positions in the three-dimensional model, so that a manager can more intuitively know the state of the pipeline, and related technologies are disclosed in CN107340125A, CN109660596A in the prior art. In order to improve the accuracy of the obtained data, the data obtained by the sensor often needs to be filtered by an upper computer, but in the prior art, when the obtained data is filtered, a single filtering algorithm is generally adopted to perform the filtering, and the single filtering algorithm has the following defects that if the filtering algorithm with higher time complexity is adopted, the higher requirement is put on the computing capability of the upper computer, higher computing power is needed, the implementation cost is higher, and if the filtering algorithm with lower time complexity is adopted, the filtering effect is not good enough when the data change is larger, and the expected filtering effect cannot be achieved.
Disclosure of Invention
The invention aims to disclose a pipeline three-dimensional visual management system based on big data, which solves the problem of reasonably setting a filtering algorithm when filtering data obtained by detecting a pipeline by a sensor, thereby balancing the calculation force requirement and the filtering effect.
In order to achieve the above purpose, the present invention provides the following technical solutions:
the invention provides a pipeline three-dimensional visual management system based on big data, which comprises: monitoring a sensor and an upper computer; the upper computer comprises a communication module, a filtering module and a visualization module;
the monitoring sensor is used for monitoring the pipeline to obtain monitoring data of the pipeline;
the monitoring sensor is also used for calculating a filtering judgment value of the monitoring data in a set time period, judging whether the filtering judgment value is smaller than a set filtering judgment value threshold value, if yes, performing filtering treatment on the obtained monitoring data according to a preset first filtering treatment algorithm to obtain filtered monitoring data, and sending the filtered monitoring data to the communication module, if no, not performing filtering treatment on the monitoring data, and directly sending the monitoring data to the communication module;
the calculation function of the filtering judgment value is as follows:
indicating that the monitoring sensor is at t m Monitoring data obtained at the moment->Is a filter judgment value, mondat t Representing monitoring data obtained by a monitoring sensor at a time T, wherein T represents a preset first time length, stdef represents a preset variance of the monitoring data, phi represents a summation coefficient, phi epsilon (0, 1), dat mid 、dat mi 、dat ma Respectively indicates that the monitoring sensor is in a set time period [ t ] m -T,t m ]Median, minimum and maximum values of the monitoring data obtained in the process;
the communication module is used for receiving monitoring data which is transmitted by the monitoring sensor and is not subjected to filtering processing or the monitoring data after filtering;
the filtering module is used for carrying out filtering processing on the monitoring data which are not subjected to filtering processing by adopting a second filtering processing algorithm to obtain filtered monitoring data; the time complexity of the second filtering processing algorithm is greater than that of the first filtering processing algorithm;
the visualization module is used for displaying the filtered monitoring data in a pre-established three-dimensional model of the pipeline.
Preferably, the monitoring sensor comprises a monitoring unit, a judging unit, a filtering unit and a communication unit;
the monitoring unit is used for monitoring the pipeline to obtain monitoring data of the pipeline;
the judging unit is used for calculating a filtering judging value of the monitoring data in a set time period and judging whether the filtering judging value is smaller than a set filtering judging value threshold value or not;
the filtering unit is used for carrying out filtering processing on the obtained monitoring data according to a preset first filtering processing algorithm when the filtering judgment value is smaller than a set filtering judgment value threshold value to obtain filtered monitoring data;
the communication unit is used for sending the filtered monitoring data to the communication module, and sending the monitoring data which is not subjected to the filtering processing to the communication module when the filtering judgment value is larger than or equal to the set filtering judgment value threshold value.
Preferably, filtering the obtained monitoring data according to a preset first filtering algorithm to obtain filtered monitoring data, including:
for the monitoring sensor t m Monitoring data of pipeline obtained at momentThe process of filtering it using the first filtering algorithm is as follows:
calculation ofCorresponding filter range coefficients;
acquiring a set of monitoring data for performing filtering processing based on the filtering range coefficients;
set pair based on monitoring dataAnd filtering to obtain filtered monitoring data.
Preferably, the filter range coefficient is calculated as:
representation->The corresponding filter range coefficient, pre, represents a preset positive integer.
Preferably, acquiring the set of monitoring data for performing the filtering process based on the filter range coefficient includes:
will beThe number of (2) is denoted->The number num of the lowest numbered monitor data in the set of monitor data for performing the filtering process min The calculation function of (2) is:
the interval of the number of the monitoring data in the set of monitoring data for performing the filtering process is
And each time the monitoring sensor obtains one monitoring data, numbering the obtained monitoring data, wherein the difference value of the numbers between two adjacent monitoring data is 1, and the numbers are sequentially increased.
Preferably, the pairs are based on a set of monitoring dataFiltering to obtain filtered monitoring data, including:
respectively carrying out filtering treatment on elements in a set of monitoring data for filtering treatment by using a median filtering algorithm to obtain a filtering set;
the filtered monitoring data is calculated using the following function:
representing the filtered monitored data, datset representing the filtered set, dist max Represents the elements in datset and +.>Maximum value of absolute value of difference of numbers between +.>Representing detection data Mondat in datset i And->Absolute value of the difference in number between.
Preferably, the monitoring data of the pipeline includes the temperature of the pipeline, the humidity of the pipeline and the pressure of the pipeline.
Preferably, the visualization module is further configured to send a prompt to a manager of the pipeline when the filtered monitoring data exceeds the set monitoring range.
Compared with the prior art, after the monitoring sensor obtains the monitoring data of the pipeline, the filtering judgment value is calculated, when the filtering judgment value is smaller than or equal to the set filtering judgment value threshold value, the monitoring data is locally filtered, then the filtered monitoring data is sent to the upper computer, and when the filtering judgment value is larger than the set filtering judgment value threshold value, the upper computer carries out filtering treatment on the monitoring data. The second filtering processing algorithm with higher time complexity is arranged in the upper computer, so that the calculation capability requirement of the monitoring sensor is reduced.
The filtering judgment value can reflect the fluctuation condition of the monitoring data in the preset first time length, when the filtering judgment value is larger, the monitoring data change is complex, and at the moment, a filtering algorithm with stronger filtering capability is needed for filtering processing, so that the filtering judgment value is sent to an upper computer for filtering calculation, and otherwise, the filtering calculation is carried out through a monitoring sensor.
In summary, the invention can reduce the calculation capability requirement of the upper computer while ensuring the filtering effect, and balance between the calculation capability requirement and the filtering effect is achieved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a three-dimensional visualization management system for pipelines based on big data according to the present invention.
FIG. 2 is a schematic diagram of a monitoring sensor according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In one embodiment as shown in fig. 1, the present invention provides a three-dimensional visualization management system for a pipeline based on big data, comprising: monitoring a sensor and an upper computer; the upper computer comprises a communication module, a filtering module and a visualization module;
the monitoring sensor is used for monitoring the pipeline to obtain monitoring data of the pipeline;
the monitoring sensor is also used for calculating a filtering judgment value of the monitoring data in a set time period, judging whether the filtering judgment value is smaller than a set filtering judgment value threshold value, if yes, performing filtering treatment on the obtained monitoring data according to a preset first filtering treatment algorithm to obtain filtered monitoring data, and sending the filtered monitoring data to the communication module, if no, not performing filtering treatment on the monitoring data, and directly sending the monitoring data to the communication module;
the calculation function of the filtering judgment value is as follows:
indicating that the monitoring sensor is at t m Monitoring data obtained at the moment->Is a filter judgment value, mondat t Representing monitoring data obtained by a monitoring sensor at a time T, wherein T represents a preset first time length, stdef represents a preset variance of the monitoring data, phi represents a summation coefficient, phi epsilon (0, 1), dat mid 、dat mi 、dat ma Respectively indicates that the monitoring sensor is in a set time period [ t ] m -T,t m ]Median, minimum and maximum values of the monitoring data obtained in the process;
the communication module is used for receiving monitoring data which is transmitted by the monitoring sensor and is not subjected to filtering processing or the monitoring data after filtering;
the filtering module is used for carrying out filtering processing on the monitoring data which are not subjected to filtering processing by adopting a second filtering processing algorithm to obtain filtered monitoring data; the time complexity of the second filtering processing algorithm is greater than that of the first filtering processing algorithm;
the visualization module is used for displaying the filtered monitoring data in a pre-established three-dimensional model of the pipeline.
Compared with the prior art, after the monitoring sensor obtains the monitoring data of the pipeline, the filtering judgment value is calculated, when the filtering judgment value is smaller than or equal to the set filtering judgment value threshold value, the monitoring data is locally filtered, then the filtered monitoring data is sent to the upper computer, and when the filtering judgment value is larger than the set filtering judgment value threshold value, the upper computer carries out filtering treatment on the monitoring data. The second filtering processing algorithm with higher time complexity is arranged in the upper computer, so that the calculation capability requirement of the monitoring sensor is reduced.
The filtering judgment value can reflect the fluctuation condition of the monitoring data in the preset first time length, when the filtering judgment value is larger, the monitoring data change is complex, and at the moment, a filtering algorithm with stronger filtering capability is needed for filtering processing, so that the filtering judgment value is sent to an upper computer for filtering calculation, and otherwise, the filtering calculation is carried out through a monitoring sensor.
In summary, the invention can reduce the calculation capability requirement of the upper computer while ensuring the filtering effect, and balance between the calculation capability requirement and the filtering effect is achieved.
Preferably, the communication module transmits the received monitoring data which is not subjected to filtering processing to the filtering module, and transmits the monitoring data obtained after the received first filtering processing algorithm is filtered to the visualization module.
And the filtering module transmits the monitoring data obtained after the filtering processing of the second filtering processing algorithm to the visualization module.
Preferably, T is in seconds. The monitoring sensor acquires a piece of monitoring data every second.
Specifically, [ t ] m -T,t m ]The monitoring data obtained in the method is raw data which is not subjected to filtering treatment.
Specifically, the filter judgment value threshold value is obtained as follows:
the first step, the historical monitoring data of the pipeline is obtained, wherein the historical monitoring data are a plurality of monitoring data obtained in a second time length H, the unit of H is seconds, and the historical monitoring data are randomly extractedThe number of monitoring data, Z represents the total number of monitoring data obtained during the second time period H, for which +.>The monitoring data are respectively endowed with randomly generated numerical values to obtain noise adding data;
step two, initializing a filtering judgment value threshold value to be 0;
dividing the monitoring data of the noise adding data into a data set A which is more than or equal to the threshold value of the initialized filter judgment value or a data set B which is less than the threshold value of the initialized filter judgment value based on the filter judgment value;
a fourth step of performing filtering processing on the data in the data set B by using a first filtering processing algorithm and a second filtering processing algorithm respectively, and calculating to obtain a filtered data set C and a filtered data set D;
calculating the similarity between the data of the data set C and the data set D, and if the similarity is greater than 0.9, entering a fifth step; if the similarity is less than or equal to 0.9, entering a sixth step;
fifthly, adding 0.1 to the value of the filtering judgment value threshold value, and entering a third step;
and sixthly, outputting the current filtering judgment value threshold value.
In particular, the method comprises the steps of,similarity siml between data of data set C and data set D C,D The calculation function of (2) is:
numB is the amount of data in dataset B, aflt k,C For the data k in the data set B, the filtering result, aflite, obtained by the first filtering algorithm k,D A filtering result is obtained for the data k in the data set B through a second filtering processing algorithm; if aflit k,C Equal to aflite k,D ，adu(aflit k,C ,aflit k,D ) Has a value of 1, otherwise adu (aflite) k,C ,aflit k,D ) The value of (2) is 0.
Specifically, the filtering judgment value threshold is obtained based on historical monitoring data, wherein the historical monitoring data is filtered data. And randomly selecting a part of monitoring data from the historical monitoring data, and adding random values into the monitoring data to serve as noise, so that the noise-added data are obtained. For noisy data, since the values of the monitored data at different positions are different, different data fluctuations are introduced. The noise of the data entering the data set B is gradually increased by gradually increasing the value of the filtering judgment value threshold, and since the filtering capability of the first filtering processing algorithm is smaller than that of the second filtering processing algorithm, when the noise of the data increases to a certain extent, the first filtering processing algorithm cannot obtain a sufficiently accurate filtering result, and at this time, the filtering judgment threshold is obtained, and the filtering judgment threshold can enable the monitoring data filtered by the first filtering processing algorithm to meet the corresponding accuracy requirement.
Preferably, the upper computer can be a desktop computer, a notebook computer, a cloud server and the like.
The data in the communication module is transmitted to the filtering module through a data bus in the upper computer.
For example, the communication module may be a communication network card of the upper computer; the filtering module can be a filtering software terminal installed in the upper computer; and after the monitoring data is received by the communication network card, the monitoring data is transmitted to the filtering software terminal for calculation through a data bus on the main board.
Preferably, if the monitoring data has been subjected to the filtering process of the monitoring sensor, a corresponding identifier is added to the frame when the data of the monitoring data is transmitted, so as to distinguish whether the monitoring data has been subjected to the filtering process.
For example, the mac frame may be modified by adding an identification field between the data field and the CRC field, where 1 or 0 is filled in the identification field, 1 indicating that the filtering process has been performed, and 0 indicating that the filtering process has not been performed.
Preferably, when the data frame is encapsulated, the monitoring data which has been subjected to the filtering process and the monitoring data which has not been subjected to the filtering process may be encapsulated by different frames, and whether or not the filtering process has been performed is determined according to the type of the frame. For example, for a mac frame in which the filtered monitor data is encapsulated, an identification for discriminating is added in front of or behind the filtered monitor data in the data field. For example, letters are added for differentiation.
Preferably, the monitoring data of the pipeline is acquired at the very beginning, i.e. at time [0, E]The monitoring data obtained in the time interval is directly sent to the upper computer. E is less than or equal to t m 。
Specifically, since the filtering judgment value of the present invention needs to refer to the monitoring data obtained in the set period of time, when the monitoring is started, the number of monitoring data cannot meet the requirement of the set period of time, and at this time, the filtering process is not performed.
Preferably, as shown in fig. 2, the monitoring sensor includes a monitoring unit, a judging unit, a filtering unit and a communication unit;
the monitoring unit is used for monitoring the pipeline to obtain monitoring data of the pipeline;
the judging unit is used for calculating a filtering judging value of the monitoring data in a set time period and judging whether the filtering judging value is smaller than a set filtering judging value threshold value or not;
the filtering unit is used for carrying out filtering processing on the obtained monitoring data according to a preset first filtering processing algorithm when the filtering judgment value is smaller than a set filtering judgment value threshold value to obtain filtered monitoring data;
the communication unit is used for sending the filtered monitoring data to the communication module, and sending the monitoring data which is not subjected to the filtering processing to the communication module when the filtering judgment value is larger than or equal to the set filtering judgment value threshold value.
Specifically, the communication unit may communicate with the communication module by means of wired communication or wireless communication.
Preferably, the communication unit communicates with the communication module via a cellular network when the pipeline is at the surface. When the pipeline is at the ground, the communication unit communicates with the communication module through wired communication modes such as optical fibers.
Preferably, filtering the obtained monitoring data according to a preset first filtering algorithm to obtain filtered monitoring data, including:
for the monitoring sensor t m Monitoring data of pipeline obtained at momentThe process of filtering it using the first filtering algorithm is as follows:
calculation ofCorresponding filter range coefficients;
acquiring a set of monitoring data for performing filtering processing based on the filtering range coefficients;
set pair based on monitoring dataAnd filtering to obtain filtered monitoring data.
In particular, in the conventional filtering algorithm, filtering is generally performed by referring to the monitoring data in a fixed range, but when the variation of the monitoring data is relatively gentle, only a small amount of monitoring data for reference is needed to obtain a correct filtering result. Therefore, the invention obtains the set of the monitoring data for filtering processing through the filtering range coefficient, so that the filtering parameter range can fluctuate along with fluctuation of the monitoring data, thereby reducing the integral calculation amount while ensuring the accuracy of the filtering result, effectively reducing the calculation force requirement of the monitoring sensor and being beneficial to saving the implementation cost of the invention.
Preferably, the filter range coefficient is calculated as:
representation->The corresponding filter range coefficient, pre, represents a preset positive integer.
Preferably, acquiring the set of monitoring data for performing the filtering process based on the filter range coefficient includes:
will beThe number of (2) is denoted->The number num of the lowest numbered monitor data in the set of monitor data for performing the filtering process min The calculation function of (2) is:
the interval of the number of the monitoring data in the set of monitoring data for performing the filtering process is
And each time the monitoring sensor obtains one monitoring data, numbering the obtained monitoring data, wherein the difference value of the numbers between two adjacent monitoring data is 1, and the numbers are sequentially increased.
Specifically, the filtering range coefficient is related to the filtering judgment value, and the larger the filtering judgment value is, the larger the filtering range coefficient is; the smaller the filtering judgment value is, the smaller the filtering range coefficient is, so that the filtering parameter range can be changed along with the change of the monitoring data.
Preferably, the pairs are based on a set of monitoring dataFiltering to obtain filtered monitoring data, including:
respectively carrying out filtering treatment on elements in a set of monitoring data for filtering treatment by using a median filtering algorithm to obtain a filtering set;
the filtered monitoring data is calculated using the following function:
representing the filtered monitored data, datset representing the filtered set, dist max Represents the elements in datset and +.>Maximum value of absolute value of difference of numbers between +.>Representing detection data Mondat in datset i And->Absolute differences in numbering betweenAnd (5) pairing values.
Specifically, before the set of monitoring data for filtering processing is used, the elements in the set are filtered through the median filtering algorithm, so that the reference value of the elements in the set is effectively improved, the influence of noise on a filtering processing result is reduced, and the accuracy of the filtering result is improved. Because the elements in the set are elements which are not subjected to filtering treatment, if the elements are directly used, the filtering result is easy to be inaccurate due to noise.
In addition, when filtering the detection data, the number and the number of the element in the filtering setThe larger the difference between the numbers of the elements, the smaller the reference degree of the element, thereby avoiding that the element with too far distance affects the filtering result. Since if the monitored data shows a trend change, the reference value of the element with the farther number is lower, if the same reference degree is adopted, the filtering result is obviously inaccurate.
Preferably, the monitoring data of the pipeline includes the temperature of the pipeline, the humidity of the pipeline and the pressure of the pipeline.
Specifically, since the monitoring data of the pipeline has a plurality of types such as temperature, humidity, pressure and the like, the invention needs to filter each type of monitoring data respectively during filtering.
Preferably, displaying the filtered monitoring data in a pre-established three-dimensional model of the pipeline comprises:
acquiring coordinates of a monitoring sensor corresponding to the filtered monitoring data in a pre-established three-dimensional model of the pipeline;
and displaying the filtered monitoring data in the vicinity of the coordinates, wherein a line segment led out from the monitoring sensor points to the detection data during display, so that the acquisition position of the monitoring data is more visual.
Preferably, the visualization module is further configured to send a prompt to a manager of the pipeline when the filtered monitoring data exceeds the set monitoring range.
Preferably, the second filter processing algorithm includes a kalman filter algorithm, a wavelet denoising filter algorithm, or the like.
Specifically, when the second filtering algorithm is used for filtering, the reference data are all the filtered monitoring parameters.
The second filtering algorithm has higher time complexity and higher computational power requirement on the machine, but the filtering effect is better.
The preferred embodiments of the invention disclosed above are intended only to assist in the explanation of the invention. The preferred embodiments are not exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.
Claims (8)
1. The pipeline three-dimensional visual management system based on big data is characterized by comprising a monitoring sensor and an upper computer; the upper computer comprises a communication module, a filtering module and a visualization module;
the monitoring sensor is used for monitoring the pipeline to obtain monitoring data of the pipeline;
the monitoring sensor is also used for calculating a filtering judgment value of the monitoring data in a set time period, judging whether the filtering judgment value is smaller than a set filtering judgment value threshold value, if yes, performing filtering treatment on the obtained monitoring data according to a preset first filtering treatment algorithm to obtain filtered monitoring data, and sending the filtered monitoring data to the communication module, if no, not performing filtering treatment on the monitoring data, and directly sending the monitoring data to the communication module;
the calculation function of the filtering judgment value is as follows:
indicating that the monitoring sensor is at t m Monitoring data obtained at the moment->Is a filter judgment value, mondat t Representing monitoring data obtained by a monitoring sensor at a time T, wherein T represents a preset first time length, stdef represents a preset variance of the monitoring data, phi represents a summation coefficient, phi epsilon (0, 1), dat mid 、dat mi 、dat ma Respectively indicates that the monitoring sensor is in a set time period [ t ] m -T,t m ]Median, minimum and maximum values of the monitoring data obtained in the process;
the communication module is used for receiving monitoring data which is transmitted by the monitoring sensor and is not subjected to filtering processing or the monitoring data after filtering;
the filtering module is used for carrying out filtering processing on the monitoring data which are not subjected to filtering processing by adopting a second filtering processing algorithm to obtain filtered monitoring data; the time complexity of the second filtering processing algorithm is greater than that of the first filtering processing algorithm;
the visualization module is used for displaying the filtered monitoring data in a pre-established three-dimensional model of the pipeline.
2. The three-dimensional visualization management system for pipelines based on big data according to claim 1, wherein the monitoring sensor comprises a monitoring unit, a judging unit, a filtering unit and a communication unit;
the monitoring unit is used for monitoring the pipeline to obtain monitoring data of the pipeline;
the judging unit is used for calculating a filtering judging value of the monitoring data in a set time period and judging whether the filtering judging value is smaller than a set filtering judging value threshold value or not;
the filtering unit is used for carrying out filtering processing on the obtained monitoring data according to a preset first filtering processing algorithm when the filtering judgment value is smaller than a set filtering judgment value threshold value to obtain filtered monitoring data;
the communication unit is used for sending the filtered monitoring data to the communication module, and sending the monitoring data which is not subjected to the filtering processing to the communication module when the filtering judgment value is larger than or equal to the set filtering judgment value threshold value.
3. The three-dimensional visualization management system for pipelines based on big data according to claim 1, wherein the filtering processing is performed on the obtained monitoring data according to a preset first filtering processing algorithm to obtain filtered monitoring data, and the method comprises the following steps:
for the monitoring sensor t m Monitoring data of pipeline obtained at momentThe process of filtering it using the first filtering algorithm is as follows:
calculation ofCorresponding filter range coefficients;
acquiring a set of monitoring data for performing filtering processing based on the filtering range coefficients;
set pair based on monitoring dataAnd filtering to obtain filtered monitoring data.
4. A three-dimensional visualization management system for a pipeline based on big data according to claim 3, wherein the filter range coefficients are calculated as:
representation->The corresponding filter range coefficient, pre, represents a preset positive integer.
5. The three-dimensional visualization management system for a pipeline based on big data according to claim 4, wherein acquiring the set of monitoring data for performing the filtering process based on the filter range coefficients comprises:
will beThe number of (2) is denoted->The number num of the lowest numbered monitor data in the set of monitor data for performing the filtering process min The calculation function of (2) is:
the interval of the number of the monitoring data in the set of monitoring data for performing the filtering process isAnd each time the monitoring sensor obtains one monitoring data, numbering the obtained monitoring data, wherein the difference value of the numbers between two adjacent monitoring data is 1, and the numbers are sequentially increased.
6. A big data based three-dimensional visualization of a pipeline according to claim 3A management system, characterized in that based on the collection of monitoring data pairsFiltering to obtain filtered monitoring data, including:
respectively carrying out filtering treatment on elements in a set of monitoring data for filtering treatment by using a median filtering algorithm to obtain a filtering set;
the filtered monitoring data is calculated using the following function:
representing the filtered monitored data, datset representing the filtered set, dist max Represents the elements in datset and +.>Maximum value of absolute value of difference of numbers between +.>Representing detection data Mondat in datset i And->Absolute value of the difference in number between.
7. The three-dimensional visualization management system for pipelines based on big data according to claim 1, wherein the monitoring data of the pipelines comprises the temperature of the pipelines, the humidity of the pipelines and the pressure of the pipelines.
8. The three-dimensional visualization management system of claim 6, wherein the visualization module is further configured to send a prompt to a manager of the pipeline when the filtered monitoring data exceeds the set monitoring range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310917633.8A CN116929451B (en) | 2023-07-25 | 2023-07-25 | Pipeline three-dimensional visual management system based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310917633.8A CN116929451B (en) | 2023-07-25 | 2023-07-25 | Pipeline three-dimensional visual management system based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116929451A CN116929451A (en) | 2023-10-24 |
| CN116929451B true CN116929451B (en) | 2023-12-29 |
Family
ID=88387501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310917633.8A Active CN116929451B (en) | 2023-07-25 | 2023-07-25 | Pipeline three-dimensional visual management system based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116929451B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117910617B (en) * | 2023-12-25 | 2024-07-16 | 江苏方洋能源科技有限公司 | Remote prediction system for faults of photovoltaic panel |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102860005A (en) * | 2010-02-05 | 2013-01-02 | 瑞典爱立信有限公司 | De-blocking filtering control |
| CN105004363A (en) * | 2015-07-01 | 2015-10-28 | 长安大学 | Sensor performance on-line test device and method based on multi-threshold wavelet under strong interference |
| CN109657525A (en) * | 2017-10-12 | 2019-04-19 | 西南科技大学 | A kind of rapid correction method for traditional LiDAR filtering error |
| CN113195867A (en) * | 2018-12-17 | 2021-07-30 | 沙特阿拉伯石油公司 | Image-based inspection of well equipment and automated inventory and supply chain management |
| CN113566811A (en) * | 2021-06-30 | 2021-10-29 | 武汉港迪电气有限公司 | Data filtering fusion algorithm and device based on multi-path position detection sensor |
| CN115859081A (en) * | 2022-11-30 | 2023-03-28 | 广东电网有限责任公司 | Visual detection method and device for pipeline |
-
2023
- 2023-07-25 CN CN202310917633.8A patent/CN116929451B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102860005A (en) * | 2010-02-05 | 2013-01-02 | 瑞典爱立信有限公司 | De-blocking filtering control |
| CN105004363A (en) * | 2015-07-01 | 2015-10-28 | 长安大学 | Sensor performance on-line test device and method based on multi-threshold wavelet under strong interference |
| CN109657525A (en) * | 2017-10-12 | 2019-04-19 | 西南科技大学 | A kind of rapid correction method for traditional LiDAR filtering error |
| CN113195867A (en) * | 2018-12-17 | 2021-07-30 | 沙特阿拉伯石油公司 | Image-based inspection of well equipment and automated inventory and supply chain management |
| CN113566811A (en) * | 2021-06-30 | 2021-10-29 | 武汉港迪电气有限公司 | Data filtering fusion algorithm and device based on multi-path position detection sensor |
| CN115859081A (en) * | 2022-11-30 | 2023-03-28 | 广东电网有限责任公司 | Visual detection method and device for pipeline |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116929451A (en) | 2023-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116929451B (en) | Pipeline three-dimensional visual management system based on big data | |
| CN109387303B (en) | Detection method and device of shaft temperature sensor | |
| CN110458157B (en) | Intelligent monitoring system for power cable production process | |
| CN106408563B (en) | A kind of snow noise detection method based on the coefficient of variation | |
| CN111970229B (en) | CAN bus data anomaly detection method aiming at multiple attack modes | |
| CN110594954B (en) | Air conditioner fault detection method and detection device | |
| CN117439827B (en) | Network flow big data analysis method | |
| US9613123B2 (en) | Data stream processing | |
| CN117437223B (en) | Intelligent defect detection method for high-speed board-to-board connector | |
| CN117092980A (en) | Electrical fault detection control system based on big data | |
| CN116522096B (en) | Three-dimensional digital twin content intelligent manufacturing method based on motion capture | |
| CN117909908B (en) | Intelligent monitoring method for running state of shared massage chair | |
| CN108563190A (en) | A kind of numerical control machine tool monitoring system based on Internet of Things | |
| CN118395091B (en) | Data efficient acquisition system based on everything interconnection | |
| CN115170589A (en) | Method and related device for detecting frosting degree of evaporator of refrigerating machine | |
| US20200173887A1 (en) | Processing apparatus, processing method, and non-transitory storage medium | |
| CN115358951A (en) | Intelligent ring main unit monitoring system based on image recognition | |
| CN117609929A (en) | Industrial production line fault online diagnosis method and system based on big data | |
| CN109894476B (en) | Fault diagnosis method and device for hydraulic equipment of cold-rolled silicon steel production line | |
| CN114328078A (en) | Threshold dynamic calculation method and device and computer readable storage medium | |
| CN116668080A (en) | Flow abnormality assessment method and device, electronic equipment and storage medium | |
| CN112415968B (en) | Chinese medicine production informatization management system based on block chain | |
| CN116061005A (en) | Workpiece processing data acquisition method and device and electronic equipment | |
| CN115111182A (en) | Method and device for detecting running state of direct current fan and terminal equipment | |
| CN111258863A (en) | Data anomaly detection method, device, server and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |