CN117290798B

CN117290798B - Method, system and chip for overhauling abnormal data in topology data of gas pipe network

Info

Publication number: CN117290798B
Application number: CN202311575145.XA
Authority: CN
Inventors: 魏海东; 栾星; 王子峥; 石岱辉; 陈宝; 周乐乐
Original assignee: Shanghai Three Zero Four Zero Technology Co ltd
Current assignee: Shanghai Three Zero Four Zero Technology Co ltd
Priority date: 2023-11-24
Filing date: 2023-11-24
Publication date: 2024-02-06
Anticipated expiration: 2043-11-24
Also published as: CN117290798A

Abstract

The application provides a method, a system and a chip for overhauling abnormal data in topology data of a gas pipe network, wherein the method comprises the following steps: obtaining topology data of a gas pipe network and carrying out standardization processing on the topology data to obtain standardized topology data, wherein the topology data comprises point table data and line table data; performing anomaly detection on the standardized topology data, screening out the anomaly data and generating corresponding anomaly record information, wherein the anomaly detection comprises one or more of the following steps: pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection; repairing the abnormal data according to the abnormal record information, wherein ID abnormal and coordinate abnormal repair are used as pre-requisite conditions for pipe diameter mutation repair, and the attribute and topology data of the gas pipe network are updated. According to the method and the device, the attribute and the topology data in the gas pipe network are automatically checked and repaired, so that the efficiency is improved, and the time cost and the labor cost of data processing are reduced.

Description

Method, system and chip for overhauling abnormal data in topology data of gas pipe network

Technical Field

The application relates to the technical field of gas pipe network data information, in particular to a method, a system and a chip for overhauling abnormal data in topology data of a gas pipe network.

Background

The urban fuel pipe network, the pipeline and the auxiliary facilities have strong concealment and high safety, and meanwhile, the pipe network terminal is connected with tens of thousands of nodes such as air sources, users, voltage regulating stations, valve wells, plugs, ball valves and the like, so that the urban fuel pipe network is huge and complex; the topology data of a network is a variety of sources including, but not limited to: an analyzer (chromatograph, etc.), a flowmeter (turbine, ultrasonic wave, millimeter wave), a bromine adding meter, a sensor, equipment (pressure maintaining, pressure remote transmission, gas internet of things), etc.

The topology data is generally composed of point table data and line table data, wherein the point table comprises various types of nodes in the topology, and each node has the attributes of node ID, position, name, attribute, type, coordinate, height and the like; the wire list includes all the pipes connected to the node, and each pipe has the attributes of pipe ID, pipe length, pipe diameter, wall thickness, coordinates, inflow endpoint, outflow endpoint and the like.

The complex topological structure and various data sources enable deviation between pipe network and pipeline data and real data, so that the pipeline integrity management of the urban fuel pipe network is affected, the accuracy of the urban fuel pipe network simulation calculation is also affected, and the process of intelligent city management is further hindered; in order to advance the smart city management process in China, the data management of the city fire control network and the city fire control pipeline needs to be completed, which is a leading link for the subsequent data restoration and the realization of the smart city construction.

All attributes of the point table and the line table should be complete and accurate, the current mainstream data inspection and repair mode is that the inspection and the screening are performed manually, the inspection and the design of the treatment mode are further performed, and finally the manual unified treatment is performed, so that the defects of long time consumption, high cost, low efficiency, high miss-inspection rate, high error rate and the like are overcome along with the increase of the data quantity.

Based on the method, the system and the chip for overhauling abnormal data in the topology data of the gas pipe network are provided, so that the prior art is improved.

Disclosure of Invention

The utility model aims to provide an overhaul method, a system and a chip for abnormal data in intelligent gas pipe network topology data, which can automatically check and repair the topology data in the gas pipe network, improve the efficiency and reduce the time cost and the labor cost of data processing.

The purpose of the application is realized by adopting the following technical scheme:

in a first aspect, the present application provides a method for overhauling abnormal data in topology data of a gas pipe network, where the method includes:

obtaining topology data of a gas pipe network and carrying out standardization processing on the topology data to obtain standardized topology data, wherein the topology data comprises point table data and line table data;

Performing anomaly detection on the standardized topology data, screening out the anomaly data and generating corresponding anomaly record information, wherein the anomaly detection comprises one or more of the following steps: pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection;

and repairing the abnormal data according to the abnormal record information, and updating the topology data of the gas pipe network.

In some alternative embodiments, the normalizing the topology data includes:

for the point table data, de-duplicating the node_id field, and rounding the non-integer ID mark in the node_id field; reserving m-bit decimal for the x_cordinate field and the y_cordinate field, wherein m is a positive integer;

performing deduplication on the pipe_id field and rounding the pipe_id field, the source field and the target field aiming at the table data; m-bit decimal is reserved for the x_source field, the y_source field, the x_target field and the y_target field, n-bit decimal is reserved for the pipe_diameter field, and n is a positive integer.

In some alternative embodiments, the anomaly detection comprises ID anomaly detection, the anomaly data comprises ID anomaly data, and the anomaly record information comprises an ID anomaly form;

the detecting the abnormality of the standardized topology data, screening out the abnormality data and generating the corresponding abnormality record information includes:

Performing row and column splicing on an x_source field and a y_source field in line table data to obtain a source end coordinate mark, and marking the source end coordinate mark as an xy_source field;

performing row and column splicing on an x_target field and a y_target field in line table data to obtain a target end coordinate mark, and marking the target end coordinate mark as an xy_target field;

splicing a pipe_ id, source, xy _source three-column field and a pipe_ id, target, xy _target three-column field in line table data according to rows, sequentially obtaining an id field, a st field and an xy field, generating a temporary table according to the id field, the st field and the xy field, and performing de-duplication on all rows in the temporary table;

taking the st field as a grouping condition, and grouping and splitting the temporary form to obtain a plurality of sub-forms with different st fields, wherein the st field is the same in each sub-form and the xy field is different;

counting the number of lines in each sub-list, taking node ID identifiers corresponding to st fields with the number of lines larger than 1 as ID abnormal data, and generating a corresponding ID abnormal list according to the ID abnormal data.

In some optional embodiments, the repairing the abnormal data according to the abnormal record information includes:

reading the ID abnormality LIST to obtain an ID abnormality LIST, and marking the ID abnormality LIST as ST_LIST;

Listing ID identifications corresponding to a source field and a target field in line table data into a first set, and screening out the largest ID identification from the first set, and marking the largest ID identification as st_max;

inserting two columns of fields into the line table data, respectively marking the two columns as a source_flag field and a target_flag field, and initializing the values of the source_flag field and the target_flag field to 0;

judging whether an ID (identity) corresponding to a source field is in ST_LIST (sequence table) or not for each row of the table data, if so, setting the source_flag field to be 1, otherwise, setting the source_flag field to be 0; judging whether an ID (identity) corresponding to a target field is in ST_LIST, if so, setting the target_flag field to be 1, otherwise, setting the target_flag field to be 0;

inserting a list of fields into the line table data, and recording the list of fields as st_tag fields, wherein if the source_flag field and/or the target_flag field in the line table data are 1, the st_tag field is set to 1, and if the source_flag field and the target_flag field in the line table data are 0, the st_tag field is set to 0;

dividing the line table into a first sub-table to be processed and a second sub-table not to be processed based on the st_tag field, wherein the st_tag field of the first sub-table is 1, and the st_tag field of the second sub-table is 0;

For the first sub-list, replacing an ID identifier corresponding to a source field with a source_flag field of 1 and an ID identifier corresponding to a target field with a target_flag field of 1 with st_max according to the sequence from top to bottom of each row and the sequence from the source field to the target field of each row, and adding 1 to st_max and filling when replacing each time;

splicing the processed first sub-list with a second sub-list which does not need to be processed to obtain processed line list data;

generating a node_id field of the point table according to a source field or a target field of the line table data, generating an x_coordinate field of the point table according to an x_source field and an x_target field of the line table data, and generating a y_coordinate field of the point table according to a y_source field and a y_target field of the line table data to obtain a new point table, and warehousing and storing the new point table.

In some alternative embodiments, the anomaly detection comprises coordinate anomaly detection, the anomaly data comprises coordinate anomaly data, and the anomaly record information comprises a coordinate anomaly form;

taking the xy field as a grouping condition, and grouping and splitting the temporary form to obtain a plurality of sub-forms with different xy fields, wherein the xy field is the same in each sub-form and the st field is different;

counting the number of lines in each sub-list, taking coordinate identifications corresponding to xy fields with the number of lines being more than 1 as coordinate abnormal data, and sequencing st fields of the sub-list from small to large;

and acquiring an xy field and a st field of the first coordinate exception data of the sub-table to generate a corresponding coordinate exception table.

Reading the coordinate anomaly LIST to obtain an ID anomaly LIST corresponding to the coordinate anomaly LIST, and respectively marking the ID anomaly LIST as XY_LIST and ST_LIST;

judging whether a coordinate identifier corresponding to an xy_source field is in an xy_list or not according to each line of the table data, if so, setting the source_flag field to be 1, otherwise, setting the source_flag field to be 0; judging whether the coordinate identifier corresponding to the xy_target field is in the xy_list, if so, setting the target_flag field to be 1, otherwise, setting the target_flag field to be 0;

for the first sub-LIST, replacing the ID identifications corresponding to the source field with the source_flag field being 1 and the ID identifications corresponding to the target field with the target_flag field being 1 with the ID identifications corresponding to the coordinate identifications of the nodes in ST_LIST according to the sequence from top to bottom of each row and the sequence from the source field to the target field of each row;

Merging the processed first sub-table and the second sub-table which does not need to be processed into line table data;

generating a node_id field of the point table according to a source field or a target field of the line table data, generating an x_coordinate field of the point table according to an x_source field and an x_target field of the line table data, generating a y_coordinate field of the point table according to a y_source field and a y_target field of the line table data, so as to obtain repaired point table data, and warehousing and storing the point table data and the line table data.

In some optional embodiments, the anomaly detection includes pipe diameter mutation detection, the anomaly data includes pipe diameter anomaly data, and the anomaly record information includes pipe diameter anomaly forms;

constructing a gas pipe network topology by taking a source field of line table data as a starting point, a target field as an ending point and a pipe_id field and a pipe_diam field as pipeline attributes;

judging whether the number of nodes in the connected subgraph is less than 3 according to each connected subgraph in the gas pipe network topology, if so, determining that the connected subgraph has no pipe diameter mutation; if the diameter of the communication subgraph is not less than 3, the pipe diameter mutation exists in the preset communication subgraph;

Marking all nodes with the channel number of 2 in a preset connected subgraph with pipe diameter mutation as abnormal nodes, calculating absolute values of pipe diameter difference values between two adjacent pipelines of the abnormal nodes aiming at each abnormal node, and if the absolute values are larger than a preset mutation threshold value, listing the abnormal nodes into a preset abnormal node list;

judging whether a single pipeline is connected between every two nodes in the abnormal node list, and if so, determining that the single pipeline is a mutation pipeline;

and taking the pipeline ID of the abrupt change pipeline as pipe diameter abnormal data, and generating a corresponding pipe diameter abnormal form according to the pipe diameter abnormal data.

reading the pipeline ID of each mutation pipeline in the pipe diameter abnormal form, and generating a mutation pipeline ID set;

acquiring the pipe_id field and the pipe_diam field of two adjacent pipes of each abrupt pipe, and respectively judging whether the pipe ID corresponding to the pipe_id field of each adjacent pipe is in an abrupt pipe ID set;

if the pipe ID corresponding to the pipe_id field of the adjacent pipe of the abrupt pipe is not in the abrupt pipe ID set, replacing the pipe_diam field of the abrupt pipe with the pipe_diam field of the adjacent pipe, and removing the pipe ID of the abrupt pipe from the abrupt pipe ID set;

If the pipe IDs corresponding to the pipe ID fields of two adjacent pipes of the abrupt change pipe are not in the abrupt change pipe ID set, the abrupt change pipe is post-processed until all the pipe IDs in the abrupt change pipe ID set are removed.

In a second aspect, the present application provides a system for overhauling abnormal data in topology data of a gas pipe network, the system comprising:

the standardized module is used for acquiring topology data of the gas pipe network and carrying out standardized processing on the topology data to obtain standardized topology data, wherein the topology data comprises point table data and line table data;

the abnormality detection module is used for carrying out abnormality detection on the standardized topology data, screening out the abnormal data and generating corresponding abnormality record information, and the abnormality detection comprises one or more of the following steps: pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection;

and the abnormality repairing module is used for repairing the abnormal data according to the abnormal record information and updating the topology data of the gas pipe network.

In a third aspect, the present application provides a chip comprising: a processor for calling and running a computer program from a memory, causing a device on which the chip is mounted to execute: a method as in any above.

The technical scheme provided by the application has the following beneficial effects: the topology data in the urban natural gas pipe network Geographic Information System (GIS) is automatically checked and repaired for pipe diameter mutation, coordinate abnormality and ID abnormality, so that the efficiency is improved, the time cost and the labor cost of data processing are reduced, a foundation is laid for natural gas pipe network simulation, and the promotion of digital operation of the natural gas pipe network is accelerated.

Drawings

The present application is further described below with reference to the drawings and examples.

Fig. 1 is a schematic flow chart of a method for overhauling abnormal data in topology data of a gas pipe network according to an embodiment of the present application.

Fig. 2 is a schematic diagram of an overall repair process of topology data of a gas pipe network according to an embodiment of the present application.

Fig. 3 is a schematic flow chart of ID anomaly detection according to an embodiment of the present application.

Fig. 4 is a schematic flow chart of repairing ID anomalies according to an embodiment of the present application.

Fig. 5 is a schematic flow chart of a coordinate anomaly detection provided in an embodiment of the present application.

Fig. 6 is a schematic flow chart of repairing coordinate anomalies according to an embodiment of the present application.

Fig. 7 is a schematic flow chart of pipe diameter mutation inspection according to an embodiment of the present application.

Fig. 8 is a schematic flow chart of repairing pipe diameter mutation according to an embodiment of the present application.

Detailed Description

In order to make the technical means, the inventive features, the achievement of the purpose and the effect achieved by the present application easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the specific drawings, and it is apparent that the described embodiments are some embodiments of the present application, not all embodiments.

All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.

It should be understood that the structures, proportions, sizes, etc. shown in the drawings are shown only in connection with the present disclosure, and are therefore not intended to limit the scope of the invention, which is defined by the claims, but are not to be limited to the specific details disclosed herein.

Also, the terms such as "upper," "lower," "left," "right," "middle," and "a" and the like recited in the present specification are merely for descriptive purposes and are not intended to limit the scope of the present application to which they may be applied, but rather to modify or adapt the relative relationship without materially altering the technical context.

Term interpretation:

GIS: urban gas pipe network geographic information system;

dot list: node information (node_ id, xcoordinate, ycoordinate) in a geographic information system GIS of the urban gas network;

line list: pipe information (pipe_ id, source, target, x _source, y_source, x_target, y_target, x_source, y_source, x_target, y_target, pipe_diam) in the urban gas pipe network geographic information system GIS;

ID anomaly: one endpoint ID in the line table corresponds to a plurality of different endpoint coordinates in the pipeline;

coordinate anomaly: one endpoint coordinate in the line table corresponds to a plurality of different endpoint IDs in the pipeline;

pipe diameter mutation: taking any one pipeline as a target pipeline, wherein the occurrence of pipe diameter mutation of the target pipeline means that the number of channels of nodes at two ends of the target pipeline is 2, and the difference value of the pipe diameters of the target pipeline and the pipeline connected with any one end is larger than a preset mutation threshold value;

node_id: node identification (i.e., ID identification of the node);

pipe_id: a pipeline identifier;

source: a pipeline starting node mark;

target: a pipeline termination node identification;

x_incorporation: the x coordinates of the point table;

y_incorporation: the y coordinate of the point table;

x_source: x coordinates of the starting point of the line table;

y_source: the y coordinate of the starting point of the line table;

xy_source: the coordinate unique identifier after the x coordinate and the y coordinate of the source end of the line table are spliced;

x_target: x coordinate of the line table end point;

y_target: the y coordinate of the line table end point;

xy_target: the coordinate of the target end of the line table after the x coordinate and the y coordinate are spliced is uniquely identified;

pipe_diameter: the pipe diameter length of the pipeline.

All the attributes of the point table and the line table should be complete and accurate, except that the ID identifier and the space coordinate position have a unique corresponding relationship, in pipe network construction, the number of channels of a node refers to the number of pipes connected by the node, the plurality of pipes connected by the node are connected by pipe connectors for the node with the number of channels being greater than or equal to three, and usually, the pipe diameter difference between the pipes connected by the node is the same or smaller under normal conditions for the node with the number of channels being 2, if the pipe diameter difference between the pipes connected by any one end of the two ends of the pipe is greater than a preset mutation threshold (for example, 1cm, 2cm or 3 cm), the pipe diameter mutation is indicated. Thus, a pipe diameter mutation is defined as: taking any one pipeline as a target pipeline, wherein the occurrence of pipe diameter mutation of the target pipeline means that the number of pipeline channels at two ends of the target pipeline is 2, and the pipe diameter difference value of the pipelines connected with any one end is larger than a preset mutation threshold value; at present, the main stream processing mode is firstly manual checking and screening, further checking and designing a treatment mode, and finally manual unified processing, and has the defects of long time consumption, high cost, lower efficiency, increased miss rate and error rate along with the increase of data quantity and the like; the QGIS system can be used for checking topology data, but only the condition of a certain specific pipeline or a certain specific node can be visualized, and the states of all abnormal data and pipe diameter mutation cannot be intuitively reflected; the abnormal data form in the topology data cannot be listed.

In the early stage, in order to solve the problems of error, missing, incapability of mapping and the like of repairing topological data, the defects can only be checked and repaired manually by a full-time staff, and the aging requirement of data management can not be met in terms of accuracy and efficiency in the mode; with the continuous development of computer technology, an ARCGIS system (an infrastructure for drawing maps and geographic information) is generally used for assisting in manually repairing topology data, but when the data volume is large, the error rate and the missed rate of the manual repair are increased, and meanwhile, the requirement on personnel capacity is high.

At present, the repair is popular by manual work and cooperation with an ARCGIS system, namely, the distribution condition in the ARCGIS system is checked by a mere human eye and then repaired, and the repair speed is low, the efficiency is low and the error rate is high. In addition, the problems of abnormal ID, abnormal coordinates and abrupt change of pipe diameters of GIS data can be systematically solved by means of a partially automatic repair tool, such as EXCEL and the like, but the GIS data is only used as a local technical scheme from the aspects of a system and the whole.

Based on the method, the system and the chip for overhauling abnormal data in the intelligent gas pipe network topology data are provided, so that the defects of the prior art are overcome, and the repairing precision and efficiency are improved. Compared with manual repair, from the aspects of the whole and the system, the method has the advantages of high result accuracy, high efficiency, high standardized module degree and short processing time.

Method embodiment

Referring to fig. 1, fig. 1 is a flow chart of a method for overhauling abnormal data in topology data of a gas pipe network according to an embodiment of the present application.

The method comprises the following steps:

step S101: obtaining topology data of a gas pipe network and carrying out standardization processing on the topology data to obtain standardized topology data, wherein the topology data comprises point table data and line table data;

step S102: performing anomaly detection on the standardized topology data, screening out the anomaly data and generating corresponding anomaly record information, wherein the anomaly detection comprises one or more of the following steps: the method comprises pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection, wherein any one pipe is used as a target pipe, and pipe diameter mutation of the target pipe means that the number of channels of nodes at two ends of the target pipe is 2, and the difference value of pipe diameters of the target pipe and the pipes connected with any one end is larger than a preset mutation threshold value;

step S103: and repairing the abnormal data according to the abnormal record information, and updating the topology data of the gas pipe network.

It should be noted that: in the embodiment of the application, the detection sequence of pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection is not limited, and pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection may be sequentially performed.

In one embodiment, in step S101, the normalizing the topology data includes:

de-duplicating the pipe_id field, and rounding non-integer ID identifications in the pipe_id field, the source field and the target field aiming at the table data; m-bit decimal is reserved for the x_source field, the y_source field, the x_target field and the y_target field, n-bit decimal is reserved for the pipe_diameter field, and n is a positive integer.

The embodiment of the application does not limit m and n, m is, for example, 2, 3, 5, 7 or 8, and n is, for example, 2, 3, 5, 7 or 8.

As an example, for normalized point table data, each node_id field has and only has a unique non-repeating integer ID, and the x_coordinate field and the y_coordinate field are longitude and latitude data retaining 3-bit decimal;

for standardized line table data, the pipe_id field, the source field and the target field are integer IDs and the pipe_id field is unique and not repeated, the x_source field, the y_source field, the x_target field and the y_target field are longitude and latitude data reserved for 3-bit decimal, and the pipe_diameter field is reserved for 3-bit decimal pipe diameter data.

Referring to fig. 2, fig. 2 is a schematic diagram of an overall repair process of topology data of a gas pipe network according to an embodiment of the present application.

Specifically, the repair process of the topology data of the gas pipe network is as follows (a-h):

a. acquiring the standardized point table and line table data;

b. checking the data of the ID abnormality and generating a form:

c: solving the data problem of the ID abnormality and updating the dotted line table;

d: checking out data of abnormal coordinates and generating a form;

e: solving the data problem of abnormal coordinates and updating the dotted line table;

f: checking the data of pipe diameter mutation and generating a form;

g: solving the data problem of pipe diameter mutation and updating the line table;

h: and saving the point table and the line table repair data.

According to the embodiment of the application, the topology data in the Geographic Information System (GIS) of the urban natural gas pipe network is automatically checked and repaired in the manners of pipe diameter mutation, ID abnormality and coordinate abnormality, so that the efficiency is improved, the time cost and the labor cost of data processing are reduced, a foundation is laid for natural gas pipe network simulation, and the promotion of digital operation of the natural gas pipe network is accelerated.

In one embodiment, the anomaly detection includes an ID anomaly detection, the anomaly data includes ID anomaly data, and the anomaly record information includes an ID anomaly form;

In step S102, abnormality detection is performed on the standardized topology data, and abnormality data is screened out and corresponding abnormality record information is generated, including:

Due to omission caused by manual transcription and the lag of information data technology during early pipe network establishment, a source field or a target field in the line table data is the same ID but corresponds to a plurality of different coordinates in the topology; this makes it impossible for a user to locate the position of the pipe in the topology by the pipe end ID, and also presents great difficulty for the pipe network simulation calculation. Therefore, in order to solve the problem of abnormal ID of topology data in a Geographic Information System (GIS) of an urban natural gas network, the application specifically performs ID abnormality inspection on pipeline data in the topology, relates to identification ID and coordinate data of the pipeline, and comprises seven fields in a line table: pipe_ id, source, target, x _source, y_source, x_target, y_target. The method and the device adopt modes of inserting coordinate identification, generating forms, grouping, screening and the like to check the ID abnormal data.

Referring to fig. 3, fig. 3 is a schematic flow chart of an ID anomaly detection provided in an embodiment of the present application.

Specifically, the ID anomaly checking process is as follows (b 1 to b 6):

b1: and obtaining the standardized line table data.

b2: splicing source end coordinate data x_source field and y_source field in the line table with "_as separator to form source end coordinate unique mark which is marked as xy_source; the target end-position data x_target field and the y_target field are spliced into a target end-position unique identifier by taking "_" as a separator, and are marked as xy_target.

For example, the normalized line table data is shown in table 1, and the line table data after concatenating x_source and y_source, x_target and y_target is shown in table 2. The coordinate uniqueness mark after the pipe_id is 1 and the source end ID is 1 is 376182.563_547951.997; similarly, the unique identification of the target end coordinate after merging the pipe_id of 1 and the target end ID of 2 is 376197.392_547981.453.

Table 1:

table 2:

b3: splicing the three columns of fields of the pipe_ id, source, xy _source and the three columns of fields of the pipe_ id, target, xy _target in the line table data according to the rows, renaming the spliced three columns of names into an id field, a st field and an xy field, generating a temporary table, and de-duplicating the rows with identical pipe_id field, st field and xy field in the temporary table.

For example, before row-wise concatenation, the three columns of the field pipe_ id, source, xy _source are shown in Table 2-1 and the three columns of the field pipe_ id, target, xy _target are shown in Table 2-2. The three columns of fields, i.e., id field, st field and xy field, after row-wise concatenation are shown in tables 2-3.

Table 2-1:

table 2-2:

table 2-3:

b4: and carrying out grouping splitting on the temporary form by taking st as a grouping condition in the temporary form so as to obtain each sub-form with different st fields, wherein in each sub-form, the st fields are identical, and the xy fields are different.

b5: counting the number of lines in each sub-list, if the number of lines is greater than 1, indicating that the same identifier at the tail end of a pipeline in the sub-list corresponds to a plurality of pipeline tail end nodes, and that the ID of the sub-list is abnormal; if the number of the rows is equal to 1, indicating that the pipeline end node identification in the sub-list is uniquely corresponding to the coordinates, and that the sub-list ID is normal; all the st end identifiers with the number of rows greater than the number of rows are reserved.

b6: and generating and storing an ID abnormal form.

In one embodiment, in step S103, repairing the abnormal data according to the abnormal record information includes:

By adopting the ID abnormality detection mode, the data list of all the ID abnormalities in the outgoing line list can be checked, but the manual processing mode is only used, which is time-consuming and labor-consuming. In order to ensure quick and accurate topology management, automatic repair of the data of the ID abnormality in the line list is required based on the checking result of the ID abnormality, and point list data is updated based on the result of line list management.

Referring to fig. 4, fig. 4 is a schematic flow chart of repairing an ID anomaly according to an embodiment of the present application.

Specifically, the process of repairing an ID anomaly is as follows (c 1-c 9):

c1: and acquiring the standardized point table and line table data.

c2: reading the ID anomaly LIST to obtain an ID anomaly LIST, which is denoted as ST_LIST.

c3: and arranging all pipeline IDs of two columns of a source field and a target field of the line table into a set, and then screening out the largest pipeline ID from the set, and marking the pipeline ID as st_max for filling the serial numbers of the ID anomalies.

c4: two columns of marks are inserted into the line table and marked as a source_flag field and a target_flag field, and the two columns of values are initialized to 0, which indicates that the IDs of the source end and the target end are not ID abnormal data.

c5: sequentially taking out the ID identifications of the source field and the target field of each row from the line table, setting the source_flag to 1 if the ID identification of the source field is in ST_LIST, otherwise setting the source_flag to 0; if the ID of the target field is identified in ST_LIST, the target_flag is set to 1, otherwise, 0.

c6: and a column of field for marking and screening the ID abnormal row data is inserted into the line table and is recorded as st_tag, if source_flag or target_flag in the line table is 1, the st_tag is set to 1, and if source_flag and target_flag in the line table are both 0, the st_tag is set to 0.

And (3) searching a sub-table with source_flag or target_flag of 1 from the line table data through the st_tag field, and marking the sub-table as a first sub-table.

c7: and replacing and filling the source field with the source_flag of 1 and the target field with the target_flag of 1 with st_max according to the sequence from top to bottom of the rows and the sequence from the source field to the target field of each row in the first sub-list, and adding 1 to st_max and filling each time of operation.

c8: and sequentially taking out the ID identifications of the source field or the target field in the line table and the X coordinate and the Y coordinate according to the rows to generate a new point table by taking the source or target end ID as the node_id and taking the x_source, the y_source, the x_target and the y_target as the point table X coordinate and the point table Y coordinate.

c9: and warehousing and storing the repaired point table and line table data.

The dot table data after the completion of the ID anomaly handling is shown in table 3, and the line table data after the completion of the ID anomaly handling is shown in table 4.

Table 3:

table 4:

in one embodiment, the anomaly detection comprises coordinate anomaly detection, the anomaly data comprises coordinate anomaly data, and the anomaly record information comprises a coordinate anomaly form;

Although the ID anomaly checking and repairing can solve the problem that an end ID of a pipeline in a pipe network corresponds to a plurality of coordinates, due to the complexity of topology, the problem that an end point coordinate corresponds to a plurality of different source or target end IDs exists at the end of the pipeline. To repair these abnormal data points, the present application uses the methods of inserting coordinate identifiers, generating forms, grouping, screening, etc. to examine these abnormal data points.

Referring to fig. 5, fig. 5 is a schematic flow chart of a coordinate anomaly detection provided in an embodiment of the present application.

Specifically, the process of the coordinate anomaly inspection is as follows (d 1 to d 6):

d1: and obtaining the standardized line table data.

d2: splicing source end coordinate data x_source field and y_source field in the line table with "_as separator to form source end coordinate unique mark which is marked as xy_source; the target end-position data x_target field and the y_target field are spliced into a target end-position unique identifier by taking "_" as a separator, and are marked as xy_target.

d3: splicing the three columns of fields of the pipe_ id, source, xy _source and the three columns of fields of the pipe_ id, target, xy _target in the line table data according to the rows, renaming the spliced three columns of names into an id field, a st field and an xy field, generating a temporary table, and de-duplicating the rows with identical pipe_id field, st field and xy field in the temporary table.

d4: and carrying out grouping splitting on the temporary form by taking xy as a grouping condition in the temporary form so as to obtain each sub-form with different xy fields, wherein in each sub-form, the xy fields are the same, and the st fields are different.

d5: counting the number of lines in each sub-list, if the number of lines is greater than 1, indicating that the same coordinates of the tail ends of the pipelines in the sub-list correspond to nodes marked by the tail ends of a plurality of pipelines, and the coordinates of the sub-list are abnormal; if the number of the rows is equal to 1, the fact that the coordinates of the tail ends of the pipelines in the sub-list are uniquely corresponding to the source or target endpoint ID is indicated, and the coordinates of the sub-list are normal; and reserving all xy coordinate identifications with the row number larger than 1.

d6: and generating and storing a coordinate anomaly form.

reading the coordinate anomaly LIST to obtain a coordinate anomaly LIST and a corresponding ID anomaly LIST, which are respectively marked as XY_LIST and ST_LIST;

dividing the line table into two sub-tables which need to be processed and do not need to be processed based on the st_tag field;

For the first sub-LIST, respectively replacing the ID identifications corresponding to the source field with the source_flag field being 1 and the ID identifications corresponding to the target field with the target_flag field being 1 with the ID identifications corresponding to the coordinate identifications of the nodes in ST_LIST according to the sequence from top to bottom of each row and from the source field to the target field of each row;

By adopting the coordinate anomaly detection mode, the data list of all coordinate anomalies in the wire list can be checked, but the condition that one coordinate corresponds to a plurality of IDs is still time-consuming and labor-consuming only by manually processing, the data of the coordinate anomalies in the wire list needs to be automatically repaired based on the checking result of the coordinate anomalies, and meanwhile, the point list data is updated based on the result of wire list management.

Referring to fig. 6, fig. 6 is a schematic flow chart for repairing coordinate anomalies according to an embodiment of the present application.

Specifically, the process of repairing the coordinate anomaly is as follows (e 1-e 9):

e1: and acquiring the standardized point table and line table data.

e2: and reading the coordinate anomaly LIST to obtain a coordinate anomaly LIST and a corresponding ID anomaly LIST, and marking the coordinate anomaly LIST as XY_LIST and ST_LIST.

And e3: splicing source end coordinate data x_source field and y_source field in the line table with "_as separator to form source end coordinate unique mark which is marked as xy_source; the target end-position data x_target field and the y_target field are spliced into a target end-position unique identifier by taking "_" as a separator, and are marked as xy_target.

And e4: two columns of marks are inserted into the line table and marked as a source_flag field and a target_flag field, and the two columns of values are initialized to 0, so that the xy_source field and the xy_target field are not coordinate abnormal data.

And e5: sequentially taking out the coordinate uniqueness identifications of each row of xy_source and xy_target from the line table, judging whether the coordinate identifications of the xy_source end xy_source and the xy_target end xy_target exist in the xy_list, setting the source_flag to be 1 if the xy_source exists in the xy_list, and otherwise setting the source_flag to be 0; if the coordinate identification of the target end xy_target exists in xy_list, the target_flag is set to 1, otherwise, is set to 0.

e6: and a column of field st_tag of the coordinate abnormal row data is inserted into the line table, if the source_flag or the target_flag in the line table is 1, the st_tag is set to be 1, otherwise, the st_tag is set to be 0.

And (5) searching out a sub-list with source_flag or target_flag of 1 from the line list data through the st_tag field.

e7: dividing the line table into a first sub-table to be processed and a second sub-table not to be processed, wherein the st_tag field of the first sub-table is 1, and the st_tag field of the second sub-table is 0;

for a first sub-table, marking nodes corresponding to a source field with a source_flag field of 1 and a target field with a target_flag field of 1 as target nodes, and replacing the ID identification of each target node with the ID identification corresponding to the coordinate identification of the target node in ST_LIST according to the sequence from top to bottom of each row and from the source field to the target field of each row;

merging the processed first sub-table and the second sub-table which does not need to be processed into line table data; .

e8: and sequentially taking out an ID (identity) mark of a source or a target in the line table and an X coordinate and a Y coordinate according to rows to generate a new point table by taking the ID of a source or a target end as a node_id and taking x_source, y_source, x_target and y_target as X coordinate and Y coordinate of the point table.

e9: and warehousing and storing the repaired point table and line table data.

In one embodiment, the anomaly detection comprises pipe diameter mutation detection, the anomaly data comprises pipe diameter anomaly data, and the anomaly record information comprises a pipe diameter anomaly form;

The mutation threshold is not limited in this application, and may be, for example, 20, 30, 40, or 50.

The pipe diameter mutation means that in the pipe network topology, pipe diameters of pipe sections connected between nodes connected with three pipes in the middle are the same or have smaller difference; however, due to the negligence of manual transcription and the lack of original data, a lot of data which does not meet the requirements is also filled in the line table, so that the difference between the simulation calculation result of the pipe network and the real acquisition value is large. In order to automatically repair the data of the pipe diameter mutation, necessary measures are taken to check the pipe data of the pipe diameter mutation in the outlet meter.

Referring to fig. 7, fig. 7 is a schematic flow chart of a pipe diameter mutation check provided in an embodiment of the present application.

Specifically, the pipe diameter mutation inspection process is as follows (f 1-f 6):

f1: and obtaining the standardized line table data and the Mutation threshold value threshold.

The normalized line table data is shown in table 5.

Table 5:

f2: and taking a source end in the line table as a starting point, a target end as an ending point, and pipe_id and pipe_diameter as pipeline attributes to construct the undirected topology of the gas pipe network.

f3: acquiring each connected subgraph in the gas pipe network topology, judging whether the number of nodes in each connected subgraph is smaller than 3, if so, indicating that only one pipeline exists in the connected subgraph, and no pipe diameter mutation exists; if not less than (more than or equal to) 3, acquiring all nodes with the channel number of 2 under the connected subgraph.

f4: in a connected subgraph, two pipelines adjacent to the node with the channel number of 2 and the pipeline ID and the pipeline diameter size of the two pipelines are obtained, the pipeline diameters of the two pipelines are directly subjected to difference, the absolute value is taken and compared with the Mutation threshold value, if the absolute value is larger than the Mutation threshold value, the two pipelines are reserved, otherwise, the next node is continuously judged until all the nodes with the channel number of 2 are traversed.

f5: counting node lists of all pipe diameter mutation, judging whether single pipe sections are connected between every two nodes in the list, and if so, counting pipe_id lists of all mutation pipe sections;

f6: and generating and storing a pipe diameter mutation form.

By adopting the pipe diameter mutation detection mode, the data list of all pipe diameter mutations in the outlet table can be checked, but only the manual mode is adopted to traverse all the pipelines in the pipe diameter mutation detection list and repair the pipelines, so that time and labor are still wasted. In order to ensure quick and accurate topology repair, it is necessary to automatically repair the mutated pipeline data in the wire list based on the inspection result of the pipe diameter mutation, and update the wire list data.

Referring to fig. 8, fig. 8 is a schematic flow chart for repairing a pipe diameter mutation according to an embodiment of the present application.

Specifically, the process of repairing the pipe diameter mutation is as follows (g 1-g 4):

g1: and obtaining the standardized line table data.

g2: and sequentially reading each pipeline ID in the pipe diameter mutation check list, and generating a mutation pipeline ID set.

g3: and acquiring two adjacent pipelines of each abrupt pipeline, and a pipe_id field and a pipe_diameter field of the adjacent pipelines, and judging whether the pipe_id field of each adjacent pipeline exists in the abrupt pipeline ID set.

If so, continuing to judge whether the next adjacent pipeline exists in the abrupt pipeline ID set;

if not, replacing the pipe diameter of the abrupt change pipe with the pipe diameter of the adjacent pipe, and removing the abrupt change pipe ID from the abrupt change pipe ID set;

if none of the two adjacent pipe IDs of the abrupt pipe ID are in the abrupt pipe ID set, the abrupt pipe ID is placed in final processing until all pipe IDs in the abrupt pipe ID set are removed.

g4: and when no pipeline ID exists in the abrupt change pipeline ID set, warehousing and storing the repaired line table data.

The method is applied to solving the problems of pipe diameter mutation, ID and coordinate abnormality in the GIS data management process, and the problems are seriously influenced and then simulation calculation is initiated. In order to find out ID and coordinate abnormal data in GIS topology, technical means such as inserting a coordinate unique identifier, grouping screening and the like are used for checking GIS data, 2, in order to repair the abnormal data of ID abnormality, coordinate abnormality and pipe diameter mutation, the abnormal data problem is solved by adopting technologies such as establishing a multi-condition judgment flow process, and 3, in order to solve the abnormal data problem rapidly, checking and repairing are performed in a flow and modularization mode, GIS repairing is systematically performed by means of ID abnormality checking, ID abnormality repairing, coordinate abnormality checking, coordinate abnormality repairing, pipe diameter mutation checking, pipe diameter mutation repairing and the like, and relatively perfect pipe network data can be obtained. Compared with manual repair, from the aspects of the whole and the system, the method has the advantages of high result accuracy, high efficiency, high standardized module degree and short processing time.

System embodiment

Based on the method for overhauling the abnormal data in the gas pipe network topology data, the application also provides an overhauling system for the abnormal data in the gas pipe network topology data, which comprises the following steps:

Chip embodiment

Based on the method for overhauling abnormal data in the topology data of the gas pipe network, the application also provides a chip, which comprises the following steps: and the processor is used for calling and running the computer program from the memory, so that the equipment provided with the chip executes the method for overhauling the abnormal data in the topology data of the gas pipe network.

The present application is directed to functional enhancement and use elements, which are emphasized by the patent laws, such as the description and drawings, of the present application, but are not limited to the preferred embodiments of the present application, and therefore, all equivalents and modifications, equivalents, and modifications, etc. of the structures, devices, features, etc. of the present application are included in the scope of the present application.

Claims

1. A method for overhauling abnormal data in topology data of a gas pipe network is characterized in that,

the method comprises the following steps:

obtaining topology data of a gas pipe network and carrying out standardization processing on the topology data to obtain standardized topology data, wherein the topology data comprises point table data and line table data which are related to each other, the point table data represent node information in a geographic information system of the urban gas pipe network, and the line table data represent pipeline information in the geographic information system of the urban gas pipe network;

performing anomaly detection on the standardized topology data, screening out the anomaly data and generating corresponding anomaly record information, wherein the anomaly detection comprises one or more of the following steps: the method comprises pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection, wherein any one pipe is used as a target pipe, and pipe diameter mutation of the target pipe means that the number of channels of nodes at two ends of the target pipe is 2, and the difference value of pipe diameters of the target pipe and the pipes connected with any one end is larger than a preset mutation threshold value;

Repairing the abnormal data according to the abnormal record information, and updating topology data of a gas pipe network;

the anomaly detection comprises pipe diameter mutation detection, the anomaly data comprises pipe diameter anomaly data, and the anomaly record information comprises a pipe diameter anomaly form;

taking the pipeline ID of the abrupt change pipeline as pipe diameter abnormal data, and generating a corresponding pipe diameter abnormal form according to the pipe diameter abnormal data;

wherein, pipe_id is used to indicate the ID identification of the pipe; the source is used for indicating a pipeline starting node identifier; the target is used for indicating the pipeline termination node identification; pipe_diameter is used for indicating the pipe diameter length of the pipeline;

the method further comprises the steps of:

and updating the point table data according to the repaired line table data to obtain the repaired point table data, and warehousing the point table data and the line table data.

2. The method for overhauling abnormal data in the topology data of the gas pipe network according to claim 1, wherein,

the standardized processing of the topology data comprises the following steps:

performing deduplication on the pipe_id field aiming at the table data, and rounding non-integer ID identifications in the pipe_id field, the source field and the target field respectively; reserving m-bit decimal for an x_source field, a y_source field, an x_target field and a y_target field, reserving n-bit decimal for a pipe_diam field, wherein n is a positive integer;

Wherein node_id is used to indicate the ID identification of the node; x_coordinate is used for indicating the x coordinate of the point table; y_coordinate is used for indicating the y coordinate of the point table; the x_source is used for indicating the x coordinate of the starting point of the line table; the y_source is used for indicating the y coordinate of the starting point of the line table; the x_target is used for indicating the x coordinate of the end point of the line table; y_target is used to indicate the line table endpoint y coordinate.

3. The method for overhauling abnormal data in the topology data of the gas pipe network according to claim 1, wherein,

the repairing of the abnormal data according to the abnormal record information includes:

4. The method for overhauling abnormal data in the topology data of the gas pipe network according to claim 1, wherein,

the anomaly detection comprises ID anomaly detection, the anomaly data comprises ID anomaly data, and the anomaly record information comprises an ID anomaly form;

Counting the number of lines in each sub-list, taking node ID identifications corresponding to st fields with the number of lines larger than 1 as ID abnormal data, and generating a corresponding ID abnormal list according to the ID abnormal data;

wherein, x_source is used for indicating the x coordinate of the starting point of the line table; the y_source is used for indicating the y coordinate of the starting point of the line table; the x_target is used for indicating the x coordinate of the end point of the line table; the y_target is used for indicating the end point y coordinate of the line table; the xy_source is used for indicating a coordinate unique identifier after x coordinates and y coordinates of the source end of the line table are spliced; the xy_target is used for indicating the unique identification of the coordinate after the x coordinate and the y coordinate of the target end of the line table are spliced.

5. The method for overhauling abnormal data in the topology data of the gas pipe network of claim 4, wherein,

6. The method for overhauling abnormal data in the topology data of the gas pipe network according to claim 1, wherein,

the anomaly detection comprises coordinate anomaly detection, the anomaly data comprises coordinate anomaly data, and the anomaly record information comprises a coordinate anomaly form;

acquiring xy fields and st fields of first coordinate exception data of the sub-table to generate a corresponding coordinate exception table;

7. The method for overhauling abnormal data in the topology data of the gas pipe network of claim 6, wherein the method comprises the steps of,

8. A maintenance system for abnormal data in topology data of a gas pipe network is characterized in that,

the system comprises:

the standardized module is used for acquiring topology data of the gas pipe network and carrying out standardized processing on the topology data to obtain standardized topology data, wherein the topology data comprises point table data and line table data which are mutually related, the point table data represent node information in a geographic information system of the urban gas pipe network, and the line table data represent pipeline information in the geographic information system of the urban gas pipe network;

The abnormality detection module is used for carrying out abnormality detection on the standardized topology data, screening out the abnormal data and generating corresponding abnormality record information, and the abnormality detection comprises one or more of the following steps: the method comprises pipe diameter mutation detection, ID abnormality detection and coordinate abnormality detection, wherein any one pipe is used as a target pipe, and pipe diameter mutation of the target pipe means that the number of channels of nodes at two ends of the target pipe is 2, and the difference value of pipe diameters of the target pipe and the pipes connected with any one end is larger than a preset mutation threshold value;

the abnormal repair module is used for repairing the abnormal data according to the abnormal record information and updating the topology data of the gas pipe network;

the abnormality detection module is used for:

the abnormal repair module is used for updating the point table data according to the repaired line table data to obtain the repaired point table data, and warehousing the point table data and the line table data.

9. A chip is characterized in that,

the chip comprises: a processor for calling and running a computer program from a memory, causing a device on which the chip is mounted to execute: the method of any one of claims 1-7.