CN111865636A - Optical cable pipeline data analysis system, method, server and storage medium - Google Patents
Optical cable pipeline data analysis system, method, server and storage medium Download PDFInfo
- Publication number
- CN111865636A CN111865636A CN201910356217.9A CN201910356217A CN111865636A CN 111865636 A CN111865636 A CN 111865636A CN 201910356217 A CN201910356217 A CN 201910356217A CN 111865636 A CN111865636 A CN 111865636A
- Authority
- CN
- China
- Prior art keywords
- data
- optical cable
- cable pipeline
- module
- inspection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
Abstract
The invention discloses an optical cable pipeline data analysis system, a method, a server and a storage medium, wherein a data acquisition module is used for acquiring stock optical cable pipeline data as a batch data analysis basis and recording business data as a real-time data analysis basis; the optical cable pipeline data of the stock are processed in batch and the recorded business data are processed in real time; the mass pipeline data are analyzed in real time by batch processing of the pipeline data and real-time processing of the service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the data are difficult to analyze and process quickly when the traditional technology manages the mass data is solved, the quality of the pipeline data is ensured, the inspection efficiency is improved, and the fault occurrence rate is reduced.
Description
Technical Field
The invention relates to the technical field of transmission networks, in particular to an optical cable pipeline data analysis system, an optical cable pipeline data analysis method, a server and a storage medium.
Background
With the rapid development of communication technology and the increasing expansion of the construction range of the mobile communication resources in China, the optical cable pipeline resources become one of the core resources of communication operators, and because the existing communication services are opened without leaving the optical cable pipeline resources, how to efficiently know and utilize the network resources of the operators becomes a consensus of the operators. At present, most of domestic optical cable pipeline systems adopt a B/S or C/S architecture, relational data (databases such as MySQL, Oracle and the like) are adopted for storing data, the traditional resource management method effectively improves the management efficiency of optical cable pipeline resources and reduces corresponding labor cost compared with the traditional manual resource management method, but with the increasing of network scale and service types, the data volume is increased sharply, the traditional storage method cannot meet the storage of mass data, the traditional resource management system cannot meet the actual requirement through simple query and display, and more intrinsic values of mining resource data, particularly the intrinsic value of mass resource data after combination, are needed, so that the actual problem is solved.
However, the existing optical cable pipeline system has the defects of difficult mass data storage and high maintenance cost. With the rapid development of communication technology, the generated data is massive, and the communication data is a difficult problem for enterprises in management; at present, data becomes a key basis for normal operation of an enterprise, and if a data disaster occurs, the work of the whole enterprise is paralyzed; it would be difficult to quickly analyze such data if such data were still managed using conventional techniques.
Disclosure of Invention
In view of the above, the present invention has been developed to provide a cable line data analysis system and a corresponding cable line data analysis method that overcome or at least partially address the above-mentioned problems.
According to an aspect of the present invention, there is provided a cable pipeline data analysis system, including: the data acquisition module is used for acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis; the data analysis module is used for carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data; the data inspection module is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module; and the data presentation module is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
According to another aspect of the present invention, there is provided a cable pipeline data analysis method, which is implemented based on the data desensitization control system described above, and includes: acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis; carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data; obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module; and displaying results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
According to still another aspect of the present invention, there is provided a server including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the optical cable pipeline data analysis method.
According to yet another aspect of the present invention, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform operations corresponding to the optical cable pipeline data analysis method.
The optical cable pipeline data analysis system and method of the invention, collect the optical cable pipeline data of stock as the basis of data analysis of batch processing and type in the business data as the basis of real-time data analysis through the data acquisition module; the optical cable pipeline data of the stock are processed in batch and the recorded business data are processed in real time; the mass pipeline data are analyzed in real time by batch processing of the pipeline data and real-time processing of the service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the data are difficult to analyze and process quickly when the traditional technology manages the mass data is solved, the quality of the pipeline data is ensured, the inspection efficiency is improved, and the fault occurrence rate is reduced.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a cable pipeline data analysis system in accordance with an exemplary embodiment of the present invention;
FIG. 2 is a cable pipeline data analysis system in accordance with an exemplary embodiment of the present invention;
FIG. 3 is a method of cable pipeline data analysis in accordance with an exemplary embodiment of the present invention;
FIG. 4 is a method of cable pipeline data analysis in accordance with an exemplary embodiment of the present invention;
fig. 5 is a schematic diagram of a server according to an exemplary embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Example one
Fig. 1 is a diagram illustrating a cable pipeline data analysis system according to an exemplary embodiment of the present disclosure, as shown in fig. 1, including:
and the data acquisition module 11 is used for acquiring the data of the optical cable pipelines of the stock as a batch data analysis basis and inputting the business data as a real-time data analysis basis.
Specifically, the data acquisition module acquires the pipeline data of the stock as a batch data analysis basis and acquires the business data input by a user through a pipeline system as a real-time data analysis basis.
The stored pipeline data has the characteristics of large data volume, complex business logic and the like, and the relational data and the HDFS are copied by using a Sqoop tool, so that data batch processing analysis can be performed by using MapReduce and Hive of Hadoop, and the stored pipeline data can be backed up.
When a user inputs pipeline service data in real time through a pipeline system, the data is firstly stored in a relational database, and simultaneously newly added service data is recorded and written into a Kafka message queue.
And the data analysis module 12 is used for batch processing of the optical cable pipeline data of the stock and real-time processing of the recorded business data.
Specifically, firstly, the data of the optical cable pipeline inventory are analyzed in batch by a MapReduce programming mode and Hive of Hadoop, and the specific analysis principle and analysis rule are as follows:
(1) principle of MapReduce analysis
The MapReduce programming mode is mainly applied to parallel operation of large-scale data sets. MapReduce highly abstracts the complex parallel computing process running on large-scale clusters to two functions: the Map function and Reduce function adopt a 'divide-and-conquer' strategy, a large-scale data set stored in a distributed file system is divided into a plurality of independent fragments, and the fragments can be processed by a plurality of Map tasks in parallel. In the analysis process, the Map function is only needed to be realized for data processing, and the processed data is written into the HDFS and the Redice cluster.
For example, when a file on the HDFS distributed file system is read, each piece of data is automatically parsed into a key-value pair in the < k, v > format, where k is the index of the piece of data and v is the data record. The map function is called once per key-value pair. In the map function, receiving data of < k, v >, acquiring v data records, performing service logic processing according to a service analysis rule, and writing processed results into an HDFS (Hadoop distributed File System) cluster and a Redis cluster respectively.
(2) Business analysis rules
Isolated point data analysis: the pipeline resources are divided into point resources and line resources. The point resources comprise manhole, electric pole, marker stone, supporting point, light splitting, light intersection, light termination and other resources, and the line resources comprise bearing section, optical cable section and other resources. The line resource is carried and the point resource is carried. If a point resource does not carry any line resource, it is called an isolated point. In general, the isolated points are abnormal data and need to be checked and confirmed in real time and eliminated in time. The system associates resources such as manhole, electric pole, marker stone, support point, light splitting, light intersection, light and the like with the bearing section and the optical cable section, if no corresponding bearing section and optical cable section data are associated with the bearing section and the optical cable section data, the bearing section and the optical cable section data are regarded as isolated point data and written into a Redis cache database (or can be persisted), and a user is reminded to use the data.
And (3) incompletely laying optical cable analysis: the line data in the pipeline resources includes both cable segment and bearer segment data. The cable segments must be laid on top of the carrier segments (e.g., pipe holes) and cannot stand alone. If a certain optical cable segment is isolated and the bearing segment exists, the data of the optical cable segment is abnormal data and needs to be checked in real time and corrected in time. The system performs correlation analysis on the optical cable segment and the carrier segment data, and if no corresponding carrier segment data corresponds to the carrier segment data, the optical cable data is written into a Redis cache database to remind a user of laying an optical cable.
Incomplete overhead cable analysis: the optical cable section data structure comprises an A section and a Z section of the optical cable section, namely two ends of the optical cable section. Generally, the a-segment and Z-segment data of the optical cable segment must be associated with the terminal data of the transmission device. And if the data at the A end or the Z end of a certain optical cable section is not related to the terminal data of the transmission equipment, the optical cable is not erected. The section of the optical cable which is not erected is abnormal data, and needs to be checked in real time and corrected in time. The system performs correlation analysis on terminal data of the optical cable segment and the equipment at two ends, and if the equipment at one end does not have end relation with the terminal, the optical cable is considered to be not completely put on shelf and written into a Redis cache database to remind a user of processing.
Data relevance analysis: the pipe resource data is a large, complex, multi-resource type, mesh data set. The system starts from point facilities and is used for correlating the carrier section data, the optical cable data and the optical path data of the pipeline data. When the user selects any one of the resources on the interface, other data associated therewith can be seen.
The data analysis is carried out on partial data of the pipeline system through the rules, the data of isolated point facilities, incompletely laid optical cable data and incompletely erected optical cable data and associated data of resources are analyzed, and a user can put forward corresponding solutions for processing the analyzed data in time, so that the data quality of the pipeline data is ensured.
Secondly, the real-time processing of the input service data is realized by a Storm cluster, and the Storm analysis principle is as follows:
storm is an open-source distributed real-time computing framework, is a real-time processing system based on data flow, and has large data throughput and high real-time performance. Several core concepts in Storm computing structures are topology, stream, spout, bolt. Topology is the most core concept in storm, and consists of streams, spots, and bases. Stream is an abstraction of data Stream in storm, and spout is a data source of topology, and is responsible for connecting the data source and transmitting data to clients, and a bolt is a data processing unit in topology, and a complex data processing logic is generally split into a plurality of simple processing logics to be handed over to each bolt.
For example, when a user enters pipeline service data in real time through a pipeline system, the data is firstly stored in a traditional relational database, and simultaneously newly added service data is recorded and written into a Kafka message queue, then the data is written into the Kafka queue according to topology required by service logic design, consumption Kafka (high-throughput distributed publish-subscribe message system) data is received in spout, and the data is transmitted to a bolt (processing unit) for data processing.
The data inspection module 13 is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
as a preferred implementation manner of this embodiment, the data inspection module 13 includes precision inspection and data sharing.
(1) Accurate inspection
And the polling time and content of each polling point and route are adjusted in time according to polling requirements and service changes, so that the polling operation of the same polling point at different times according to different requirements is met. To realize accurate inspection of different lines and inspection points, inspection time and attention items can be defined for different lines and inspection points in different inspection plans, inspection lines and inspection points can be organized in a layered design mode, and data items collected by the lines can be uniformly defined. And associating the inspection line with the starting time, the ending time, the inspection period, personnel, inspection equipment and the alarm condition to generate different inspection plans. The patrol time and the attention of each patrol point can be separately defined in each plan.
(2) Data sharing
The polling personnel can upload polling time and position through GPRS mobile phone terminal equipment in the polling process, and can upload data such as pictures, voice, video, text information and the like.
In the inspection process in which multiple persons participate, in order to improve the inspection efficiency, data needs to be shared in real time, besides position and message sharing, position and time factors and specific geographical areas and lines can be set as alarm prompts, and the system can actively push corresponding messages by taking the changes of the factors as event driving, so that the system can serve as an invisible coordinator in the inspection work. If the A group of inspectors arrive at the first inspection area, informing the B group of inspectors to leave the second inspection area; and after the polling time is started, when the polling personnel in the group A do not check the released latest message, prompting the personnel in the group A or other related personnel by the short message. The real-time video communication provides powerful support for realizing work cooperation, and people who are not in the same place can know more comprehensive information through real-time field pictures.
And the data presentation module 14 is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
By adopting the system provided by the embodiment, the data acquisition module is used for acquiring the optical cable pipeline data of the stock as a batch data analysis basis and inputting the business data as a real-time data analysis basis; the optical cable pipeline data of the stock are processed in batch and the recorded business data are processed in real time; the mass pipeline data are analyzed in real time by batch processing of the pipeline data and real-time processing of the service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the data are difficult to analyze and process quickly when the traditional technology manages the mass data is solved, the quality of the pipeline data is ensured, the inspection efficiency is improved, and the fault occurrence rate is reduced.
Example two
FIG. 2 is another fiber optic cable pipeline data analysis system shown in an exemplary embodiment of the invention, comprising:
and the data acquisition module 21 is used for acquiring the data of the optical cable pipelines of the stock as a batch data analysis basis and inputting the business data as a real-time data analysis basis.
And the data analysis module 22 is used for batch processing of the optical cable pipeline data of the stock and real-time processing of the recorded business data.
Optionally, the data analysis module 22 includes:
the batch processing module 201 is configured to perform batch data analysis on the stocked optical cable pipeline data through a Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines, and incompletely set optical cable pipeline data in the optical cable pipeline data;
and the real-time processing module 202 is configured to perform real-time data analysis processing on the entered service data through the Storm cluster to obtain important data of the cable pipeline for inspection.
Optionally, the real-time processing module 202 further includes:
an optical cable pipeline service number counting submodule 2001, configured to count a current data bearer service number;
an optical cable pipeline density analyzing submodule 2002 for analyzing the density of the position where the optical cable pipeline is located;
and the optical cable pipeline position and construction road matching analysis submodule 2003 is used for analyzing whether the data is in the construction road.
Specifically, the real-time analysis of the data is realized by storm cluster, and the specific analysis principle and analysis strategy are as follows:
(1) storm principle of analysis:
storm is an open-source distributed real-time computing framework, is a real-time processing system based on data flow, and has large data throughput and high real-time performance. Several core concepts in Storm computing structures are topology, stream, spout, bolt. Topology is the most core concept in storm, and consists of streams, spots, and bases. Stream is an abstraction of data Stream in storm, spout is a data source of topology, and is responsible for connecting the data source and transmitting data to clients, and a bolt is a data processing unit in topology, and a complex data processing logic is generally split into a plurality of simple processing logics to be handed over to each bolt.
When a user inputs pipeline service data in real time through a pipeline system, the data is firstly stored in a traditional relational database, and simultaneously newly added service data is recorded and written into a Kafka (high-throughput distributed publish-subscribe message system) message queue, next, topology meeting the service logic requirement of the user needs to be designed, when the user writes the data into the Kafka queue, the user receives consumption Kafka data in spout and transmits the data to a bolt (processing unit) for data processing, wherein three bolts are designed, and the three bolts are respectively: cable service quantity statistics bolt (cableservivicecountbolt), cable density analysis bolt (cabledensebolt), and cable location and construction road matching analysis bolt (cableandstructurroadbolt). In the service quantity counting bolt, a current data carrying service quantity is counted, an optical cable density analyzing bolt is used for analyzing the density of the position of an optical cable, and the optical cable position and construction road matching analyzing bolt is used for analyzing whether the data is near the construction road. Finally, writing the analyzed optical cable data structure into a Redis cache database (or performing persistence).
(2) Optical cable intelligent inspection analysis strategy
The intelligent routing inspection analysis of the optical cable mainly comprises the following steps: and evaluating and analyzing the three aspects of the number of the optical cable related services, the optical cable density degree of a certain area and the road condition of the area where the optical cable is located.
Calculating the quantity of the optical cable associated services: according to the number of the services related to the optical cable section, the importance degree of the optical cable is judged, and the more the number of the services borne by the optical cable is, the more important the optical cable is, and inspection needs to be focused.
Secondly, calculating the density degree of the optical cable: when one optical cable and a plurality of optical cables are crossed or overlapped, the optical cable is recognized
For a higher density of the cable, the number of crossed cables is a measure of its density. The more optical cables cross the optical cable, the more important the optical cable needs to be inspected. By performing the spatial calculation using the spatial function of Arcgis, it can be determined whether the two cables cross or overlap.
Construction conditions of roads in the area where the optical cable is located: a road condition information module of a highway administration of a traffic transportation hall in Hebei province is crawled through a program to obtain road data of the Hebei province.
The data inspection module 23 is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
and the data presentation module 24 is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
As a preferred implementation manner of this embodiment, the data presentation module 24 includes: presenting Internet GIS map optical cable pipeline resources; presentation of cable pipeline data correlation; presenting and counting isolated point data; presentation of incomplete racking/cabling cable pipeline data; displaying a GIS of the routing inspection route; and displaying the inspection result.
By adopting the system provided by the embodiment, the data acquisition module is used for acquiring the optical cable pipeline data of the stock as a batch data analysis basis and inputting the business data as a real-time data analysis basis; the optical cable pipeline data of the stock are processed in batch and the recorded business data are processed in real time; the mass pipeline data are analyzed in real time by batch processing of the pipeline data and real-time processing of the service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the data are difficult to analyze and process quickly when the traditional technology manages the mass data is solved, the quality of the pipeline data is ensured, the inspection efficiency is improved, and the fault occurrence rate is reduced.
EXAMPLE III
Fig. 3 is a cable pipeline data analysis method according to an embodiment of the present disclosure, where the method is implemented based on the cable pipeline data analysis system, and the method includes:
s31: acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis;
s32: carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data;
s33: obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
s34: and displaying results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
By adopting the method provided by the embodiment, the optical cable pipeline data of the collected stock is used as a batch data analysis basis and the input service data is used as a real-time data analysis basis; the optical cable pipeline data of the stock are processed in batch and the recorded business data are processed in real time; the mass pipeline data are analyzed in real time by batch processing of the pipeline data and real-time processing of the service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the data are difficult to analyze and process quickly when the traditional technology manages the mass data is solved, the quality of the pipeline data is ensured, the inspection efficiency is improved, and the fault occurrence rate is reduced.
Example four
Fig. 4 is another optical cable pipeline data analysis method provided in the embodiment of the present disclosure, and as shown in fig. 4, the method includes:
s41: acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis;
s42: carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data;
optionally, the batch processing of the optical cable pipeline data of the inventory and the real-time processing of the entered service data further include:
carrying out batch data analysis on the stocked optical cable pipeline data through a Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines and incompletely erected optical cable pipeline data in the optical cable pipeline data;
and analyzing and processing the input service data in real time through the Storm cluster to obtain important data of the inspection optical cable pipeline.
Optionally, the analyzing and processing the entered service data in real time by the Storm cluster to obtain the important inspection optical cable pipeline data further includes:
s421: counting the number of the current data bearing services;
s422: analyzing the density of the position of the optical cable pipeline;
S423: and analyzing whether the data is in the construction road.
S43: obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
s44: and the data processing module is used for displaying results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
By adopting the method provided by the embodiment, the optical cable pipeline data of the collected stock is used as a batch data analysis basis and the input service data is used as a real-time data analysis basis; the optical cable pipeline data of the stock are processed in batch and the recorded business data are processed in real time; the mass pipeline data are analyzed in real time by batch processing of the pipeline data and real-time processing of the service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the data are difficult to analyze and process quickly when the traditional technology manages the mass data is solved, the quality of the pipeline data is ensured, the inspection efficiency is improved, and the fault occurrence rate is reduced.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention, and as shown in fig. 5, the terminal may include: a processor (processor)502, a Communications Interface 504, a memory 506, and a communication bus 508.
Wherein:
the processor 502, communication interface 504, and memory 506 communicate with one another via a communication bus 508.
A communication interface 504 for communicating with network elements of other devices, such as clients or other servers.
The processor 502, configured to execute the program 510, may specifically execute the relevant steps in the above-described cable pipeline data analysis method embodiment.
In particular, program 510 may include program code that includes computer operating instructions.
The processor 502 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the invention. The terminal comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 506 for storing a program 510. The memory 506 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 510 may specifically be used to cause the processor 502 to perform the following operations:
acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis; carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data; obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module; and displaying the inspection result according to the inspection result of the data inspection module.
EXAMPLE six
An embodiment of the present application provides a non-volatile computer storage medium, where the computer storage medium stores at least one executable instruction, and the computer executable instruction may execute the optical cable pipeline data analysis method in any of the above method embodiments.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in a data analysis system based on fiber optic cable lines. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (10)
1. A fiber optic cable pipeline data analysis system, comprising:
the data acquisition module is used for acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis;
The data analysis module is used for carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data;
the data inspection module is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
and the data presentation module is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
2. The system of claim 1, wherein the data analysis module comprises:
the batch processing module is used for carrying out batch data analysis on the stocked optical cable pipeline data through a Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines and/or incompletely erected optical cable pipeline data in the optical cable pipeline data;
and the real-time processing module is used for analyzing and processing the input service data in real time through the Storm cluster to obtain important data of the inspection optical cable pipeline.
3. The system of claim 2, wherein the real-time processing module comprises:
the optical cable pipeline service number counting submodule is used for counting the number of the current data bearing services;
the optical cable pipeline density analyzing submodule is used for analyzing the density of the position where the optical cable pipeline is located;
And the optical cable pipeline position and construction road matching analysis submodule is used for analyzing whether the data is in the construction road.
4. The system of claim 1, wherein the data presentation module is specifically configured to:
presenting Internet GIS map optical cable pipeline resources;
presentation of cable pipeline data correlation;
presenting and counting isolated point data;
presentation of incomplete racking/cabling cable pipeline data;
displaying a GIS of the routing inspection route;
and displaying the inspection result.
5. A cable line data analysis method implemented based on the cable line data analysis system according to any one of claims 1 to 4, the method comprising:
acquiring optical cable pipeline data of stock as a batch data analysis basis and recording business data as a real-time data analysis basis;
carrying out batch processing on the optical cable pipeline data of the stock and carrying out real-time processing on the input business data;
obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
and displaying results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
6. The method of claim 5, wherein the batch processing of the inventory of fiber optic cable line data and the real-time processing of the logged business data further comprises:
carrying out batch data analysis on the stocked optical cable pipeline data through a Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines and incompletely erected optical cable pipeline data in the optical cable pipeline data;
and analyzing and processing the input service data in real time through the Storm cluster to obtain important data of the inspection optical cable pipeline.
7. The method of claim 6, wherein the performing real-time data analysis processing on the logged service data through Storm cluster to obtain important patrol cable pipeline data further comprises:
counting the number of the current data bearing services;
analyzing the density of the position of the optical cable pipeline;
and analyzing whether the data is in the construction road.
8. The method of claim 5, wherein presenting results based on the processed data of the data collection module, the data analysis module, and the data inspection module further comprises:
presenting Internet GIS map optical cable pipeline resources;
Presentation of cable pipeline data correlation;
presenting and counting isolated point data;
presentation of incomplete racking/cabling cable pipeline data;
displaying a GIS of the routing inspection route;
and displaying the inspection result.
9. A server, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the optical cable pipeline data analysis method according to any one of claims 5-8.
10. A computer storage medium having stored therein at least one executable instruction that causes a processor to perform operations corresponding to the fiber optic cable pipeline data analysis method of any of claims 5-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910356217.9A CN111865636B (en) | 2019-04-29 | 2019-04-29 | Optical cable pipeline data analysis system, method, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910356217.9A CN111865636B (en) | 2019-04-29 | 2019-04-29 | Optical cable pipeline data analysis system, method, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111865636A true CN111865636A (en) | 2020-10-30 |
| CN111865636B CN111865636B (en) | 2023-04-28 |
Family
ID=72965444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910356217.9A Active CN111865636B (en) | 2019-04-29 | 2019-04-29 | Optical cable pipeline data analysis system, method, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111865636B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114584208A (en) * | 2022-04-28 | 2022-06-03 | 高勘(广州)技术有限公司 | Method, device and equipment for checking optical cable facility points and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8655167B1 (en) * | 2011-01-05 | 2014-02-18 | Google Inc. | Fiber diagnosis system for point-to-point optical access networks |
| CN107181522A (en) * | 2017-05-18 | 2017-09-19 | 云南电网有限责任公司红河供电局 | Based on patterned optical cable intelligent total management operational system |
| CN109344218A (en) * | 2018-10-15 | 2019-02-15 | 中国电信股份有限公司广东传送网络运营中心 | Lightguide cable link method for inspecting and its system |
| CN109542859A (en) * | 2018-10-18 | 2019-03-29 | 天津大学 | A kind of Information Maritime processing model based on cloud computing |
-
2019
- 2019-04-29 CN CN201910356217.9A patent/CN111865636B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8655167B1 (en) * | 2011-01-05 | 2014-02-18 | Google Inc. | Fiber diagnosis system for point-to-point optical access networks |
| CN107181522A (en) * | 2017-05-18 | 2017-09-19 | 云南电网有限责任公司红河供电局 | Based on patterned optical cable intelligent total management operational system |
| CN109344218A (en) * | 2018-10-15 | 2019-02-15 | 中国电信股份有限公司广东传送网络运营中心 | Lightguide cable link method for inspecting and its system |
| CN109542859A (en) * | 2018-10-18 | 2019-03-29 | 天津大学 | A kind of Information Maritime processing model based on cloud computing |
Non-Patent Citations (1)
| Title |
|---|
| 王向东等: "基于WebGIS的电力通信光缆资源管理平台设计与应用分析", 《信息通信》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114584208A (en) * | 2022-04-28 | 2022-06-03 | 高勘(广州)技术有限公司 | Method, device and equipment for checking optical cable facility points and storage medium |
| CN114584208B (en) * | 2022-04-28 | 2022-07-05 | 高勘(广州)技术有限公司 | Method, device and equipment for checking optical cable facility points and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111865636B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177222B (en) | Model testing method and device, computing equipment and storage medium | |
| CN110457190B (en) | Full link monitoring method, device and system based on block chain | |
| US7739138B2 (en) | Automated utility supply management system integrating data sources including geographic information systems (GIS) data | |
| Yadranjiaghdam et al. | Developing a real-time data analytics framework for twitter streaming data | |
| CN111949834B (en) | Site selection method and site selection platform system | |
| CN105631026A (en) | Security data analysis system | |
| CN105159951A (en) | Open tourism multi-source heterogeneous data fusion method and system | |
| CN105577454A (en) | Method for quickly positioning service fault based on log | |
| CN106815254B (en) | Data processing method and device | |
| CN108170832B (en) | Monitoring system and monitoring method for heterogeneous database of industrial big data | |
| CN111245672A (en) | Monitoring method and system for general extensible tracking service full link | |
| CN111400288A (en) | Data quality inspection method and system | |
| CN111325428A (en) | Work order pushing method and device and storage medium | |
| CN111737364B (en) | Safe multi-party data fusion and federal sharing method, device, equipment and medium | |
| CN112148779A (en) | Method, device and storage medium for determining service index | |
| CN107704357B (en) | Log generation method and device | |
| CN105184435A (en) | Field staff management method and system | |
| CN113704178A (en) | Big data management method, system, electronic device and storage medium | |
| CN111865636B (en) | Optical cable pipeline data analysis system, method, server and storage medium | |
| CN106257447A (en) | The video storage of cloud storage server and search method, video cloud storage system | |
| CN112232662A (en) | Service monitoring system and method | |
| CN115439015A (en) | Local area power grid data management method, device and equipment based on data middleboxes | |
| CN114723397A (en) | Flow execution method and device | |
| US20190347838A1 (en) | Passing system with an interactive user interface | |
| KR101878291B1 (en) | Big data management system and management method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |