CN111865636B - Optical cable pipeline data analysis system, method, server and storage medium - Google Patents
Optical cable pipeline data analysis system, method, server and storage medium Download PDFInfo
- Publication number
- CN111865636B CN111865636B CN201910356217.9A CN201910356217A CN111865636B CN 111865636 B CN111865636 B CN 111865636B CN 201910356217 A CN201910356217 A CN 201910356217A CN 111865636 B CN111865636 B CN 111865636B
- Authority
- CN
- China
- Prior art keywords
- data
- optical cable
- module
- cable pipeline
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
Abstract
The invention discloses an optical cable pipeline data analysis system, an optical cable pipeline data analysis method, a server and a storage medium, wherein the data acquisition module is used for acquiring the stored optical cable pipeline data as a batch processing data analysis basis and recording business data as a real-time data analysis basis; batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data; real-time analysis is carried out on mass pipeline data through pipeline data batch processing and real-time processing of service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the mass data are difficult to analyze and process rapidly in the traditional technology is solved, the quality of pipeline data is ensured, the inspection efficiency is improved, and the occurrence rate of faults is reduced.
Description
Technical Field
The invention relates to the technical field of transmission networks, in particular to an optical cable pipeline data analysis system, an optical cable pipeline data analysis method, a server and a storage medium.
Background
With the rapid development of communication technology and the increasing expansion of the construction range of China mobile communication resources, optical cable pipeline resources become one of core resources of communication operators, and because the existing communication service is opened without leaving the optical cable pipeline resources, how to efficiently know and utilize own network resources becomes the consensus of each operator. At present, a domestic optical cable pipeline system mostly adopts a B/S or C/S architecture, relational data (MySQL, oracle and other databases) are adopted for storing data, the traditional resource management effectively improves the management efficiency of optical cable pipeline resources and reduces corresponding labor cost compared with the traditional manual resource management mode, but the traditional storage mode cannot meet the requirement of storing mass data under the condition that the network scale and service types are increased gradually, and the traditional resource management system cannot meet the actual requirement when simply inquiring and displaying, so that the actual problem is solved by the need of more internal values of excavating resource data, especially the internal values of the mass resource data after being combined.
However, the existing optical cable pipeline system has the defects of difficult mass data storage and high maintenance cost. With the rapid development of communication technology, the generated data is massive, and the management of the communication data is a difficult problem for enterprises; today, data becomes a key foundation for normal operation of enterprises, and if a data disaster is encountered, the work of the whole enterprise is paralyzed; it would be difficult to quickly analyze such data if conventional techniques were still used to manage such data.
Disclosure of Invention
The present invention has been made in view of the above problems, and it is an object of the present invention to provide a cable line data analysis system and a corresponding cable line data analysis method that overcome or at least partially solve the above problems.
According to one aspect of the present invention, there is provided a fiber optic cable line data analysis system comprising: the data acquisition module is used for acquiring the optical cable pipeline data stored as a batch processing data analysis basis and recording service data as a real-time data analysis basis; the data analysis module is used for carrying out batch processing on the stored optical cable pipeline data and carrying out real-time processing on the recorded service data; the data inspection module is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module; and the data presentation module is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
According to another aspect of the present invention, there is provided a method for analyzing data of a cable line, the method being implemented based on the above-described data desensitization control system, the method comprising: collecting stock optical cable pipeline data as a batch processing data analysis basis and recording service data as a real-time data analysis basis; batch processing is carried out on the stored optical cable pipeline data, and real-time processing is carried out on the recorded service data; obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module; and presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
According to still another aspect of the present invention, there is provided a server including: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the optical cable pipeline data analysis method.
According to still another aspect of the present invention, there is provided a computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the above-described optical cable pipeline data analysis method.
According to the optical cable pipeline data analysis system and method, the data acquisition module acquires the stored optical cable pipeline data as a batch processing data analysis basis and records service data as a real-time data analysis basis; batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data; real-time analysis is carried out on mass pipeline data through pipeline data batch processing and real-time processing of service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the mass data are difficult to analyze and process rapidly in the traditional technology is solved, the quality of pipeline data is ensured, the inspection efficiency is improved, and the occurrence rate of faults is reduced.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a diagram of a fiber optic cable line data analysis system according to an exemplary embodiment of the present invention;
FIG. 2 is a diagram of a fiber optic cable line data analysis system according to an exemplary embodiment of the present invention;
FIG. 3 is a diagram illustrating a method of analyzing data of a fiber optic cable line in accordance with an exemplary embodiment of the present invention;
FIG. 4 is a diagram illustrating a method of analyzing data of a fiber optic cable line in accordance with an exemplary embodiment of the present invention;
fig. 5 is a schematic diagram of a server according to an exemplary embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Example 1
FIG. 1 is a diagram of a fiber optic cable line data analysis system, as shown in FIG. 1, according to an exemplary embodiment of the present disclosure, including:
the data acquisition module 11 is used for acquiring the optical cable pipeline data of stock as a batch processing data analysis basis and recording business data as a real-time data analysis basis.
Specifically, the data acquisition module acquires the stock pipeline data as a batch processing data analysis basis and acquires the logging service data of a user through a pipeline system as a real-time data analysis basis.
The stored pipeline data has the characteristics of large data size, complex business logic and the like, and the Sqoop tool is used for copying the data of the relational data and the HDFS, so that MapReduce, hive of Hadoop can be used for carrying out data batch analysis, and the stored pipeline data can be backed up.
When a user inputs pipeline service data in real time through a pipeline system, the data is firstly stored in a relational database, and the newly added service data is recorded and written into a Kafka message queue.
The data analysis module 12 is used for batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data.
Specifically, firstly, the batch processing of the stored optical cable pipeline data analyzes the data through a MapReduce programming mode and Hive of Hadoop, and the specific analysis principle and analysis rule are as follows:
(1) Principle of MapReduce analysis
The MapReduce programming mode is mainly applied to parallel operation of a large-scale data set. MapReduce highly abstracts complex, parallel computing processes running on large-scale clusters into two functions: the Map function and Reduce function, using a "divide-and-conquer" strategy, a large-scale data set stored in a distributed file system, is split into a number of independent partitions that can be processed in parallel by multiple Map tasks. In the analysis process, the Map function is only needed to be realized to process data, and the processed data is written into the HDFS and the redirect cluster.
For example, when a file on the HDFS distributed file system is read, each piece of data is automatically parsed into a key pair in the format < k, v >, where k is the index of the piece of data and v is the record of the piece of data. Each key value calls a map function once. In the map function, the data of < k, v > is accepted, the v data record is obtained, then the business logic processing is carried out according to the business analysis rule, and the processed result is written into the HDFS and Redis clusters respectively.
(2) Business analysis rules
Isolation point data analysis: pipeline resources are divided into point resources and line resources. The point resources comprise resources such as a manhole, an electric pole, a marker stone, a supporting point, optical splitting, optical intersection, optical termination and the like, and the line resources comprise resources such as a bearing section, an optical cable section and the like. The line resources are carried over the point resources. If a point resource does not carry any line resources, it is referred to as an orphan point. In general, isolated points are abnormal data, and need to be checked and confirmed in real time and eliminated in time. The system associates the resources such as a manhole, an electric pole, a marker stone, a supporting point, light splitting, light intersection, light and the like with the bearing section and the optical cable section, and if no corresponding bearing section and optical cable section data are associated with the bearing section and the optical cable section, the corresponding bearing section and the optical cable section data are considered to be isolated point data, the isolated point data are written into a Redis cache database (can be also subjected to persistence), and a user is reminded of using the isolated point data.
Incomplete cabling analysis: the line data in the pipeline resource includes cable section and carrier section data. The cable segments must be laid out and carried over the carrier segments (e.g., tube holes) and are unlikely to exist in isolation. If a certain optical cable section is isolated and the bearing section exists, the data of the optical cable section is abnormal data, and the data needs to be checked in real time and corrected in time. The system carries out association analysis on the optical cable section and the bearing section data, if no corresponding bearing section data corresponds to the optical cable section, the optical cable data is written into a Redis cache database, and a user is reminded to lay the optical cable.
Incomplete on-shelf cable analysis: the optical cable section data structure comprises an A section and a Z section of the optical cable section, namely two ends of the optical cable section. Typically, the a-segment and Z-segment data of the cable segment must be associated with the terminal data of the transmission device. If the A end or Z end data of a certain optical cable section is not associated with the terminal data of the transmission equipment, the optical cable is not put on shelf. The optical cable section which is not put on shelf is abnormal data and needs to be checked in real time and corrected in time. The system performs association analysis on terminal data of the optical cable section and the two-end equipment, if one end equipment has no end-to-end relation with the terminal, the optical cable is considered to be incompletely put on the shelf, and the optical cable is written into a Redis cache database to remind a user of processing.
Data correlation analysis: pipeline resource data is a large, complex, multi-resource type, mesh data set. The system starts from point facilities and correlates bearing section data, optical cable data and optical path data of pipeline data. When a user selects any one of the resources at the interface, other data associated therewith can be seen.
And the data analysis is carried out on partial data of the pipeline system through the rules, so that isolated point facilities, incompletely laid and incompletely erected optical cable data and associated data of resources are analyzed, and users can timely put forward corresponding solutions for processing the analyzed data, so that the data quality of the pipeline data is ensured.
Secondly, real-time processing of the input business data is realized through a Storm cluster, and a Storm analysis principle is as follows:
storm is a distributed real-time computing framework of open source, and is a real-time processing system based on data flow, and the data throughput is high and the real-time performance is high. Several core concepts in the Storm computing structure are topology, stream, spout, bolt. The technology is the most core concept in storm, and consists of streams, bolts. Stream is an abstraction of a data Stream in store, spout is a data source of topology, responsible for connecting the data source, delivering data to bolts, which are data processing units in topology, and complex data processing logic is typically split into multiple simple processing logic responsible for each bolt.
For example, when a user performs real-time entry of pipeline service data through a pipeline system, the data is firstly stored in a traditional relational database, meanwhile, newly added service data is recorded and written into a Kafka message queue, then the data is written into the Kafka queue according to the required topology of service logic design, consumed Kafka (high throughput distributed publish-subscribe message system) data is accepted in spout, and the data is transmitted to a bolt (processing unit) for data processing.
The data inspection module 13 is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
as a preferred implementation manner of this embodiment, the data inspection module 13 includes accurate inspection and data sharing.
(1) Accurate inspection
The inspection time and the content of each inspection point and each route are timely adjusted according to the inspection requirements and the service changes, so that the inspection work of the same inspection point is executed according to different requirements at different times. To realize accurate inspection of different lines and inspection points, namely, inspection time and notes can be defined for different lines and inspection points in different inspection plans, inspection lines and inspection points can be organized in a layered design mode, and data items collected by the lines can be uniformly defined. And then, associating the inspection line with the starting and ending time, the inspection period, personnel, inspection equipment and alarm conditions to generate different inspection plans. The inspection time and notice of each inspection point can be defined in each plan independently.
(2) Data sharing
The patrol personnel can upload patrol time and position through the GPRS mobile phone terminal equipment in the patrol process, and can upload data such as pictures, voice, video and text information.
In the process of inspection participated by multiple persons, in order to improve the inspection efficiency, data needs to be shared in real time, besides position and message sharing, position and time factors and specific geographic areas and lines can be set as alarm prompts, and the system can actively push corresponding messages by taking the changes of the factors as event driving, so that the system can serve as an invisible coordinator in the inspection work. If the group A inspector reaches the first inspection area, notifying the group B inspector to leave the second inspection area; when the inspection time starts and the group A inspector does not view the released latest information yet, the short message prompts the group A personnel or other related personnel. The real-time video communication provides powerful support for realizing work coordination, so that people who are not in the same place can know more comprehensive information through real-time field pictures.
And the data presentation module 14 is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
By adopting the system provided by the embodiment, the data acquisition module acquires the stored optical cable pipeline data as a batch processing data analysis basis and the recorded service data as a real-time data analysis basis; batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data; real-time analysis is carried out on mass pipeline data through pipeline data batch processing and real-time processing of service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the mass data are difficult to analyze and process rapidly in the traditional technology is solved, the quality of pipeline data is ensured, the inspection efficiency is improved, and the occurrence rate of faults is reduced.
Example two
FIG. 2 is another fiber optic cable line data analysis system according to an exemplary embodiment of the present invention, comprising:
the data acquisition module 21 is used for acquiring the stock of optical cable pipeline data as a batch data analysis basis and logging service data as a real-time data analysis basis.
The data analysis module 22 is used for batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data.
Optionally, the data analysis module 22 includes:
the batch processing module 201 is configured to perform batch data analysis on the stored optical cable pipeline data through Hadoop clusters, so as to obtain isolated point data, incompletely laid optical cable pipelines, and incompletely set up optical cable pipeline data in the optical cable pipeline data;
the real-time processing module 202 is configured to perform real-time data analysis processing on the input service data through the Storm cluster, so as to obtain important patrol optical cable pipeline data.
Optionally, the real-time processing module 202 further includes:
a cable line traffic statistics submodule 2001 for counting the current data-carrying traffic;
the optical cable pipeline density analyzing submodule 2002 is used for analyzing the density of the positions of the optical cable pipelines;
the cable line position and construction road matching analysis submodule 2003 is used for analyzing whether the piece of data is on a construction road or not.
Specifically, the real-time analysis of the data is realized through a storm cluster, and the specific analysis principle and analysis strategy are as follows:
(1) Storm analysis principle:
storm is a distributed real-time computing framework of open source, and is a real-time processing system based on data flow, and the data throughput is high and the real-time performance is high. Several core concepts in the Storm computing structure are topology, stream, spout, bolt. The technology is the most core concept in storm, and consists of streams, bolts. Stream is an abstraction of a data Stream in store, spout is a data source of topology, responsible for connecting the data source, delivering data to bolts, which are data processing units in topology, and complex data processing logic is typically split into multiple simple processing logic responsible for each bolt.
When a user inputs pipeline service data in real time through a pipeline system, the data is firstly stored in a traditional relational database, and meanwhile, newly added service data is recorded and written into a Kafka (high throughput distributed publish-subscribe message system) message queue, then, the design of a topology meeting the needs of our service logic is needed, when the user writes the data into the Kafka queue, the user receives consumption Kafka data in spout and transmits the data to a bolt (processing unit) for data processing, and three bolts are designed: and (5) carrying out statistics bolt (CableServiceCountBolt) on the service quantity of the optical cable, analyzing bolt (CableDenseBolt) on the density degree of the optical cable, and analyzing bolt (CableAndConstrucRoadBolt) on the matching of the optical cable position and the construction road. In the service quantity counting bolt, the current data bearing service quantity is counted, the optical cable density degree analysis bolt is used for analyzing the density of the position where the optical cable is located, and the optical cable position and construction road matching analysis bolt is used for analyzing whether the piece of data is near the construction road or not. Finally, the analyzed optical cable data structure is written into a Redis cache database (can be subjected to persistence).
(2) Optical cable intelligent inspection analysis strategy
The intelligent inspection analysis of the optical cable mainly comprises the following steps: and carrying out evaluation analysis on three aspects of the number of the optical cable associated services, the optical cable density degree in a certain area and the road condition of the area where the optical cable is positioned.
(1) Calculating the number of optical cable associated services: according to the service quantity associated with the optical cable section, the importance degree of the optical cable is judged, and the more the service quantity borne by the optical cable is, the more important the optical cable is, and the important inspection is needed.
(2) Calculating the optical cable density: when one optical cable has a crossed or overlapped relationship with a plurality of optical cables, then
For this cable to be denser, the number of cables it crosses is taken as a measure of its density. The greater the number of cables intersecting it, the more important the cable needs to be patrolled. By performing a spatial calculation using the spatial function of Arcgis, it can be determined whether two optical cables cross or overlap.
(3) The road construction condition of the area where the optical cable is located: and acquiring road data of the Hebei province by a program crawling 'road condition information of the road management bureau of the Hebei province' module.
The data inspection module 23 is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
and the data presentation module 24 is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
As a preferred implementation of this embodiment, the data presentation module 24 includes: presenting the optical cable pipeline resources of the Internet GIS map; presenting a cable pipeline data association; presenting and counting isolated point data; presentation of incomplete racking/cabling pipeline data; GIS presentation of the tour-inspection route; and presenting the inspection result.
By adopting the system provided by the embodiment, the data acquisition module acquires the stored optical cable pipeline data as a batch processing data analysis basis and the recorded service data as a real-time data analysis basis; batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data; real-time analysis is carried out on mass pipeline data through pipeline data batch processing and real-time processing of service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the mass data are difficult to analyze and process rapidly in the traditional technology is solved, the quality of pipeline data is ensured, the inspection efficiency is improved, and the occurrence rate of faults is reduced.
Example III
Fig. 3 is a schematic diagram of a method for analyzing data of a cable line according to an embodiment of the disclosure, the method being implemented based on the system for analyzing data of a cable line, the method comprising:
s31: collecting stock optical cable pipeline data as a batch processing data analysis basis and recording service data as a real-time data analysis basis;
s32: batch processing is carried out on the stored optical cable pipeline data, and real-time processing is carried out on the recorded service data;
s33: obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
s34: and presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
By adopting the method provided by the embodiment, the data of the optical cable pipeline in stock is collected to be used as a batch processing data analysis basis, and the recorded service data is used as a real-time data analysis basis; batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data; real-time analysis is carried out on mass pipeline data through pipeline data batch processing and real-time processing of service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the mass data are difficult to analyze and process rapidly in the traditional technology is solved, the quality of pipeline data is ensured, the inspection efficiency is improved, and the occurrence rate of faults is reduced.
Example IV
FIG. 4 is a schematic diagram of another method for analyzing data of a fiber optic cable line according to an embodiment of the present disclosure, as shown in FIG. 4, the method comprising:
s41: collecting stock optical cable pipeline data as a batch processing data analysis basis and recording service data as a real-time data analysis basis;
s42: batch processing is carried out on the stored optical cable pipeline data, and real-time processing is carried out on the recorded service data;
optionally, the batch processing of the stock of optical cable pipeline data and the real-time processing of the logging service data further comprises:
carrying out batch data analysis on the stored optical cable pipeline data through the Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines and incompletely put on-shelf optical cable pipeline data in the optical cable pipeline data;
and carrying out real-time data analysis processing on the input service data through the Storm cluster to obtain important patrol optical cable pipeline data.
Optionally, the real-time data analysis processing is performed on the input service data through the Storm cluster, and obtaining important patrol optical cable pipeline data further includes:
s421: counting the number of current data bearing services;
s422: analyzing the concentration of the positions of the optical cable pipelines;
s423: and analyzing whether the piece of data is on the construction road.
S43: obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
s44: and the data processing module is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module.
By adopting the method provided by the embodiment, the data of the optical cable pipeline in stock is collected to be used as a batch processing data analysis basis, and the recorded service data is used as a real-time data analysis basis; batch processing of the stock of optical cable pipeline data and real-time processing of the logging service data; real-time analysis is carried out on mass pipeline data through pipeline data batch processing and real-time processing of service data, so that an optical cable pipeline data inspection strategy is obtained according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module. Therefore, the problem that the mass data are difficult to analyze and process rapidly in the traditional technology is solved, the quality of pipeline data is ensured, the inspection efficiency is improved, and the occurrence rate of faults is reduced.
Example five
Fig. 5 shows a schematic structural diagram of a server according to a fifth embodiment of the present invention, and as shown in fig. 5, the terminal may include: a processor 502, a communication interface (Communications Interface) 504, a memory 506, and a communication bus 508.
Wherein:
processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508.
A communication interface 504 for communicating with network elements of other devices, such as clients or other servers.
Processor 502 is configured to execute program 510, and may specifically perform the relevant steps of the above-described embodiments of the method for analyzing fiber optic cable line data.
In particular, program 510 may include program code including computer-operating instructions.
The processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the terminal may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
A memory 506 for storing a program 510. Memory 506 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 510 may be specifically operable to cause the processor 502 to:
collecting stock optical cable pipeline data as a batch processing data analysis basis and recording service data as a real-time data analysis basis; batch processing is carried out on the stored optical cable pipeline data, and real-time processing is carried out on the recorded service data; obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module; and presenting the inspection result according to the inspection result of the data inspection module.
Example six
A sixth embodiment of the present application provides a non-volatile computer storage medium storing at least one executable instruction for performing the optical cable pipeline data analysis method in any of the above method embodiments.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that microprocessors or Digital Signal Processors (DSPs) may be used in practice to implement some or all of the functions of some or all of the components in the data analysis system according to the fiber optic cable lines. The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
Claims (8)
1. A fiber optic cable line data analysis system, comprising:
the data acquisition module is used for acquiring the optical cable pipeline data stored as a batch processing data analysis basis and recording service data as a real-time data analysis basis;
the data analysis module is used for carrying out batch processing on the stored optical cable pipeline data and carrying out real-time processing on the recorded service data;
the data inspection module is used for obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
the data presentation module is used for presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module;
wherein, the data analysis module includes: the real-time processing module is used for carrying out real-time data analysis processing on the input service data through the Storm cluster to obtain important patrol optical cable pipeline data;
the real-time processing module further comprises:
the optical cable pipeline service quantity counting sub-module is used for counting the current data bearing service quantity;
the optical cable pipeline density degree analysis submodule is used for analyzing the density degree of the positions of the optical cable pipelines;
and the optical cable pipeline position and construction road matching analysis submodule is used for analyzing whether the piece of data is positioned on the construction road or not.
2. The system of claim 1, wherein the data analysis module comprises:
and the batch processing module is used for carrying out batch data analysis on the stored optical cable pipeline data through the Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines and/or incompletely put on-shelf optical cable pipeline data in the optical cable pipeline data.
3. The system according to claim 1, wherein the data presentation module is specifically configured to:
presenting the optical cable pipeline resources of the Internet GIS map;
presenting a cable pipeline data association;
presenting and counting isolated point data;
presentation of incomplete racking/cabling pipeline data;
GIS presentation of the tour-inspection route;
and presenting the inspection result.
4. A method of analyzing fiber optic cable line data, the method being implemented based on the fiber optic cable line data analysis system of any one of claims 1-3, the method comprising:
collecting stock optical cable pipeline data as a batch processing data analysis basis and recording service data as a real-time data analysis basis;
batch processing is carried out on the stored optical cable pipeline data, and real-time processing is carried out on the recorded service data;
obtaining an optical cable pipeline data inspection strategy according to the data analyzed by the data analysis module;
presenting results according to the processing data of the data acquisition module, the data analysis module and the data inspection module;
wherein the batch processing of the stock of optical cable pipeline data and the real-time processing of the logging service data further comprise: real-time data analysis processing is carried out on the input service data through a Storm cluster, so that important patrol optical cable pipeline data are obtained;
the real-time data analysis processing is carried out on the input service data through the Storm cluster, and the obtaining of the important patrol optical cable pipeline data further comprises the following steps:
counting the number of current data bearing services;
analyzing the concentration of the positions of the optical cable pipelines;
and analyzing whether the piece of data is on the construction road.
5. The method of claim 4, wherein the batch processing of the stock of fiber optic cable line data and the real-time processing of the logging service data further comprises:
and carrying out batch data analysis on the stored optical cable pipeline data through the Hadoop cluster to obtain isolated point data, incompletely laid optical cable pipelines and incompletely put on-shelf optical cable pipeline data in the optical cable pipeline data.
6. The method of claim 4, wherein the presenting results from the processed data of the data acquisition module, the data analysis module, and the data inspection module further comprises:
presenting the optical cable pipeline resources of the Internet GIS map;
presenting a cable pipeline data association;
presenting and counting isolated point data;
presentation of incomplete racking/cabling pipeline data;
GIS presentation of the tour-inspection route;
and presenting the inspection result.
7. A server, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
the memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to the method for analyzing fiber optic cable line data as set forth in any one of claims 4-6.
8. A computer storage medium having stored therein at least one executable instruction that causes a processor to perform operations corresponding to the method for optical cable pipeline data analysis as recited in any one of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910356217.9A CN111865636B (en) | 2019-04-29 | 2019-04-29 | Optical cable pipeline data analysis system, method, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910356217.9A CN111865636B (en) | 2019-04-29 | 2019-04-29 | Optical cable pipeline data analysis system, method, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111865636A CN111865636A (en) | 2020-10-30 |
| CN111865636B true CN111865636B (en) | 2023-04-28 |
Family
ID=72965444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910356217.9A Active CN111865636B (en) | 2019-04-29 | 2019-04-29 | Optical cable pipeline data analysis system, method, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111865636B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114584208B (en) * | 2022-04-28 | 2022-07-05 | 高勘(广州)技术有限公司 | Method, device and equipment for checking optical cable facility points and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8655167B1 (en) * | 2011-01-05 | 2014-02-18 | Google Inc. | Fiber diagnosis system for point-to-point optical access networks |
| CN107181522A (en) * | 2017-05-18 | 2017-09-19 | 云南电网有限责任公司红河供电局 | Based on patterned optical cable intelligent total management operational system |
| CN109344218A (en) * | 2018-10-15 | 2019-02-15 | 中国电信股份有限公司广东传送网络运营中心 | Lightguide cable link method for inspecting and its system |
| CN109542859A (en) * | 2018-10-18 | 2019-03-29 | 天津大学 | A kind of Information Maritime processing model based on cloud computing |
-
2019
- 2019-04-29 CN CN201910356217.9A patent/CN111865636B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8655167B1 (en) * | 2011-01-05 | 2014-02-18 | Google Inc. | Fiber diagnosis system for point-to-point optical access networks |
| CN107181522A (en) * | 2017-05-18 | 2017-09-19 | 云南电网有限责任公司红河供电局 | Based on patterned optical cable intelligent total management operational system |
| CN109344218A (en) * | 2018-10-15 | 2019-02-15 | 中国电信股份有限公司广东传送网络运营中心 | Lightguide cable link method for inspecting and its system |
| CN109542859A (en) * | 2018-10-18 | 2019-03-29 | 天津大学 | A kind of Information Maritime processing model based on cloud computing |
Non-Patent Citations (1)
| Title |
|---|
| 基于WebGIS的电力通信光缆资源管理平台设计与应用分析;王向东等;《信息通信》;20181231(第12期);第244-246页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111865636A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7739138B2 (en) | Automated utility supply management system integrating data sources including geographic information systems (GIS) data | |
| CN110147398A (en) | A kind of data processing method, device, medium and electronic equipment | |
| Yadranjiaghdam et al. | Developing a real-time data analytics framework for twitter streaming data | |
| CN105631026A (en) | Security data analysis system | |
| CN106940677A (en) | One kind application daily record data alarm method and device | |
| CN112241402A (en) | Empty pipe data supply chain system and data management method | |
| CN113254466A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110162512A (en) | A kind of log searching method, apparatus and storage medium | |
| CN112287195A (en) | News media processing method and system, computer storage medium and processor | |
| CN115423429A (en) | Multimode integrated distribution network operation system based on image and sound information | |
| CN111865636B (en) | Optical cable pipeline data analysis system, method, server and storage medium | |
| CN106257447A (en) | The video storage of cloud storage server and search method, video cloud storage system | |
| CN112232662A (en) | Service monitoring system and method | |
| CN112231274A (en) | Log summarizing method and device, electronic equipment and storage medium | |
| CN115439015A (en) | Local area power grid data management method, device and equipment based on data middleboxes | |
| CN108494668A (en) | A kind of method and apparatus executing email processing jobs | |
| CN114723397A (en) | Flow execution method and device | |
| CN114756301A (en) | Log processing method, device and system | |
| KR101878291B1 (en) | Big data management system and management method thereof | |
| US20190347838A1 (en) | Passing system with an interactive user interface | |
| CN111209290A (en) | Land patrol management system based on Internet of things | |
| Manick et al. | ERP and GIS relevance for Fire Rescue Services | |
| CN112463587B (en) | Ice service monitoring method based on QOS | |
| Ning et al. | Application Research of Big Data in the Fifth Generation Mobile Communication Technology | |
| CN109426433A (en) | Read-write Catrol, car networking data processing method and service background, car networking system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |