CN103500173A - Method for inquiring rail transit monitoring data - Google Patents

Method for inquiring rail transit monitoring data Download PDF

Info

Publication number
CN103500173A
CN103500173A CN201310395393.6A CN201310395393A CN103500173A CN 103500173 A CN103500173 A CN 103500173A CN 201310395393 A CN201310395393 A CN 201310395393A CN 103500173 A CN103500173 A CN 103500173A
Authority
CN
China
Prior art keywords
data
index
data file
monitoring
monitoring data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201310395393.6A
Other languages
Chinese (zh)
Other versions
CN103500173B (en
Inventor
鲍侠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BEIJING TAILEDE INFORMATION TECHNOLOGY Co Ltd
Original Assignee
BEIJING TAILEDE INFORMATION TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BEIJING TAILEDE INFORMATION TECHNOLOGY Co Ltd filed Critical BEIJING TAILEDE INFORMATION TECHNOLOGY Co Ltd
Priority to CN201310395393.6A priority Critical patent/CN103500173B/en
Publication of CN103500173A publication Critical patent/CN103500173A/en
Application granted granted Critical
Publication of CN103500173B publication Critical patent/CN103500173B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method for inquiring rail transit monitoring data. The method for inquiring the rail transit monitoring data comprises the steps that (1) a data storage system checks newly stored monitoring data files in real time and coverts the newly stored monitoring data files into a binary mode to be stored, wherein a cloud computing platform is adopted in the data storage system, and the data storage system comprises a distributed storage system and an index inquiring system; (2) the data storage system carries out parallel processing on the monitoring data files, an index is created for each monitoring data file, and index data corresponding to all the monitoring data files are inserted into the index inquiring system; (3) according to input monitoring inquiring requests, the index inquiring system obtains a monitoring data file list through inquiring and reads monitoring data records from the monitoring data files in the distributed storage system according to the monitoring data file list. The method for inquiring the rail transit monitoring data greatly improves the creating speed of the indexes and inquiring efficiency.

Description

A kind of querying method of track traffic Monitoring Data
Technical field
The invention provides a kind of querying method of track traffic Monitoring Data, belong to the cloud computing technology field.
Background technology
Along with China railways comprises the Large scale construction of government railway, enterprise railway and City Rail Transit System, railway equipment, particularly the technology of Source of Railway Communication and Signalling equipment is more and more advanced, but railway electrical maintenance maintenance technology is not followed the Large scale construction of railway and synchronized development causes railway electrical maintenance maintenance technology to become the technology weak link of railway territory.Two main market players of railway electrical: railway electrical department (railway main office, Railway Bureau, electricity business section, electricity business workshop, electricity work district) and Source of Railway Communication and Signalling equipment vendors, bearing huge in linear pressure, bearing again the O&M pressure of the day by day heavy equipment of reaching the standard grade, they have the urgently satisfied market demand for the e-manufacturing instrument that improves railway electrical maintenance maintenance technical merit and production safety ability.
There is multiple monitoring system for Source of Railway Communication and Signalling equipment at present, the truck-mounted computer (ATP) that comprises the high ferro signal monitoring, railway wireless network (GSM-R), radio block center (RBC), temporary speed limitation (TSRS) and existing signal monitoring (CSM): interlocking (CBI), dispatching concentration (CTC) He Liekong center (TCC) and the monitoring of other basis signal, comprise track switch, track circuit, teleseme and power supply panel etc., except these equipment monitoring systems, AT STATION, electricity business section, city rail cluster rolling stock section and gauge lines center, city also all disposed the CSM system, be used for managing concentratedly Monitoring Data and the warning information of various device, and notify electric business personnel to carry out maintenance maintenance according to these information.
At present, the subject matter that railway electrical department exists is to continue to use traditional plan to repair the electricity business mode of production of repairing with fault in electricity business production run, safeguarding numerous advanced persons' communication and signal equipment, inevitably, according to the blind area that has become hidden trouble of equipment, blind spot, electricity business production field ubiquity surplus and is repaiied and phenomenon in bad repair.Surplus is repaiied, and what bring is the huge waste of person property's power; And in bad repair, may bring the driving accident of car crash.Vast railway electrical personnel, under heavy psychological pressure and labour intensity load, safeguarding with manpower and can't bear the heavy load the normal operation of communication and signal equipment.They are in the urgent need to a set of tool of production that can separate electric discharge business yield-power---synthesization, intelligentized electricity business monitoring O&M system, thereby repair, plan to repair the traditional electricity business safety in production O&M pattern into guiding by current with fault, progressively change into state and repair the informationalized electricity business safety in production O&M pattern into guiding, thereby realize looking into hidden danger, controlling hidden danger, hidden trouble of equipment is eliminated in bud, realized the electricity business Safety Production Target of guarantee driving safety.
The subject matter that Source of Railway Communication and Signalling equipment vendors exist at present is, the relative dispersion of monitoring information and isolated, make the utilization of monitoring information very inconvenient, and utilization factor is not high.Be relatively independent information island each other between the monitoring information that the part monitoring system produces, also do not realize concentrating monitoring fully; The management method of existing track traffic Monitoring Data is that data used are all adopted to relational database, as oracle database, carries out store and management.In order to tackle the data of magnanimity, usually adopt the cluster formed by a plurality of database servers, as Oracle RAC.In relational database, data are highly independently, and along with the increase gradually of monitoring and storage data volume, the data store and management mode of relational database cluster has following deficiency:
Being stored current topmost method for high-volume database is to use 3 grades of storeies to be stored and walk abreast to store and inquiring technology.But these methods are for the expense of hardware larger for the weak point of the storage maximum of data, and need the special Database Systems of exploitation to manage it, and their the most essential thinking is exactly to expand hardware device to obtain large storage space, the processing time of the inquiry greatly increased when adding large storage capacity.
Although the processing speed of the high speed that the parallel database technology utilizes a plurality of processors to obtain, cost is to increase hardware spending, and the growth rate of processing will be lower than the processing speed of hardware device.When data volume is very large, the time overhead of the action needs such as data query and analysis costs is very large, can't meet rapid data and browse the needs with analyzing and processing.
Summary of the invention
For the problems referred to above, the purpose of this invention is to provide a kind of querying method of track traffic Monitoring Data, improve the search efficiency of various resources and track traffic Monitoring Data by cloud platform, cloud memory technology.
For achieving the above object, the technical solution used in the present invention is as follows:
A kind of querying method of track traffic Monitoring Data, the steps include:
1) the numeric type monitor data file of the new storage of data-storage system real-time inspection, and be converted into binary mode and stored; Wherein, data-storage system adopts cloud computing platform, and it comprises a distributed memory system and a search index system;
2) data-storage system carries out parallel processing to described monitor data file, each is detected to data file and create an index, and corresponding index data is inserted in described search index system by each monitor data file;
3) described search index system is according to the detection inquiry request of input, and inquiry obtains the monitor data file list, then according to this detection data file list, reads the Monitoring Data record from the monitor data file of described distributed memory system.
Further, described distributed memory system is the HDFS storage system, and described search index system is Hbase search index system.
Further, described cloud computing platform is provided with host node and some child nodes; Utilize host node to check the monitor data file of new storage in distributed HDFS storage system, and create the index data of this detection data file; Described cloud computing platform is distributed to by described monitor data file the parallel establishment that different child nodes is carried out index.
Further, described distributed memory system comprises a meta data file, and described distributed memory system extracts the metadata of monitor data file, and it is stored in this meta data file as a data record.
Further, described metadata comprises side-play amount, the monitoring file size of field in this monitor data file in filename, monitor data file.
Further, describedly each detected to the method that data file creates an index be:
1) child node, to the distributed newly-built data stream of each monitor data file index creation task, is recorded the filename of this monitor data file simultaneously;
2) child node is extracted metadata according to this document name from described meta data file, and according to this metadata record, this monitoring file data is resolved, and utilizes described side-play amount to extract the index field of setting it is added in the index data list;
3) child node is set up a concordance list to each index field in the index data list; Information in described concordance list comprises: the memory address of index field, data acquisition time, Monitoring Data record.
Further, the line unit of described concordance list is arranged according to the syllable sequence order.
Further, described cloud computing platform adopts Zookeeper that host node and child node are set, and described cloud computing platform is carried out to load balancing.
Further, described monitor data file comprises the equipment state record of some collections; Described equipment state record comprises the time of data acquisition, and some switching values of corresponding monitoring of equipment signal and the value of analog quantity under this time.
Further, described monitoring inquiry request comprises time range and the monitoring parameter of Monitoring Data.
The track traffic Monitoring Data is with the form generation of the switching value of monitoring of equipment signal and analog quantity and preserves.Then data acquisition system (DAS) is merged into an equipment state record by the switching value in a certain moment and analog data, is stored in the centralized monitoring system of each electricity business section.This equipment state record has comprised the time of data acquisition, and some switching values of corresponding monitoring of equipment signal and the value of analog quantity under this time.Comprehensive O&M platform is collected and concentrated the Monitoring Data of each electricity business section is stored in data-storage system.Data-storage system in the present invention adopts cloud computing platform, utilize numerous X86-based computers, foundation has the distributed cloud computing platform of good reliability and extensibility, can carry out the Real-Time Monitoring processing to the Monitoring Data up to the PB level, the multiple business supports such as real-time query analysis of Monitoring Data are provided, and are the framework of Monitoring Data collection and inquiry system in the device data monitoring system shown in Fig. 1.
As shown in Figure 1, Monitoring Data gathers out from the device signal acquisition system continuously, and then these raw data will be processed through synthesis system, to generate the Monitoring Data record.Synthetic Monitoring Data record is what with the form close to text data, to mean, comprises the dozens of data field in each record, as information such as track circuit voltage, track circuit phasing degree.Monitoring Data usually be take a regular time (as one minute) and is remained in the database of data acquisition system (DAS) as unit.Therefore, per minute all can have Monitoring Data to produce, and, according to the difference of equipment frequency acquisition, can comprise a lot of Monitoring Data records in the monitor database of every day.The storage system of these data of centralized stores more needs to safeguard the Monitoring Data of storage in a plurality of electricity business sections.
In order to be inquired about these Monitoring Data records, need to be converted into binary mode.Adopting the advantage of binary storage is to save storage space, and can store data with fixed byte length.Because side-play amount between the record of binary mode is fixed value, therefore be convenient to the inquiry that creates index and be correlated with.Table 1 is a kind of concrete form of raw readings structured storage, is converted into binary mode.Store and management for these structural datas, solution of the present invention is: Monitoring Data be take to every day as the synthetic data file of unit, be stored in the HDFS system, simultaneously for the fast query processing power is provided, to set up search index for the Monitoring Data record, and these search indexes will be left in HBase.
The concrete form that table 1 is the raw readings structured storage
Numbering Device name Merit goes out voltage (volt) Merit goes out electric current (milliampere) Carrier frequency (hertz) Low frequency (hertz)
1 3DG 114.40 358.00 2298.70 27.80
2 7DG 114.40 415.00 2598.70 27.90
3 9DG 78.60 173.00 1701.40 26.70
4 13DG 76.20 202.00 2001.40 27.90
5 17DG 111.50 355.00 2298.70 27.80
For this reason, to from gathering and synthesis system, the monitor data file that per minute produces continually, need to check these data files in real time, and carry out in time the index creation processing.Index data comprises acquisition time, monitoring parameter and the corresponding memory address of Monitoring Data.
Because the Monitoring Data amount is huge, need to use multiple servers to carry out parallel processing, so just need a supvr to carry out the newly-generated monitor data file of scanography, and the index creation task of different monitor data files is distributed to different nodes and go to process.Therefore, need to consider to carry out parallelization task scheduling and load balance process on the calculating cluster, complete above search index Processing tasks.
In concrete design, overall plan can be divided into the two large divisions.
First is that reading of monitor data file and the search index based on HBase create.In this process, a host node need to be set for detection of monitor data file new in HDFS, then, this host node will be distributed to different child nodes to the monitor data file detected and complete the search index establishment, and every index data corresponding to monitoring record is inserted in HBase.In order to prevent that the host node single node lost efficacy and handled load balancing well, managed whole calculating cluster with Zookeeper.
Second portion is based on the search index of HBase, receive and process user's monitoring inquiry request (inquiry request comprises time range and the monitoring parameter of Monitoring Data), complete concrete query processing in HBase, after obtaining Monitoring Data record queries list as a result, read detailed Monitoring Data record again in the monitor data file that is stored in HDFS, and return to one by one client.
Compared with prior art, good effect of the present invention is:
(1) adopt the document data bank system to store large-scale data, solved the cost of relational database cluster maintenance and expansion.
(2) structure data is stored by binary form, reduced to take the space of storage.
(3) adopt parallel method to set up data directory, improved the establishment speed of index, and improved the efficiency of inquiry.
The accompanying drawing explanation
Fig. 1 track traffic Monitoring Data Management System structural framing figure;
Fig. 2 has showed above Monitoring Data query processing flow process.
Embodiment
Below by specific embodiments and the drawings, the present invention is described in detail.
The structural framing of track traffic Monitoring Data Management System as shown in Figure 1 comprises following major part:
(1) storage of the distributed document based on HDFS
The Main Function of this part is to provide the storage platform of Monitoring Data.Distributed file system is positioned at the bottom of whole system, and it provides unified memory access interface to computation model layer, database layer, and other module is by the access data in distributed file system easily that calls of interface.Simultaneously, the mechanism such as the replication policy that this layer provides, load balancing have also guaranteed availability and the reliability of storage platform, for whole system provides stable, dependable storage space.
(2) index management based on HBase
The Main Function of this part is to provide the data storage and supports, can provide the data persistence function to operation layer and computation layer.And, by rational major key setting and Indexing Mechanism, provide efficient data access capabilities to upper strata.Adopt HBase as database, can meet that the system height is concurrent, the demand of enhanced scalability.About the data of monitoring equipment information, store also in HBase simultaneously.
(3) management of the distributed scheduling based on Zookeeper
The Main Function of this part is that the server cluster to whole system manages, and the global configuration information management function is provided simultaneously.By calling the index generating algorithm, to being stored in the data parallel ground generating indexes in distributed file system.
(4) data analysis based on MapReduce
The Main Function of this part is to adopt the data analysis algorithm to being stored in the data analysis in distributed file system, and the output analysis result.
(5) data memory interface
The Main Function of this part is to receive from data acquisition system (DAS) the equipment condition monitoring data that gather, and stores data into distributed file system.
(6) data query interface
The Main Function of this part is to receive the data query request from the data exhibiting layer, inquires about required data from distributed file system by data directory, and the data return data is represented to layer.
Implementation step of the present invention is as follows:
(1) storage of Monitoring Data
The Monitoring Data obtained from data acquisition system (DAS), the form of its storage is as shown in table 1.For the space of saving storage and the efficiency that improves inquiry, these structurized data are used as to the type of storage with the structured data type of a set form.These structured data types have comprised the data type of each and the side-play amount in structure, and these information are kept in meta data file together with the zero-time of storage data in each file.Each data supplementing constantly that data memory interface will obtain stores the ending of the respective file of distributed file system into.When file size reaches limit value, store data into a new file, and generate a new data recording in metadata.
(2) detection of monitor data file and index creation task scheduling.
Monitoring Data is with the form tissue of file, and the processing of present stage is directly data to be stored in file with binary form, does not comprise any redundant data.Follow-up inquiry will be obtained data by filename and side-play amount.Because Monitoring Data is all the time all to produce, we are new monitoring file in scanning system in real time, then monitor data file is assigned to different index creation nodes.Fault-tolerant for load balancing and the single-point of realizing system, will monitor file scan service and index creation service distribution on different nodes, so just need a supvr Zookeeper manage these clusters.
Based on calculating cluster, completing unified Basic Design and the implementation method of controlling with Zookeeper is, all nodes in the calculating cluster are all included in to the management of Zookeeper, select wherein two computing nodes to be registered as the Master service node (being responsible for the scanning of new monitor data file and the scheduling distribution of index creation task) of Zookeeper, Zookeeper is a primary server joint of election in two server nodes automatically.When primary server joint breaks down, Zookeeper can be promoted to by the server node be left the master server that primary server joint is carried out taking over failing automatically, with this, guarantee that distribution and the dispatch deal of parallel computation task not there will be single point failure.
Select the node of a free time in the current child node effectively that master server is safeguarded in Zookeeper, and an index creation task is distributed to this node; When a plurality of index creation tasks arrive, host node can be distributed to calculation task different computing nodes effectively, realizes the load balancing of parallel computation task on whole clustered node.
(3) from the HDFS reading out data and create index.
In order to create search index, needs are read to the monitor data file be stored in HDFS, and each the Monitoring Data record in file is lined by line scan, extract the field commonly used that need to set up index and import in HBase.
This step comprises:
1) Monitoring Data is lined by line scan.
Whenever child node receives a monitor data file index creation task, program will a newly-built data stream, and the monitoring file data that will read from HDFS reads in internal memory, records the title (being made as fileName) of this document simultaneously in program.Resolve interface by call request, extract metadata from the meta data file of distributed memory system: side-play amount, the monitoring file size of field in monitor data file in filename, monitor data file, these metadata are passed to raw data file and resolve interface method, resolve interface by raw data file and can be processed data.
2) field commonly used is built to table.
According to metadata, original detection data file is resolved, extract the Monitoring Data record, the Monitoring Data record extracted is lined by line scan, the Monitoring Data record in the appointment fileName file that will read from HDFS reads in internal memory.In the Monitoring Data record, some field is inoperative to searching processing, therefore, only needs the index fields commonly used such as the voltage relevant to equipment state, electric current are extracted, and adds in the index data list.
3) set up concordance list.
Therefore only every Monitoring Data records tens fields, and wherein for the field of inquiry, only has severally, need to set up several concordance lists in HBase according to these fields.Below to take by track circuit Voltage Establishment concordance list be example.In concordance list ConstructRailVoltageIndexItem, the information in each concordance list comprises " memory address (being Index) of index field (such as track circuit voltage)+data acquisition time+record ".
(4) inquiry Monitoring Data information.
In HBase, the line unit of a table (rowkey) is arranged according to the syllable sequence order.
Here as follows to the design of concordance list: rowkey is: the field of inquiry.Row bunch: Offset: Monitoring Data is recorded in the position in monitor data file, and filename adds side-play amount.The row name here is set to sky, in order to reduce data volume, improves insertion speed.
Query steps is as follows:
1) for the given query condition, be spliced into rational inquiry syllable sequence (with rowkey), can be directly targeted to the upper rowkey of this rowkey or this rowkey by the API of HBase, so just navigate to fast the Monitoring Data index data satisfied condition.
2) read follow-up data, read the Monitoring Data positional information satisfied condition, search index record successively, when finding that rowkey does not satisfy condition, search index is complete.
3) according to obtained all Monitoring Data positional information set that satisfy condition, read out all Monitoring Data records in the corresponding monitor data file from HDFS.
The exploitation of this example and running environment, mainly used HDFS, HBase and Zookeeper.HDFS is for storing primary monitoring data, and HBase is for the storing queries index, and Zookeeper is for management cluster, scheduling index creation task.Wherein the function of interface module is as follows:
(1) monitor data file detects and the index creation task dispatch.
The host node server is responsible for monitor data file and is detected and the index creation task scheduling, and its main processing capacity is:
1) detect and whether produce new monitor data file;
2) filename of new monitor data file is integrated into to the index creation request, is distributed to child node and processes;
3) if child node lost efficacy, the index creation request on child node is transferred on other child nodes.
The function of host node service routine part is as follows:
1) a certain moment, only have a service routine that service is provided.
2) solve Single Point of Faliure.
Solved the Single Point of Faliure problem of host node by Zookeeper: if the primary server joint scanned had lost efficacy, other standby host nodes are taken over its work, continue the new file of scanning; If a certain host node had lost efficacy, the file destination below /APP/Server/TMP will change, and monitor that each standby host node of this catalogue will be received an event request, then again choose new host node; The standby host node of not chosen continues to wait for the arrival of new event.
The function that child node is partly monitored the request of processing is as follows:
Whether 1) based on Zookeeper, detect request arrives.
Whether 2) detect this request is rational request.
(2) reading Monitoring Data and index creation processes.
Over against the index creation task requests of accepting, child node reads the Monitoring Data of HDFS, and concrete function is as follows:
1) read the HDFS file.
2) process the data in the HDFS file and insert in the HBase concordance list.
The function that Monitoring Data is scanned and creates to index is as follows:
The generating indexes data, and insert in HBase.
Illustrate: every row of concordance list comprises following data:
1) rowkey: form is rail_voltage(8 byte)+a sample_time_s(4 byte)+an index(4 byte).
2) row bunch: Offset, row bunch in comprise row (these row are unnamed).Form is: the side-play amount (8 byte) of a filename ID(8 byte)+data in file.
(3) Monitoring Data inquiry.
The major function of Monitoring Data inquiry is as follows:
Concordance list in inquiry HBase.
Illustrate:
(1) generate interface by the line unit value Monitoring Data field (as, track circuit voltage) is converted into to the syllable sequence identical with the rowKey form of concordance list with the time, for example, track circuit voltage+data acquisition time.Find by the data file scan interface a line that rowkey is identical with this syllable sequence, if there is no point to previous row.
(2) inquiry is set and points to qualified row.
(3) ergodic data backward, if data do not meet querying condition (track circuit voltage and sampling time), inquire about complete.
(4) in this example, the form of index is: track circuit voltage+sampling time+index.If number has surpassed, directly exit; If track circuit voltage conforms condition, whether detection time is eligible.

Claims (10)

1. the querying method of a track traffic Monitoring Data, the steps include:
1) the numeric type monitor data file of the new storage of data-storage system real-time inspection, and be converted into binary mode and stored; Wherein, data-storage system adopts cloud computing platform, and it comprises a distributed memory system and a search index system;
2) data-storage system carries out parallel processing to described monitor data file, each is detected to data file and create an index, and corresponding index data is inserted in described search index system by each monitor data file;
3) described search index system is according to the detection inquiry request of input, and inquiry obtains the monitor data file list, then according to this detection data file list, reads the Monitoring Data record from the monitor data file of described distributed memory system.
2. the method for claim 1, is characterized in that described distributed memory system is the HDFS storage system, and described search index system is Hbase search index system.
3. method as claimed in claim 2, is characterized in that described cloud computing platform is provided with host node and some child nodes; Utilize host node to check the monitor data file of new storage in distributed HDFS storage system, and create the index data of this detection data file; Described cloud computing platform is distributed to by described monitor data file the parallel establishment that different child nodes is carried out index.
4. method as described as claim 1 or 3, is characterized in that described distributed memory system comprises a meta data file, and described distributed memory system extracts the metadata of monitor data file, and it is stored in this meta data file as a data record.
5. method as claimed in claim 4, is characterized in that described metadata comprises side-play amount, the monitoring file size of field in this monitor data file in filename, monitor data file.
6. method as claimed in claim 5 is characterized in that describedly each is detected to the method that data file creates an index being:
1) child node, to the distributed newly-built data stream of each monitor data file index creation task, is recorded the filename of this monitor data file simultaneously;
2) child node is extracted metadata according to this document name from described meta data file, and according to this metadata record, this monitoring file data is resolved, and utilizes described side-play amount to extract the index field of setting it is added in the index data list;
3) child node is set up a concordance list to each index field in the index data list; Information in described concordance list comprises: the memory address of index field, data acquisition time, Monitoring Data record.
7. method as claimed in claim 6, is characterized in that the line unit of described concordance list is arranged according to the syllable sequence order.
8. method as claimed in claim 3, is characterized in that described cloud computing platform adopts Zookeeper that host node and child node are set, and described cloud computing platform carried out to load balancing.
9. as claim 1 or 2 or 3 described methods, it is characterized in that described monitor data file comprises the equipment state record of some collections; Described equipment state record comprises the time of data acquisition, and some switching values of corresponding monitoring of equipment signal and the value of analog quantity under this time.
10. as claim 1 or 2 or 3 described methods, it is characterized in that described monitoring inquiry request comprises time range and the monitoring parameter of Monitoring Data.
CN201310395393.6A 2013-09-03 2013-09-03 A kind of querying method of track traffic Monitoring Data Active CN103500173B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310395393.6A CN103500173B (en) 2013-09-03 2013-09-03 A kind of querying method of track traffic Monitoring Data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310395393.6A CN103500173B (en) 2013-09-03 2013-09-03 A kind of querying method of track traffic Monitoring Data

Publications (2)

Publication Number Publication Date
CN103500173A true CN103500173A (en) 2014-01-08
CN103500173B CN103500173B (en) 2017-07-28

Family

ID=49865384

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310395393.6A Active CN103500173B (en) 2013-09-03 2013-09-03 A kind of querying method of track traffic Monitoring Data

Country Status (1)

Country Link
CN (1) CN103500173B (en)

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103795572A (en) * 2014-01-24 2014-05-14 北京京东尚科信息技术有限公司 Method for switching master server and slave server and monitoring server
CN103970842A (en) * 2014-04-24 2014-08-06 河海大学 Water conservancy big data access system and method for field of flood control and disaster reduction
CN104320486A (en) * 2014-11-10 2015-01-28 连云港杰瑞电子有限公司 Intelligent traffic platform data integration method based on big data
CN105574593A (en) * 2015-12-18 2016-05-11 中南大学 Track state static-state detection and control system and method based on cloud computing and big data
CN105592484A (en) * 2015-12-18 2016-05-18 潘小胜 Railway interlock system performance optimizing apparatus
CN105630847A (en) * 2014-11-21 2016-06-01 深圳市腾讯计算机系统有限公司 Data storage method as well as data query method, apparatus and system
CN105930426A (en) * 2016-04-18 2016-09-07 华信咨询设计研究院有限公司 Wireless monitoring data query method
CN105930441A (en) * 2016-04-18 2016-09-07 华信咨询设计研究院有限公司 Query method of radio monitoring data
CN106485514A (en) * 2016-11-01 2017-03-08 安徽拾穗谷生态科技有限公司 A kind of agricultural product quality and safety reviews big data processing method
CN107463706A (en) * 2017-08-18 2017-12-12 国网上海市电力公司 A kind of storage of magnanimity recorder data and parsing method and system based on Hadoop
CN107807608A (en) * 2017-11-02 2018-03-16 腾讯科技(深圳)有限公司 Data processing method, data handling system and storage medium
CN108737503A (en) * 2018-04-25 2018-11-02 江苏鸣鹤云科技有限公司 A kind of efficient big data distributed transmission system and method
CN108959527A (en) * 2018-06-28 2018-12-07 卡斯柯信号有限公司 The method for reading display interlocking log based on Windows file mapping technology
CN109783449A (en) * 2018-12-13 2019-05-21 深圳壹账通智能科技有限公司 Data query processing method, platform, system and readable storage medium storing program for executing
CN110022257A (en) * 2018-01-08 2019-07-16 北京京东尚科信息技术有限公司 Distributed information system
CN110795498A (en) * 2019-09-16 2020-02-14 华东交通大学 Railway power supply dispatching cluster monitoring system visualization method based on column database reverse index
CN111125171A (en) * 2019-12-22 2020-05-08 浪潮(北京)电子信息产业有限公司 Monitoring data access method, device, equipment and readable storage medium
CN111737255A (en) * 2020-06-02 2020-10-02 通号城市轨道交通技术有限公司 Method and system for storing interlocking monitoring data
CN113839919A (en) * 2021-08-06 2021-12-24 上海富欣智能交通控制有限公司 Transmission data structure configuration method, data transceiving method and communication system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070033340A1 (en) * 2005-08-08 2007-02-08 International Business Machines Corporation System and method for providing content based anticipative storage management
CN101917237A (en) * 2010-07-27 2010-12-15 北京全路通信信号研究设计院 Railway signal monitoring method and system
CN101950297A (en) * 2010-09-10 2011-01-19 北京大学 Method and device for storing and inquiring mass semantic data
CN102063486A (en) * 2010-12-28 2011-05-18 东北大学 Multi-dimensional data management-oriented cloud computing query processing method
CN102508913A (en) * 2011-11-17 2012-06-20 张真 Cloud computing system with data cube storage index structure

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070033340A1 (en) * 2005-08-08 2007-02-08 International Business Machines Corporation System and method for providing content based anticipative storage management
CN101917237A (en) * 2010-07-27 2010-12-15 北京全路通信信号研究设计院 Railway signal monitoring method and system
CN101950297A (en) * 2010-09-10 2011-01-19 北京大学 Method and device for storing and inquiring mass semantic data
CN102063486A (en) * 2010-12-28 2011-05-18 东北大学 Multi-dimensional data management-oriented cloud computing query processing method
CN102508913A (en) * 2011-11-17 2012-06-20 张真 Cloud computing system with data cube storage index structure

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103795572A (en) * 2014-01-24 2014-05-14 北京京东尚科信息技术有限公司 Method for switching master server and slave server and monitoring server
CN103795572B (en) * 2014-01-24 2017-07-21 北京京东尚科信息技术有限公司 The switching method and monitoring server of principal and subordinate's server
CN103970842A (en) * 2014-04-24 2014-08-06 河海大学 Water conservancy big data access system and method for field of flood control and disaster reduction
CN104320486A (en) * 2014-11-10 2015-01-28 连云港杰瑞电子有限公司 Intelligent traffic platform data integration method based on big data
CN105630847B (en) * 2014-11-21 2019-06-07 深圳市腾讯计算机系统有限公司 Date storage method, data query method, apparatus and system
CN105630847A (en) * 2014-11-21 2016-06-01 深圳市腾讯计算机系统有限公司 Data storage method as well as data query method, apparatus and system
CN105574593A (en) * 2015-12-18 2016-05-11 中南大学 Track state static-state detection and control system and method based on cloud computing and big data
CN105592484A (en) * 2015-12-18 2016-05-18 潘小胜 Railway interlock system performance optimizing apparatus
CN105574593B (en) * 2015-12-18 2020-05-05 中南大学 Track state static detection and control system and method based on cloud computing and big data
CN105930441A (en) * 2016-04-18 2016-09-07 华信咨询设计研究院有限公司 Query method of radio monitoring data
CN105930426A (en) * 2016-04-18 2016-09-07 华信咨询设计研究院有限公司 Wireless monitoring data query method
CN105930441B (en) * 2016-04-18 2019-04-26 华信咨询设计研究院有限公司 A kind of radio monitoring data query method
CN105930426B (en) * 2016-04-18 2019-03-08 华信咨询设计研究院有限公司 Radio monitoring data query method
CN106485514A (en) * 2016-11-01 2017-03-08 安徽拾穗谷生态科技有限公司 A kind of agricultural product quality and safety reviews big data processing method
CN107463706B (en) * 2017-08-18 2020-06-23 国网上海市电力公司 Hadoop-based mass wave recording data storage and analysis method and system
CN107463706A (en) * 2017-08-18 2017-12-12 国网上海市电力公司 A kind of storage of magnanimity recorder data and parsing method and system based on Hadoop
CN107807608A (en) * 2017-11-02 2018-03-16 腾讯科技(深圳)有限公司 Data processing method, data handling system and storage medium
CN110022257A (en) * 2018-01-08 2019-07-16 北京京东尚科信息技术有限公司 Distributed information system
CN108737503A (en) * 2018-04-25 2018-11-02 江苏鸣鹤云科技有限公司 A kind of efficient big data distributed transmission system and method
CN108959527A (en) * 2018-06-28 2018-12-07 卡斯柯信号有限公司 The method for reading display interlocking log based on Windows file mapping technology
CN108959527B (en) * 2018-06-28 2023-06-09 卡斯柯信号有限公司 Method for reading and displaying interlocking log based on Windows file mapping technology
CN109783449A (en) * 2018-12-13 2019-05-21 深圳壹账通智能科技有限公司 Data query processing method, platform, system and readable storage medium storing program for executing
CN110795498A (en) * 2019-09-16 2020-02-14 华东交通大学 Railway power supply dispatching cluster monitoring system visualization method based on column database reverse index
CN111125171A (en) * 2019-12-22 2020-05-08 浪潮(北京)电子信息产业有限公司 Monitoring data access method, device, equipment and readable storage medium
CN111737255A (en) * 2020-06-02 2020-10-02 通号城市轨道交通技术有限公司 Method and system for storing interlocking monitoring data
CN113839919A (en) * 2021-08-06 2021-12-24 上海富欣智能交通控制有限公司 Transmission data structure configuration method, data transceiving method and communication system
CN113839919B (en) * 2021-08-06 2023-04-18 上海富欣智能交通控制有限公司 Transmission data structure configuration method, data transceiving method and communication system

Also Published As

Publication number Publication date
CN103500173B (en) 2017-07-28

Similar Documents

Publication Publication Date Title
CN103500173A (en) Method for inquiring rail transit monitoring data
WO2016004775A1 (en) Method and system for integrated operation and maintenance of rail transportation signals based on cloud computing
CN108964996B (en) Urban and rural integrated information grid system and information sharing method based on same
CN107784098A (en) Real-time data warehouse platform
CN105357311A (en) Secondary equipment big data storage and processing method by utilizing cloud computing technology
CN102609463A (en) Data cluster management system based on quasi-realtime platform
CN111930835B (en) Intelligent operation and maintenance big data management system and method for urban rail transit
CN108860223B (en) Data processing system and method
CN104933070A (en) Catalog management system used for government affairs information platform
CN111178742B (en) Comprehensive traffic cooperative operation system and method based on multi-level index system
Liu et al. A big data framework for electric power data quality assessment
CN103955510A (en) Massive electricity marketing data integration method uploaded by ETL cloud platform
CN106649687A (en) Method and device for on-line analysis and processing of large data
CN107798062A (en) A kind of transformer station's historical data unifies storage method and system
Xianglan Digital construction of coal mine big data for different platforms based on life cycle
CN103489139A (en) Comprehensive analysis and management system for urban distribution network planning
CN104320486B (en) A kind of intelligent transportation platform data integrated approach based on big data
Liu et al. A Cloud-computing and big data based wide area monitoring of power grids strategy
Dong et al. Research on Architecture of Power Big Data High-Speed Storage System for Energy Interconnection
CN115276233A (en) Intelligent operation and maintenance system for urban rail subway power supply
CN201757899U (en) Graded control centralized management digital archive
Zhang et al. A Big Data based Decision Framework for Public Management and Service in Tourism
CN112055065A (en) Relay protection SaaS layer application data processing method based on regulation and control cloud
CN110738586A (en) weather integrated service system based on CIMISS and comprehensive database data
Zhang et al. Research on the Construction of Big Data Management Platform of Shuohuang Railway Locomotive Operation and Maintenance

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant