CN111831658A - Rail transit big data analysis method and system - Google Patents
Rail transit big data analysis method and system Download PDFInfo
- Publication number
- CN111831658A CN111831658A CN202010621999.7A CN202010621999A CN111831658A CN 111831658 A CN111831658 A CN 111831658A CN 202010621999 A CN202010621999 A CN 202010621999A CN 111831658 A CN111831658 A CN 111831658A
- Authority
- CN
- China
- Prior art keywords
- data
- analysis
- rail transit
- module
- cloud platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007405 data analysis Methods 0.000 title claims abstract description 43
- 238000000034 method Methods 0.000 title claims abstract description 15
- 238000004458 analytical method Methods 0.000 claims abstract description 53
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 230000002159 abnormal effect Effects 0.000 claims abstract description 5
- 238000004891 communication Methods 0.000 claims description 14
- 238000012937 correction Methods 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 8
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000013501 data transformation Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 3
- 238000010295 mobile communication Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
-
- G06Q50/40—
Abstract
The invention relates to the technical field of big data analysis, in particular to a method and a system for analyzing big data of rail transit, wherein the method comprises the following steps: collecting and summarizing original data of rail transit; storing the acquired rail transit original data; preprocessing the original data of the rail transit, correcting abnormal data, and removing data which cannot be corrected; uploading the preprocessed rail transit data to a cloud platform, and storing and calling the data by the cloud platform; the method comprises the following steps that a user sends a required analysis requirement to a cloud platform through a client, and the cloud platform sends an analysis result to the user client after data analysis; the method and the device can effectively eliminate error information in the acquired data, improve the accuracy of data analysis, perform predictive analysis on the data in a period of time in the future, provide visual data analysis for a user, quickly call after receiving the same analysis request again, and reduce waiting time.
Description
Technical Field
The invention relates to the technical field of big data analysis, in particular to a method and a system for analyzing big data of rail transit.
Background
Rail transit refers to a type of vehicle or transportation system in which operating vehicles need to travel on a particular rail. The most typical rail transit is a railway system consisting of conventional trains and standard railways. With the diversified development of train and railway technologies, rail transit is more and more types, and is not only distributed in long-distance land transportation, but also widely applied to medium-short distance urban public transportation. According to the difference of service ranges, rail transit is generally divided into three major categories of national railway systems, intercity rail transit and urban rail transit. The rail transit generally has the advantages of large transportation volume, high speed, dense shift, safety, comfort, high punctuality rate, all weather, low transportation cost, energy conservation, environmental protection and the like, but is usually accompanied by higher early investment, technical requirements and maintenance cost, and the occupied space is usually larger.
The rapid development of rail transit is very important for analyzing rail transit big data, for example, urban rail transit passenger flow data is analyzed, passenger flow prediction is well made, and the rail transit system is favorable for running management. However, the large scale of the rail transit big data is huge, and how to quickly and accurately analyze the big data of the mass rail transit big data is a problem which needs to be solved urgently.
Based on the above, the invention designs a rail transit big data analysis method and a rail transit big data analysis system to solve the above problems.
Disclosure of Invention
The invention aims to solve the problems in the background art and provides a rail transit big data analysis method and a rail transit big data analysis system.
In order to achieve the purpose, the invention provides the following technical scheme: a rail transit big data analysis method comprises the following steps:
step S1: data acquisition, namely acquiring and summarizing original data of the rail transit;
step S2: storing original data, namely storing the acquired rail transit original data;
step S3: preprocessing data, namely preprocessing the original data of the rail transit, correcting abnormal data and removing the data which cannot be corrected;
step S4: uploading data, namely uploading the preprocessed rail transit data to a cloud platform, and storing and calling the data by the cloud platform;
step S5: and (3) inquiring by the user, sending the required analysis requirement to the cloud platform through the client, and sending the analysis result to the user client after the cloud platform analyzes the data.
Further, the data acquisition mode in step S1 is a distributed data acquisition mode.
Further, the data preprocessing in the step S3 includes data cleaning, data integration, data transformation, and data reduction.
The big data analysis system for the rail transit comprises a data acquisition module, wherein the data acquisition module is in signal connection with an original database, the original database is in signal connection with a preprocessing module, the preprocessing module is in signal connection with a data transmission module, the data transmission module is in signal connection with a cloud platform, the cloud platform is in signal connection with a cloud database, the cloud platform is in bidirectional signal connection with a data analysis module and a wireless communication module, the data analysis module is in bidirectional signal connection with the cloud database, and the wireless communication module is in two-wire signal connection with a client.
Further, the original database is a NoSQL database, and the cloud database is a MySQL database.
Further, the data analysis module comprises an analysis demand receiving unit, a data modeling unit, a prediction analysis unit, a correction unit and a result backup unit, wherein the analysis demand receiving unit is used for receiving analysis demands of users, the data modeling unit is used for building a model according to the analysis demands received by the analysis demand receiving unit, the prediction analysis unit can estimate data results in a period of time in the future according to the model built by the data modeling unit, the correction unit adopts a Storm topological structure architecture to correct the deviation of big data analysis in real time, and the result backup unit can backup an analysis structure and can be called quickly after receiving the same analysis request again.
Further, the cloud platform 5 is a Hadoop-based distributed computing server cluster, and includes a plurality of distributed computing servers.
Further, the wireless communication module 8 is one of a GPRS module, an SMS module, a wifi module, a ZigBee communication module, and a 5G mobile communication module.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, the error information in the acquired rail transit data can be effectively eliminated through the set preprocessing module, the accuracy of data analysis is improved, the information required by a user can be analyzed and modeled through the set data analysis module, the data in a period of time in the future can be subjected to predictive analysis, visual data analysis can be provided for the user, the deviation of big data analysis can be corrected through the set correction unit, the set result backup unit can backup the analysis structure, the analysis structure can be called quickly after the same analysis request is received again, and the waiting time of the user is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a block diagram of the system architecture of the present invention;
FIG. 3 is a schematic diagram of a data analysis module according to the present invention.
In the drawings, the components represented by the respective reference numerals are listed below:
1. a data acquisition module; 2. an original database; 3. a preprocessing module; 4. a data transmission module; 5. a cloud platform; 6. a cloud database; 7. a data analysis module; 71. an analysis demand receiving unit; 72. a data modeling unit; 73. a prediction analysis unit; 74. a correction unit; 75. a result backup unit; 8. a wireless communication module; 9. and (4) a client.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-3, the present embodiment provides a technical solution: a rail transit big data analysis method comprises the following steps:
step S1: data acquisition, namely acquiring and summarizing original data of the rail transit;
step S2: storing original data, namely storing the acquired rail transit original data;
step S3: preprocessing data, namely preprocessing the original data of the rail transit, correcting abnormal data and removing the data which cannot be corrected;
step S4: uploading data, namely uploading the preprocessed rail transit data to a cloud platform, and storing and calling the data by the cloud platform;
step S5: and (3) inquiring by the user, sending the required analysis requirement to the cloud platform through the client, and sending the analysis result to the user client after the cloud platform analyzes the data.
In step S1, the data collection mode is a distributed data collection mode. The data preprocessing in step S3 includes data cleaning, data integration, data transformation, and data reduction.
The utility model provides a big data analysis system of track traffic, including data acquisition module 1, data acquisition module 1 signal connection has original database 2, original database 2 signal connection has preprocessing module 3, 3 signal connection of preprocessing module has data transmission module 4, 4 signal connection cloud platforms 5 of data transmission module, 5 signal connection of cloud platforms has high in the clouds database 6, 5 two-way signal connection data analysis module 7 and wireless communication module 8 of cloud platforms, 7 two-way signal connection cloud databases 6 of data analysis module, 8 two-line signal connection customer ends 9 of wireless communication module. The original database 2 adopts a NoSQL database, and the cloud database 6 adopts a MySQL database. The data analysis module 7 includes an analysis requirement receiving unit 71, a data modeling unit 72, a prediction analysis unit 73, a correction unit 74 and a result backup unit 75, the analysis requirement receiving unit 71 is configured to receive an analysis requirement of a user, the data modeling unit 72 is configured to construct a model according to the analysis requirement received by the analysis requirement receiving unit 71, the prediction analysis unit 73 can estimate a data result in a period of time in the future according to the model constructed by the data modeling unit 72, the correction unit 74 adopts a Storm topological structure architecture to correct a deviation of a big data analysis in real time, and the result backup unit 75 can backup an analysis structure and can call the analysis structure quickly after receiving the same analysis request again. The cloud platform 5 is a Hadoop-based distributed computing server cluster and comprises a plurality of distributed computing servers. The wireless communication module 8 is one of a GPRS module, an SMS module, a wifi module, a ZigBee communication module and a 5G mobile communication module.
When the data analysis system is used, the data acquisition module 1 acquires rail transit original data of each point location, the original data are stored in the original database 2, the preprocessing module 3 calls the data in the original database 2 to preprocess the data, abnormal data are corrected, data which cannot be corrected are removed, the processed data are transmitted to the cloud platform 5 through the data transmission module 4, the cloud platform 5 stores the data into the cloud database 6, when a user needs to analyze the data, the analysis requirement is sent through the client 9, the wireless communication module 8 sends the analysis requirement to the cloud platform 5, the analysis requirement receiving unit 71 in the data analysis module 7 receives the analysis requirement, corresponding data information in the cloud database 6 is called according to the analysis requirement, the data modeling unit 72 models the called information, the prediction analysis unit 73 can predict the number in a period of time in the future according to a model constructed by the data modeling unit 72 As a result, the correction unit 74 corrects the bias of the analysis of the big data in real time, and as a result the backup unit 75 can backup the analysis structure and can quickly call it up when the same analysis request is received again.
In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (8)
1. A rail transit big data analysis method is characterized by comprising the following steps:
step S1: data acquisition, namely acquiring and summarizing original data of the rail transit;
step S2: storing original data, namely storing the acquired rail transit original data;
step S3: preprocessing data, namely preprocessing the original data of the rail transit, correcting abnormal data and removing the data which cannot be corrected;
step S4: uploading data, namely uploading the preprocessed rail transit data to a cloud platform, and storing and calling the data by the cloud platform;
step S5: and (3) inquiring by the user, sending the required analysis requirement to the cloud platform through the client, and sending the analysis result to the user client after the cloud platform analyzes the data.
2. The rail transit big data analysis method according to claim 1, characterized in that: the data acquisition mode in step S1 is a distributed data acquisition mode.
3. The rail transit big data analysis method according to claim 1, characterized in that: the data preprocessing in the step S3 includes data cleaning, data integration, data transformation and data reduction.
4. A rail transit big data analysis system which characterized in that: including data acquisition module (1), data acquisition module (1) signal connection has original database (2), original database (2) signal connection has preprocessing module (3), preprocessing module (3) signal connection has data transmission module (4), data transmission module (4) signal connection cloud platform (5), cloud platform (5) signal connection has high in the clouds database (6), cloud platform (5) two-way signal connection data analysis module (7) and wireless communication module (8), data analysis module (7) two-way signal connection high in the clouds database (6), wireless communication module (8) two-wire signal connection client (9).
5. The rail transit big data analysis system according to claim 4, wherein: the original database (2) adopts a NoSQL database, and the cloud database (6) adopts a MySQL database.
6. The rail transit big data analysis system according to claim 4, wherein: the data analysis module (7) comprises an analysis demand receiving unit (71), a data modeling unit (72), a prediction analysis unit (73), a correction unit (74) and a result backup unit (75), wherein the analysis demand receiving unit (71) is used for receiving analysis demands of users, the data modeling unit (72) is used for constructing a model according to the analysis demands received by the analysis demand receiving unit (71), the prediction analysis unit (73) can estimate data results in a future period of time according to the model constructed by the data modeling unit (72), the correction unit (74) adopts a Storm topological structure architecture to correct the deviation of large data analysis in real time, and the result backup unit (75) can backup an analysis structure and can be called quickly after receiving the same analysis request again.
7. The rail transit big data analysis system according to claim 4, wherein: the cloud platform (5) is a Hadoop-based distributed computing server cluster and comprises a plurality of distributed computing servers.
8. The rail transit big data analysis system according to claim 4, wherein: the wireless communication module (8) is one of a GPRS module, an SMS module, a wifi module, a ZigBee communication module and a 5G mobile communication module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621999.7A CN111831658A (en) | 2020-06-30 | 2020-06-30 | Rail transit big data analysis method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621999.7A CN111831658A (en) | 2020-06-30 | 2020-06-30 | Rail transit big data analysis method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111831658A true CN111831658A (en) | 2020-10-27 |
Family
ID=72899932
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010621999.7A Pending CN111831658A (en) | 2020-06-30 | 2020-06-30 | Rail transit big data analysis method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111831658A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449876A (en) * | 2021-06-11 | 2021-09-28 | 北京四维图新科技股份有限公司 | Processing method, system and storage medium for deep learning training data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106092959A (en) * | 2016-06-30 | 2016-11-09 | 上海仪器仪表研究所 | A kind of near-infrared food quality based on cloud platform monitoring system |
| CN107656974A (en) * | 2017-09-05 | 2018-02-02 | 北京天平检验行有限公司 | A kind of big data analysis system |
| CN108062395A (en) * | 2017-12-19 | 2018-05-22 | 深圳交控科技有限公司 | A kind of track traffic big data analysis method and system |
| CN111062651A (en) * | 2020-03-18 | 2020-04-24 | 南京中电科能技术有限公司 | Safe power utilization management system and method based on edge calculation and big data analysis |
| CN111339146A (en) * | 2020-03-06 | 2020-06-26 | 东莞理工学院 | Method and device for inquiring rail transit monitoring data |
-
2020
- 2020-06-30 CN CN202010621999.7A patent/CN111831658A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106092959A (en) * | 2016-06-30 | 2016-11-09 | 上海仪器仪表研究所 | A kind of near-infrared food quality based on cloud platform monitoring system |
| CN107656974A (en) * | 2017-09-05 | 2018-02-02 | 北京天平检验行有限公司 | A kind of big data analysis system |
| CN108062395A (en) * | 2017-12-19 | 2018-05-22 | 深圳交控科技有限公司 | A kind of track traffic big data analysis method and system |
| CN111339146A (en) * | 2020-03-06 | 2020-06-26 | 东莞理工学院 | Method and device for inquiring rail transit monitoring data |
| CN111062651A (en) * | 2020-03-18 | 2020-04-24 | 南京中电科能技术有限公司 | Safe power utilization management system and method based on edge calculation and big data analysis |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449876A (en) * | 2021-06-11 | 2021-09-28 | 北京四维图新科技股份有限公司 | Processing method, system and storage medium for deep learning training data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4030365A1 (en) | Multi-mode multi-service rail transit analog simulation method and system | |
| CN105427594B (en) | A kind of public transport section volume of the flow of passengers acquisition methods and system based on two-way passenger flow of getting on the bus | |
| CN102982678B (en) | Traffic data information service system and method for realizing traffic data information service | |
| GB2572717A (en) | Method for detecting traffic anomaly of urban road | |
| CN112874589B (en) | Automatic train planned operation diagram adjusting system and method | |
| CN100542330C (en) | Mobile object's position update method based on transportation network and GPS | |
| CN110926439B (en) | Operation tunnel structure health monitoring system based on BIM technology | |
| CN104200667A (en) | Traffic jam grade detection method based on cellphone signal data | |
| CN115063978B (en) | Bus arrival time prediction method based on digital twins | |
| CN106447137A (en) | Traffic passenger flow forecasting method based on information fusion and Markov model | |
| CN111190891A (en) | Multi-semantic track data segmented storage method | |
| CN111806516A (en) | Health management device and method for intelligent train monitoring and operation and maintenance | |
| CN111831658A (en) | Rail transit big data analysis method and system | |
| CN108447257B (en) | Web-based traffic data analysis method and system | |
| CN105489010A (en) | System and method for monitoring and analyzing fast road travel time reliability | |
| CN116204576B (en) | Method and system for generating GTFS format data by public transportation data | |
| CN116386305A (en) | Riding method based on information feedback | |
| CN114524004B (en) | Locomotive operation statistical method, device, equipment and storage medium | |
| CN114202922A (en) | Public transport travel service big data acquisition and analysis platform | |
| CN110675080A (en) | Tramcar operation scheme generation method, evaluation method and generation system | |
| CN111563664B (en) | Full load rate calculation method based on train weighing data | |
| CN112884233B (en) | Multimode fusion late prediction method for high-speed railway system | |
| CN112396835B (en) | Subway and bus passenger flow analysis system based on big data | |
| CN213894861U (en) | Escalator full life cycle data management system based on data center station | |
| CN116382173A (en) | Industrial flow machinery monitoring system built based on SaaS platform technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |