CN114266472A - Subway station evacuation risk analysis method based on Spark - Google Patents
Subway station evacuation risk analysis method based on Spark Download PDFInfo
- Publication number
- CN114266472A CN114266472A CN202111570993.2A CN202111570993A CN114266472A CN 114266472 A CN114266472 A CN 114266472A CN 202111570993 A CN202111570993 A CN 202111570993A CN 114266472 A CN114266472 A CN 114266472A
- Authority
- CN
- China
- Prior art keywords
- risk
- evacuation
- data
- subway station
- spark
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a Spark-based subway station evacuation risk analysis method, which comprises the steps of carrying out cause analysis processing on historical related accidents of subway station evacuation failure, determining main risk factors causing accidents, classifying the main risk factors and determining evacuation risk indexes; building a distributed Spark Streaming framework to construct a data warehouse, processing the data of the evacuation risk indexes in the data warehouse, and establishing a risk index library; and performing mining analysis of strong association rules on each evacuation risk index by adopting an association rule algorithm and a parallel association rule mining algorithm, and establishing a risk analysis model of the parallel association rule mining algorithm. According to the risk analysis method, a data warehouse is built to complete data processing work, a parallelization association rule mining algorithm association rule algorithm is adopted to further analyze and mine risk index data, a risk analysis result is output through an obtained risk analysis model, and finally a countermeasure is given according to the risk analysis result.
Description
Technical Field
The invention relates to the technical field of emergency evacuation management of subway stations, in particular to a subway station evacuation risk analysis method based on Spark.
Background
The subway accident can cause great property loss and casualties, the main casualty loss is caused by the harmfulness of the accident, but a part of casualty loss is caused by the fact that the accident severity is enlarged and even secondary accidents are caused because safe evacuation is not carried out timely and effectively after the accident happens. Due to the sealing property of the subway station and the complexity of the structure of the subway station, when an emergency happens, the serious degree of the accident can be further upgraded due to the unfavorable evacuation, and additional casualties are caused. The safety hidden trouble of several times is hidden behind each large lifting accident. If people can find out and eliminate the hidden dangers in the process of emergency evacuation from the hidden dangers of the subway station in advance, the possibility of occurrence of major accidents can be reduced.
Under the current era, the research of various industry safety problems is an irreversible trend towards the development of big data, and the evacuation risk of the subway station is analyzed by using an association rule algorithm based on historical accident information under the big data environment, so that the potential safety hazard in the operation process of the subway station is eliminated in time, and the safety of passengers and workers is guaranteed. Therefore, safety management personnel of the subway station can comprehensively know the current operation condition of the subway station, and once a malignant accident and risk occur, appropriate reaction measures can be timely taken to avoid the possible risk.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a subway station evacuation risk analysis method based on Spark.
The subway station evacuation risk analysis method based on Spark provided by the invention comprises the following steps:
carrying out cause analysis processing on historical related accidents of subway station evacuation failure, determining main risk factors causing accidents, classifying the main risk factors, and determining evacuation risk indexes;
building a distributed streaming processing framework to construct a data warehouse, processing the data of the evacuation risk indexes in the data warehouse, and building a risk index library;
and performing mining analysis of strong association rules on each evacuation risk index in the risk index library by adopting an association rule algorithm and a parallel association rule mining algorithm, and establishing a risk analysis model of the parallel association rule mining algorithm.
Optionally, the cause analysis is used for performing detailed analysis and screening from four aspects of personnel factors, equipment factors, management factors and environmental factors to determine four types of human cause risks, equipment risks, management risks and environmental risks.
Optionally, evacuation risk index data and subway station passenger flow are collected by a Flume cluster and an HDFS cluster and stored in a data warehouse.
Optionally, the processing of the data of the evacuation risk indicator includes extracting, cleaning, converting and counting.
Optionally, performing mining analysis of strong association rules on each evacuation risk indicator by using an association rule algorithm and a parallel association rule mining algorithm further includes:
distributing a transaction set in a string function form to a plurality of machines to generate a frequent 1-item set using a flatMap, distributing the transaction set in a string function form to a plurality of machines using the flatMap;
accumulating the number of candidate item 1-item sets by using redecbykey, and screening out the item sets smaller than the minimum support degree by using a filter;
a set of candidate k +1 terms is generated from the frequent k term set.
Optionally, the data analysis result output by the risk analysis model is called and displayed in real time by fusing the vue.js framework and the Echarts open source library.
Compared with the prior art, the invention has the following beneficial effects:
according to the subway station evacuation risk analysis method based on Spark, provided by the invention, firstly, a data warehouse is built to complete data processing work, then, a parallelized apriori association rule algorithm is adopted to further analyze and mine risk index data, and finally, a risk analysis result is output through an obtained risk analysis model. Safety management personnel can find out emergency evacuation leaks in the subway station through analysis results, specific prevention countermeasures and suggestions are provided, the establishment of an in-station evacuation management method and a plan and the design of emergency evacuation facilities are facilitated, corresponding improvement measures are provided, the safety of the subway station is improved, and a certain reference value is provided for safe operation of the subway station in a big data era.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is a general flowchart of a subway station evacuation risk analysis method based on Spark according to an embodiment of the present invention;
fig. 2 is an architectural design diagram of a subway station evacuation risk analysis method based on Spark according to an embodiment of the present invention;
fig. 3 is a hierarchy diagram of a subway station evacuation risk analysis method based on Spark according to an embodiment of the present invention;
fig. 4 is a flow chart of risk analysis processing based on parallelization Apriori algorithm according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
Before explaining the embodiments of the present invention, the terms involved will be described in conjunction with the embodiments:
apriori: the method is a frequent item set algorithm for mining association rules, and the core idea is to mine a frequent item set through two stages of candidate set generation and downward closed detection of plot. Moreover, algorithms have been widely applied to various fields such as commerce, network security and the like;
spark Streaming framework: spark Streaming is a real-time computing framework based on Spark Core that can consume and process data from many data sources.
Spark Streaming is an extension of Spark core API, and can implement processing of high-throughput real-time Streaming data with a fault-tolerant mechanism, and support acquiring data from various data sources, Kafka, flux, Twitter, ZeroMQ, Kinesis and TCP sockets. On the basis of the One Stack rule all, other subframes of Spark, such as cluster learning, graph calculation, etc., can be used for processing the flow data.
map: the role of this is easily understood to be that the function operations performed on elements in rdd (elastic distributed data set) are mapped one by one to another rdd.
A flatMap: it is a function applied to each element in rdd that constructs all of the contents of the returned iterator into a new rdd, typically used to segment words.
reduceByKey: the method has the function of processing the data of the same key, and finally only one record is reserved for each key.
saveasesequencefile: it is used to save rdd on HDFS in a sequence file format;
flume: the system is a high-availability, high-reliability and distributed system for collecting, aggregating and transmitting mass logs;
kafka: it is a high throughput distributed publish-subscribe messaging system;
js framework: the method is a JavaScript MVVM library, and is a set of progressive framework for constructing a user interface;
echarts open source library: the data visualization chart library based on the JavaScript provides a data visualization chart which is intuitive, vivid, interactive and customizable.
Fig. 1 is a schematic block diagram of a subway station evacuation risk analysis method based on Spark, as shown in fig. 1, the method of the present invention may include the following steps:
cause analysis is carried out on historical related accidents of subway station evacuation failure, main risk factors causing accidents are determined, the main risk factors are classified, and evacuation risk indexes are determined;
in the embodiment, the cause analysis is analyzed and screened from four aspects of personnel factors, equipment factors, management factors and environmental factors;
thereby determining four types of the determined human factor risk, equipment risk, management risk and environmental risk;
wherein, the human risk: the method comprises unsafe behaviors of passenger reversing, frightening, turning back and going backwards, charging congestion, low safety consciousness of subway station workers, poor emergency capacity and the like;
equipment risk: the system comprises a power system fault, an escalator fault, an automatic gate fault, a signal system fault, an alarm system fault, a fire protection system fault, a lighting system fault, a broadcasting system fault, an evacuation identification system fault and the like;
the management risks comprise unreasonable evacuation plan formulation, low inspection frequency of station equipment, untimely maintenance of evacuation facilities, low evacuation drilling effectiveness, imperfect emergency organization, less evacuation safety information propaganda, untimely inspection of risk hidden dangers in the station, inadequate staff safety training, ineffective safety specification, unsmooth information communication, insufficient emergency evacuation facilities and the like;
the environmental risks comprise fire, flood, explosion, terrorist attack, slippery ground, low visibility in the station, higher crowd density than subway station bearing, substandard subway tunnel construction and the like.
Constructing a distributed Spark Streaming framework to construct a data warehouse, processing data of evacuation risk indexes in the data warehouse, and establishing a risk index library;
referring to fig. 2, in this embodiment, a Flume cluster acquires data in a log server, evacuation risk index data and subway station passenger flow are acquired and stored in a data warehouse through the Flume cluster and an HDFS cluster, the data warehouse adopts a Hive data warehouse, the Hive data warehouse includes a data target source layer, a data analysis layer, a data operation layer and a data storage layer, and a Kafka cluster acquires data in the Flume cluster acquired log server and sends the data to a processing system, wherein the processing system can perform real-time calculation and offline calculation to complete cause analysis, and then the evacuation analysis system processes evacuation risk index data in the data warehouse in the manners of extraction, cleaning, conversion and statistics;
referring to fig. 4, the process of converting the data of the evacuation risk indicator in the HDFS cluster includes:
reading data in the evacuation risk index text file, obtaining risk factors, and forming evacuation risk index data by a plurality of risk factors; then, the user can use the device to perform the operation,
stag1, sequentially carrying out flatMap and map operations on evacuation risk index data;
and the Stag2 is used for carrying out redepbykey processing on the data subjected to the flatMap and map operations, and finally saving the file on the HDFS cluster in a sequence File format through saveAssequence File.
And (3) mining and analyzing strong association rules of each evacuation risk index by adopting an association rule algorithm parallel Apriori (association rule mining algorithm), and establishing a risk analysis model of the parallel Apriori.
In this embodiment, the establishing of the risk analysis model of parallel Apriori specifically includes two steps:
the first step is to generate a frequent 1-item set, first distribute the transaction set D to multiple machines in the form of < String, 1> using a fltMap, where the form of < String, 1> is a String function; then accumulating the number of the candidate 1-item sets by using the reduceByKey, and screening the item sets smaller than the minimum support degree by using a filter;
the second step is to generate a frequent k +1 term set from the frequent k term set. A set of candidate k +1 terms is generated from the set of frequent k terms and a set of frequent k +1 terms is generated according to the method of the first step.
In an alternative embodiment, the data analysis result output by the risk analysis model is called and displayed in real time through fusion of the Vue.
In this embodiment, the data analysis result is called and displayed in real time by the method, and the risk analysis result can be visualized, where visualization refers to representing original data in a graphical manner after a series of processing is performed on the original data, so that complicated data information which cannot be directly read is converted into a data result and a conclusion. Relatively complex raw data is more intuitive and easier to understand. The visualization of the evacuation risk analysis result needs to generate a plurality of charts, such as charts with various shapes and styles, such as a line chart, a pie chart and the like, and nested combinations of the charts, and the charts can be interactive, and a tool library for data visualization is required to be selected for realizing the function.
The ECharts is a Baidu open source library based on javascript language for realizing data visualization. And a chart library with complete functions and different forms is formed by basic components such as tool boxes with different characteristics. Typically including pie charts, trend charts, histograms, word clouds. Js is an open source progressive JavaScript framework for building user interfaces, which is a Web application framework for building single-page applications.
Js is focused on the operation of the view, and the frame of the Vue js can be perfectly fused with EChats so as to realize the visualization of the evacuation risk analysis result of the subway station.
Referring to fig. 3, a subway station evacuation risk analysis system based on Spark can be constructed by the above method, and the system includes:
and (3) a service layer: the service layer comprises subway safety management personnel and subway workers;
and (4) a service layer: the method comprises the functions of various risk analysis result chart query, risk index management, historical accident information uploading and query, subway station staff log management and the like;
an application layer: the method comprises the functions of data query, data analysis, data visualization and the like;
and (3) a data layer: the method comprises the functions of data acquisition, data storage, data processing, data integration and the like, wherein the data acquired by the data acquisition comprises structured data and unstructured data, the data type of the data storage comprises SQL (structured query language) and Non-SQL (structured query language), the data processing comprises data standard management and data quality management, and the data integration comprises format conversion and data splicing;
technical layer: the system comprises an acquisition framework, a real-time computing, an off-line computing, a system integration and other functions, wherein the acquisition framework comprises a cloud computing, an open source tool, an acquisition system and a stream processing platform, the real-time computing comprises a data warehouse, a data technology, a parallel computing and a distributed database, the off-line computing comprises a full-text search engine, an HDFS, a computing engine and a distributed database, and the system integration comprises application system integration, business process integration and enterprise application integration;
equipment layer: the intelligent terminal comprises a server, a memory, input equipment, an intelligent terminal, an arithmetic unit, a controller, output equipment and information safety equipment.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (6)
1. A subway station evacuation risk analysis method based on Spark is characterized by comprising the following steps:
carrying out cause analysis processing on historical related accidents of subway station evacuation failure, determining main risk factors causing accidents, classifying the main risk factors, and determining evacuation risk indexes;
building a distributed streaming processing framework to construct a data warehouse, processing the data of the evacuation risk indexes in the data warehouse, and building a risk index library;
and performing mining analysis of strong association rules on each evacuation risk index in the risk index library by adopting an association rule algorithm and a parallel association rule mining algorithm, and establishing a risk analysis model of the parallel association rule mining algorithm.
2. A Spark-based subway station evacuation risk analysis method as claimed in claim 1, wherein said cause analysis is detailed analysis and screening from four aspects of personnel factor, equipment factor, management factor and environment factor to determine four types of personnel risk, equipment risk, management risk and environment risk.
3. A Spark-based subway station evacuation risk analysis method according to claim 1, wherein evacuation risk index data and subway station passenger flow are collected through a Flume cluster and an HDFS cluster and stored in a data warehouse.
4. A Spark-based subway station evacuation risk analysis method as claimed in claim 1, wherein said data of evacuation risk indicator is processed including extraction, cleaning, conversion and statistics.
5. A Spark-based subway station evacuation risk analysis method as claimed in claim 1, wherein said mining analysis of strong association rule for each said evacuation risk index using association rule algorithm parallel association rule mining algorithm further comprises:
distributing a transaction set in a string function form to a plurality of machines to generate a frequent 1-item set using a flatMap, distributing the transaction set in a string function form to a plurality of machines using the flatMap;
accumulating the number of candidate item 1-item sets by using redecbykey, and screening out the item sets smaller than the minimum support degree by using a filter;
a set of candidate k +1 terms is generated from the frequent k term set.
6. A Spark-based subway station evacuation risk analysis method as claimed in claim 3, wherein said risk analysis model output data analysis result is called and displayed in real time by fusing Vue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111570993.2A CN114266472A (en) | 2021-12-21 | 2021-12-21 | Subway station evacuation risk analysis method based on Spark |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111570993.2A CN114266472A (en) | 2021-12-21 | 2021-12-21 | Subway station evacuation risk analysis method based on Spark |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114266472A true CN114266472A (en) | 2022-04-01 |
Family
ID=80828186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111570993.2A Pending CN114266472A (en) | 2021-12-21 | 2021-12-21 | Subway station evacuation risk analysis method based on Spark |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114266472A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117010697A (en) * | 2023-09-25 | 2023-11-07 | 山东财经大学 | Visual enterprise risk assessment method based on artificial intelligence |
-
2021
- 2021-12-21 CN CN202111570993.2A patent/CN114266472A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117010697A (en) * | 2023-09-25 | 2023-11-07 | 山东财经大学 | Visual enterprise risk assessment method based on artificial intelligence |
| CN117010697B (en) * | 2023-09-25 | 2023-12-19 | 山东财经大学 | Visual enterprise risk assessment method based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bosch et al. | Scatterblogs2: Real-time monitoring of microblog messages through user-guided filtering | |
| Hidayatullah et al. | Road traffic topic modeling on Twitter using latent dirichlet allocation | |
| Lagerstrom et al. | Image classification to support emergency situation awareness | |
| CN105553957A (en) | Network safety situation awareness early-warning method and system based big data | |
| Alomari et al. | Road traffic event detection using twitter data, machine learning, and apache spark | |
| CN113254594B (en) | Smart power plant-oriented safety knowledge graph construction method and system | |
| US20190197074A1 (en) | Big data analysis platform for travel and tourism | |
| CN109993189A (en) | A kind of network failure method for early warning, device and medium | |
| US20160225171A1 (en) | Identifying cyclic patterns of complex events | |
| CN111966746B (en) | Meteorological disaster prevention and reduction process monitoring system and monitoring method thereof | |
| Zhang et al. | Data-driven AI emergency planning in process industry | |
| CN114266472A (en) | Subway station evacuation risk analysis method based on Spark | |
| CN105930255A (en) | Method and apparatus for predicting health degree of system | |
| Wang et al. | Safety intelligence toward safety management in a big-data environment: A general model and its application in urban safety management | |
| e Silva et al. | A data analytics framework for anomaly detection in flight operations | |
| Chair et al. | Towards a social media-based framework for disaster communication | |
| Al Nuaimi et al. | Social media analytics for sentiment analysis and event detection in smart cities | |
| CN116624226A (en) | Coal mine disaster data acquisition, analysis and visual display system | |
| Abesamis et al. | Improving aviation incidents using association rule mining algorithm and time series analysis | |
| CN115408236A (en) | Log data auditing system, method, equipment and medium | |
| Li et al. | An evaluation of factors influencing the community emergency management under compounding risks perspective | |
| Gottipati et al. | Exploring the impact of COVID-19 on aviation industry: A text mining approach | |
| Hassel et al. | Method to study cascading effects | |
| Benjamin et al. | Distributed information gathering, exploration and sensemaking toolkit (DIGEST) | |
| CN110096532A (en) | A kind of safety in production big data analysis method for digging and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |