CN111581500A - Network public opinion-oriented data distributed directional storage method and device - Google Patents

Network public opinion-oriented data distributed directional storage method and device Download PDF

Info

Publication number
CN111581500A
CN111581500A CN202010332298.1A CN202010332298A CN111581500A CN 111581500 A CN111581500 A CN 111581500A CN 202010332298 A CN202010332298 A CN 202010332298A CN 111581500 A CN111581500 A CN 111581500A
Authority
CN
China
Prior art keywords
public opinion
data
network
network public
internet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010332298.1A
Other languages
Chinese (zh)
Inventor
兰海翔
李卫群
张涛
陶心万
潘忠瑞
李鑫
陆苇
雷厚宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guizhou Lichuang Technology Development Co ltd
Original Assignee
Guizhou Lichuang Technology Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guizhou Lichuang Technology Development Co ltd filed Critical Guizhou Lichuang Technology Development Co ltd
Priority to CN202010332298.1A priority Critical patent/CN111581500A/en
Publication of CN111581500A publication Critical patent/CN111581500A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to a data distribution type directional storage method and a device facing to network public sentiment, wherein the method comprises the following steps of S1, screening out network public sentiment data from network big data; s2, carrying out risk assessment on the online public opinion data, and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels; and S3, directionally storing each network public opinion sub data set to each corresponding distributed database. The invention relates to a network public opinion-oriented data distributed directional storage method and device, which are used for screening out network public opinion data from big data and storing the network public opinion data instead of storing all the big data, so that the storage space can be saved; in addition, the public opinion data are stored in a directional distributed mode according to the risk level, stable storage can be achieved, and data transmission and maintenance are facilitated.

Description

Network public opinion-oriented data distributed directional storage method and device
Technical Field
The invention relates to the field of big data, in particular to a method and a device for distributed directional storage of data for network public sentiment.
Background
The network public opinion is a theory and a viewpoint which have strong influence and tendency on some hot spots and focus problems in real life by the public spreading through the internet, and is mainly realized and strengthened by BBS forum, blogs, news posters, reprints and the like. The existing mode for storing network public opinion data has the following problems: and (4) if the storage data amount is too large, carrying out full storage. The data transmission and maintenance are not facilitated, and the data loss is easily caused by equipment failure, so that serious statistical loss is caused; the storage cost is high, the configuration requirement of equipment is high, and the operation cost can be increased due to the overlarge data amount stored when the big data era comes; how to store public opinion data stably and with low cost is a problem to be solved urgently at present.
Disclosure of Invention
The invention aims to solve the technical problem of providing a data distributed directional storage method and device for network public sentiment, which can realize the stable and low-cost storage of public sentiment data.
The technical scheme for solving the technical problems is as follows: a data distribution type directional storage method facing to network public sentiment comprises the following steps,
s1, screening out network public opinion data from the network big data;
s2, carrying out risk assessment on the online public opinion data, and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels;
and S3, directionally storing each network public opinion sub data set to each corresponding distributed database.
On the basis of the technical scheme, the invention can be further improved as follows.
Further, the S1 is specifically configured to configure a plurality of keywords for the internet public sentiment event, and construct an internet public sentiment event model using the plurality of keywords; training the network public opinion event model to obtain a network public opinion identification model; and inputting the network big data into the network public opinion identification model to obtain the network public opinion data.
Further, the step S2 is that a network public opinion risk assessment model is constructed based on the keywords.
Further, in the S2, the internet public opinion data is evaluated according to the internet public opinion risk evaluation model, and a correlation coefficient between the internet public opinion data and each keyword in the internet public opinion risk evaluation model is calculated; and classifying the network public opinion data according to the correlation coefficient to obtain network public opinion subdata sets with a plurality of risk levels.
Further, the step of laying out the distributed database is also included between the steps of S3.
Based on the data distributed directional storage method facing to the network public sentiment, the invention also provides a data distributed directional storage device facing to the network public sentiment.
A data distribution type directional storage device facing to network public sentiment comprises the following modules,
the network public opinion data screening module is used for screening network public opinion data from the network big data;
the online public opinion data classification module is used for carrying out risk assessment on the online public opinion data and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels;
and the oriented distributed storage module is used for directionally storing each network public opinion sub data set into each corresponding distributed database.
On the basis of the technical scheme, the invention can be further improved as follows.
Further, the network public opinion data screening module is specifically used for configuring a plurality of keywords for a network public opinion event and constructing a network public opinion event model by using the plurality of keywords; training the network public opinion event model to obtain a network public opinion identification model; and inputting the network big data into the network public opinion identification model to obtain the network public opinion data.
Further, the online public opinion risk assessment model building module is used for building an online public opinion risk assessment model based on the keywords.
Further, the online public opinion data classification module is specifically configured to evaluate the online public opinion data according to the online public opinion risk evaluation model, and calculate a correlation coefficient between the online public opinion data and each keyword in the online public opinion risk evaluation model; and classifying the network public opinion data according to the correlation coefficient to obtain network public opinion subdata sets with a plurality of risk levels.
The system further comprises a distributed database construction module, wherein the distributed database construction module is used for laying the distributed database.
The invention has the beneficial effects that: the invention relates to a network public opinion-oriented data distributed directional storage method and device, which are used for screening out network public opinion data from big data and storing the network public opinion data instead of storing all the big data, so that the storage space can be saved; in addition, the public opinion data are stored in a directional distributed mode according to the risk level, stable storage can be achieved, and data transmission and maintenance are facilitated.
Drawings
Fig. 1 is a flowchart of a method for distributed directional storage of data for internet public opinion according to the present invention;
fig. 2 is a block diagram of a data distributed directional storage device for internet public opinion according to the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, a data distributed directional storage method facing network public sentiment comprises the following steps,
s1, screening out network public opinion data from the network big data;
s2, carrying out risk assessment on the online public opinion data, and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels;
and S3, directionally storing each network public opinion sub data set to each corresponding distributed database.
In this embodiment, the step S1 is specifically to configure a plurality of keywords for the internet public opinion event, and construct an internet public opinion event model using the plurality of keywords; training the network public opinion event model to obtain a network public opinion identification model; and inputting the network big data into the network public opinion identification model to obtain the network public opinion data.
In this embodiment, the step S2 further includes the step of constructing a cyber public opinion risk assessment model based on the keywords.
In this embodiment, the step S2 is specifically to evaluate the internet public opinion data according to the internet public opinion risk evaluation model, and calculate a correlation coefficient between the internet public opinion data and each keyword in the internet public opinion risk evaluation model; and classifying the network public opinion data according to the correlation coefficient to obtain network public opinion subdata sets with a plurality of risk levels.
In this embodiment, the step of laying out the distributed database is further included between S3.
Based on the data distributed directional storage method facing to the network public sentiment, the invention also provides a data distributed directional storage device facing to the network public sentiment.
As shown in fig. 2, a data distributed directional storage device for internet public opinion comprises the following modules,
the network public opinion data screening module is used for screening network public opinion data from the network big data;
the online public opinion data classification module is used for carrying out risk assessment on the online public opinion data and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels;
and the oriented distributed storage module is used for directionally storing each network public opinion sub data set into each corresponding distributed database.
In this embodiment, the internet public opinion data screening module is specifically configured to configure a plurality of keywords for an internet public opinion event, and construct an internet public opinion event model using the plurality of keywords; training the network public opinion event model to obtain a network public opinion identification model; and inputting the network big data into the network public opinion identification model to obtain the network public opinion data.
In this embodiment, the system further comprises a cyber public opinion risk assessment model building module, wherein the cyber public opinion risk assessment model building module is used for building a cyber public opinion risk assessment model based on the keywords.
In this embodiment, the internet public opinion data classification module is specifically configured to evaluate the internet public opinion data according to the internet public opinion risk evaluation model, and calculate a correlation coefficient between the internet public opinion data and each keyword in the internet public opinion risk evaluation model; and classifying the network public opinion data according to the correlation coefficient to obtain network public opinion subdata sets with a plurality of risk levels.
In this embodiment, the system further includes a distributed database construction module, where the distributed database construction module is used to lay out the distributed database.
The invention relates to a network public opinion-oriented data distributed directional storage method and device, which are used for screening out network public opinion data from big data and storing the network public opinion data instead of storing all the big data, so that the storage space can be saved; in addition, the public opinion data are stored in a directional distributed mode according to the risk level, stable storage can be achieved, and data transmission and maintenance are facilitated.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (10)

1. A data distribution type directional storage method facing to network public sentiment is characterized in that: comprises the following steps of (a) carrying out,
s1, screening out network public opinion data from the network big data;
s2, carrying out risk assessment on the online public opinion data, and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels;
and S3, directionally storing each network public opinion sub data set to each corresponding distributed database.
2. The internet public opinion-oriented data distributed directional storage method according to claim 1, characterized in that: the S1 specifically includes configuring a plurality of keywords for the internet public sentiment event, and constructing an internet public sentiment event model by using the plurality of keywords; training the network public opinion event model to obtain a network public opinion identification model; and inputting the network big data into the network public opinion identification model to obtain the network public opinion data.
3. The internet public opinion oriented data distributed directional storage method according to claim 2, characterized in that: the step of constructing a network public opinion risk assessment model based on the keywords is further included between the S2.
4. The internet public opinion-oriented data distributed directional storage method according to claim 3, characterized in that: the S2 is specifically configured to evaluate the internet public opinion data according to the internet public opinion risk evaluation model, and calculate a correlation coefficient between the internet public opinion data and each keyword in the internet public opinion risk evaluation model; and classifying the network public opinion data according to the correlation coefficient to obtain network public opinion subdata sets with a plurality of risk levels.
5. The internet public opinion-oriented data distributed directional storage method according to any one of claims 1 to 4, characterized in that: the step of laying out the distributed database is also included between the S3.
6. The utility model provides a directional storage device of data distribution formula towards network public opinion which characterized in that: comprises the following modules which are used for realizing the functions of the system,
the network public opinion data screening module is used for screening network public opinion data from the network big data;
the online public opinion data classification module is used for carrying out risk assessment on the online public opinion data and classifying the online public opinion data according to a risk assessment result to obtain online public opinion subdata sets with a plurality of risk levels;
and the oriented distributed storage module is used for directionally storing each network public opinion sub data set into each corresponding distributed database.
7. The internet public opinion-oriented data distributed directional storage device according to claim 6, wherein: the network public opinion data screening module is specifically used for configuring a plurality of keywords for network public opinion events and constructing a network public opinion event model by using the plurality of keywords; training the network public opinion event model to obtain a network public opinion identification model; and inputting the network big data into the network public opinion identification model to obtain the network public opinion data.
8. The internet public opinion-oriented data distributed directional storage device according to claim 7, wherein: the online public opinion risk assessment model building module is used for building an online public opinion risk assessment model based on the keywords.
9. The internet public opinion-oriented data distributed directional storage device according to claim 8, wherein: the network public opinion data classification module is specifically used for evaluating the network public opinion data according to the network public opinion risk evaluation model and calculating the association coefficient of the network public opinion data and each keyword in the network public opinion risk evaluation model; and classifying the network public opinion data according to the correlation coefficient to obtain network public opinion subdata sets with a plurality of risk levels.
10. The internet public opinion-oriented data distributed directional storage device according to any one of claims 6 to 9, wherein: the system also comprises a distributed database construction module, wherein the distributed database construction module is used for laying the distributed database.
CN202010332298.1A 2020-04-24 2020-04-24 Network public opinion-oriented data distributed directional storage method and device Pending CN111581500A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010332298.1A CN111581500A (en) 2020-04-24 2020-04-24 Network public opinion-oriented data distributed directional storage method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010332298.1A CN111581500A (en) 2020-04-24 2020-04-24 Network public opinion-oriented data distributed directional storage method and device

Publications (1)

Publication Number Publication Date
CN111581500A true CN111581500A (en) 2020-08-25

Family

ID=72124491

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010332298.1A Pending CN111581500A (en) 2020-04-24 2020-04-24 Network public opinion-oriented data distributed directional storage method and device

Country Status (1)

Country Link
CN (1) CN111581500A (en)

Citations (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101414300A (en) * 2008-11-28 2009-04-22 电子科技大学 Method for sorting and processing internet public feelings information
CN102881125A (en) * 2012-09-25 2013-01-16 杭州立高科技有限公司 Alarm monitoring system based on multi-information fusion centralized processing platform
CN102929892A (en) * 2011-08-12 2013-02-13 莫润刚 Accurate information promoting system and method based on social network
CN103841216A (en) * 2014-04-01 2014-06-04 深圳市科盾科技有限公司 Network public opinion monitoring system based on cloud platform
CN107451710A (en) * 2017-04-27 2017-12-08 北京鼎泰智源科技有限公司 A kind of Information Risk grade five-category method and system
CN107741909A (en) * 2017-10-13 2018-02-27 郑州云海信息技术有限公司 A kind of automated testing method and system for distributed block storage
CN108021651A (en) * 2017-11-30 2018-05-11 中科金联(北京)科技有限公司 Network public opinion risk assessment method and device
CN108052586A (en) * 2017-12-11 2018-05-18 上海壹账通金融科技有限公司 The analysis of public opinion method, system, computer equipment and storage medium
CN108446306A (en) * 2018-01-31 2018-08-24 佛山市聚成知识产权服务有限公司 A kind of processing equipment of big data
CN108509492A (en) * 2018-02-12 2018-09-07 郑长敬 Big data processing based on real estate industry and system
CN109271445A (en) * 2018-08-21 2019-01-25 平安科技(深圳)有限公司 Utilize the method, apparatus and electronic equipment of block chain storing data
WO2019028236A1 (en) * 2017-08-02 2019-02-07 Salesforce.Com, Inc. Fencing out nodes in a distributed clustered system
CN109460922A (en) * 2018-11-13 2019-03-12 电子科技大学 A kind of Internet public opinion analysis and aid decision-making system with power industry feature
CN109635072A (en) * 2018-11-12 2019-04-16 平安科技(深圳)有限公司 Public sentiment data distributed storage method, device, storage medium and terminal device
CN109902099A (en) * 2019-01-31 2019-06-18 平安科技(深圳)有限公司 Public sentiment tracking, device and computer equipment based on picture and text big data
CN110263238A (en) * 2019-06-21 2019-09-20 浙江华坤道威数据科技有限公司 A kind of public sentiment listening system based on big data
CN110334259A (en) * 2019-04-22 2019-10-15 新分享科技服务(深圳)有限公司 Webpage data acquiring method, device and computer readable storage medium
CN110502517A (en) * 2019-08-23 2019-11-26 中国南方电网有限责任公司 It is a kind of for storing the distributed memory system of power grid real-time running data
CN110597985A (en) * 2019-08-15 2019-12-20 重庆金融资产交易所有限责任公司 Data classification method, device, terminal and medium based on data analysis
CN110619568A (en) * 2019-09-17 2019-12-27 王文斌 Risk assessment report generation method, device, equipment and storage medium
CN110704542A (en) * 2019-10-15 2020-01-17 南京莱斯网信技术研究院有限公司 Data dynamic partitioning system based on node load
CN111045847A (en) * 2019-12-18 2020-04-21 Oppo广东移动通信有限公司 Event auditing method and device, terminal equipment and storage medium

Patent Citations (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101414300A (en) * 2008-11-28 2009-04-22 电子科技大学 Method for sorting and processing internet public feelings information
CN102929892A (en) * 2011-08-12 2013-02-13 莫润刚 Accurate information promoting system and method based on social network
CN102881125A (en) * 2012-09-25 2013-01-16 杭州立高科技有限公司 Alarm monitoring system based on multi-information fusion centralized processing platform
CN103841216A (en) * 2014-04-01 2014-06-04 深圳市科盾科技有限公司 Network public opinion monitoring system based on cloud platform
CN107451710A (en) * 2017-04-27 2017-12-08 北京鼎泰智源科技有限公司 A kind of Information Risk grade five-category method and system
WO2019028236A1 (en) * 2017-08-02 2019-02-07 Salesforce.Com, Inc. Fencing out nodes in a distributed clustered system
CN107741909A (en) * 2017-10-13 2018-02-27 郑州云海信息技术有限公司 A kind of automated testing method and system for distributed block storage
CN108021651A (en) * 2017-11-30 2018-05-11 中科金联(北京)科技有限公司 Network public opinion risk assessment method and device
CN108052586A (en) * 2017-12-11 2018-05-18 上海壹账通金融科技有限公司 The analysis of public opinion method, system, computer equipment and storage medium
CN108446306A (en) * 2018-01-31 2018-08-24 佛山市聚成知识产权服务有限公司 A kind of processing equipment of big data
CN108509492A (en) * 2018-02-12 2018-09-07 郑长敬 Big data processing based on real estate industry and system
CN109271445A (en) * 2018-08-21 2019-01-25 平安科技(深圳)有限公司 Utilize the method, apparatus and electronic equipment of block chain storing data
CN109635072A (en) * 2018-11-12 2019-04-16 平安科技(深圳)有限公司 Public sentiment data distributed storage method, device, storage medium and terminal device
CN109460922A (en) * 2018-11-13 2019-03-12 电子科技大学 A kind of Internet public opinion analysis and aid decision-making system with power industry feature
CN109902099A (en) * 2019-01-31 2019-06-18 平安科技(深圳)有限公司 Public sentiment tracking, device and computer equipment based on picture and text big data
CN110334259A (en) * 2019-04-22 2019-10-15 新分享科技服务(深圳)有限公司 Webpage data acquiring method, device and computer readable storage medium
CN110263238A (en) * 2019-06-21 2019-09-20 浙江华坤道威数据科技有限公司 A kind of public sentiment listening system based on big data
CN110597985A (en) * 2019-08-15 2019-12-20 重庆金融资产交易所有限责任公司 Data classification method, device, terminal and medium based on data analysis
CN110502517A (en) * 2019-08-23 2019-11-26 中国南方电网有限责任公司 It is a kind of for storing the distributed memory system of power grid real-time running data
CN110619568A (en) * 2019-09-17 2019-12-27 王文斌 Risk assessment report generation method, device, equipment and storage medium
CN110704542A (en) * 2019-10-15 2020-01-17 南京莱斯网信技术研究院有限公司 Data dynamic partitioning system based on node load
CN111045847A (en) * 2019-12-18 2020-04-21 Oppo广东移动通信有限公司 Event auditing method and device, terminal equipment and storage medium

Similar Documents

Publication Publication Date Title
Dunn et al. Fragility curves for assessing the resilience of electricity networks constructed from an extensive fault database
Hernandez-Fajardo et al. Probabilistic study of cascading failures in complex interdependent lifeline systems
KR102418969B1 (en) System and method for predicting communication apparatuses failure based on deep learning
US9652723B2 (en) Electrical transformer failure prediction
CN106293892B (en) Distributed stream computing system, method and apparatus
CN105868369B (en) A kind of data model verifying System and method for based on Merkle tree construction
US20070286071A1 (en) Communication-efficient distributed monitoring of thresholded counts
CN1763778A (en) System and method for problem determination using dependency graphs and run-time behavior models
CN110619019B (en) Distributed storage method and system for data
CN112926941B (en) Management method and device of financial auditing rules, storage medium and server
CN107844914A (en) Risk management and control system and implementation method based on group management
KR20190022434A (en) Method of optimizing a database system, system, electronic device and storage medium
CN108446842B (en) Power management and distribution wind control management method and system
CN109783553A (en) A kind of power distribution network mass data increased quality system
D′ Amico et al. Reliability Measures of Second‐Order Semi‐Markov Chain Applied to Wind Energy Production
Saha et al. Learning network traffic dynamics using temporal point process
CN106326462A (en) Video index grading method and device
Fu et al. Component importance and interdependence analysis for Transmission, Distribution and Communication systems
Khuntia et al. Risk‐based security assessment of transmission line overloading considering spatio‐temporal dependence of load and wind power using vine copula
CN116418653A (en) Fault positioning method and device based on multi-index root cause positioning algorithm
Raza et al. EWMA and DEWMA control charts for Poisson‐Exponential distribution: conditional median approach for censored data
CN109740789B (en) Wiring management method, device, equipment and storage medium
CN111581500A (en) Network public opinion-oriented data distributed directional storage method and device
US11158007B2 (en) Dynamic energy consumption and harvesting with feedback
CN113132405B (en) Defense strategy generation method and system for industrial control system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200825

RJ01 Rejection of invention patent application after publication