CN116881395A - Public opinion information detection method and device - Google Patents

Public opinion information detection method and device Download PDF

Info

Publication number
CN116881395A
CN116881395A CN202310736949.7A CN202310736949A CN116881395A CN 116881395 A CN116881395 A CN 116881395A CN 202310736949 A CN202310736949 A CN 202310736949A CN 116881395 A CN116881395 A CN 116881395A
Authority
CN
China
Prior art keywords
public opinion
data
information
event
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310736949.7A
Other languages
Chinese (zh)
Inventor
郭欣怡
杨彬
徐海泉
张佳琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202310736949.7A priority Critical patent/CN116881395A/en
Publication of CN116881395A publication Critical patent/CN116881395A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33Querying
    • G06F16/3331Query processing
    • G06F16/334Query execution
    • G06F16/3344Query execution using natural language analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36Creation of semantic tools, e.g. ontology or thesauri
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/279Recognition of textual entities
    • G06F40/289Phrasal analysis, e.g. finite state techniques or chunking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/30Semantic analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • G06N3/0442Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention provides a public opinion information detection method and device, which can be used in the technical field of artificial intelligence, and the method comprises the following steps: obtaining public opinion data to be detected; carrying out key information extraction on public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information; the key public opinion information is matched with a public opinion event map set constructed in advance to obtain corresponding public opinion event information, and the key public opinion information of the current public opinion event is matched with the public opinion event map set constructed in advance to match a plurality of associated public opinion events, so that the association relation among the plurality of public opinion events is excavated, the public opinion events are responded rapidly and accurately, the completeness of data and the accuracy of an emotion analysis method are not required to be completely relied on, the time consumption and the manpower resource cost are reduced, and the detection accuracy and efficiency are improved.

Description

Public opinion information detection method and device
Technical Field
The invention relates to the technical field of computers, in particular to the technical field of artificial intelligence, and particularly relates to a public opinion information detection method and device.
Background
With increasing demands of people and increasing awareness of rights, people are in contact with commercial financial institutions more and more, and requirements of the commercial financial institutions are more and more strict, and once commercial financial institutions with huge user groups and wide coverage range fail to effectively prevent and cope with public opinion risks, social images and reputation of the financial institutions are seriously damaged, and future development is hindered. Public opinion events have strong burstiness and unpredictability, in the related technology, analysis results of the public opinion events are obtained through data analysis and information processing by manually knowing and investigating public opinion information, but the method cannot quickly respond to the public opinion events, and only positive and negative directions of single public opinion events can be identified depending on completeness of crawling data and accuracy of emotion analysis methods, so that time consumption is long, detection accuracy and efficiency are low, and human resource cost is high.
Disclosure of Invention
An object of the present invention is to provide a public opinion information detection method, which can match a plurality of public opinion events associated with each other by matching key public opinion information of a current public opinion event with a public opinion event map set constructed in advance, and mine association relations among the plurality of public opinion events, so as to rapidly and accurately respond to the public opinion events without completely depending on completeness of data and accuracy of an emotion analysis method, reduce time consumption and human resource cost, and improve detection accuracy and efficiency. Another object of the present invention is to provide a public opinion information detection apparatus. It is yet another object of the present invention to provide a computer readable medium. It is a further object of the invention to provide a computer device.
In order to achieve the above objective, an aspect of the present invention discloses a public opinion information detection method, including:
obtaining public opinion data to be detected;
carrying out key information extraction on public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information;
and matching the key public opinion information with a public opinion event atlas constructed in advance to obtain corresponding public opinion event information.
Preferably, before extracting key information from the public opinion data to be detected through the pre-constructed information extraction model and emotion analysis model to obtain key public opinion information, the method further comprises:
acquiring historical public opinion data;
carrying out structural preprocessing on the historical public opinion data, and structuring a public opinion data set;
training a bidirectional long-short-term memory network in combination with a double-layer attention mechanism algorithm according to the structured public opinion data set, and constructing an information extraction model.
Preferably, the historical public opinion data comprises free text data, work order data and feedback data;
obtaining historical public opinion data, including:
acquiring free text data by a web crawler technology;
and acquiring work order data and feedback data through a system database.
Preferably, the structural preprocessing is performed on the historical public opinion data, and the structural public opinion data set includes:
performing word segmentation processing on the historical public opinion data to obtain word segmentation data;
and carrying out attribute labeling and emotion labeling on the word segmentation data to obtain a structured public opinion data set.
Preferably, the method further comprises:
encoding the structured public opinion data set through a sentence embedding model to obtain encoded data;
performing context information learning on the coded data through a bidirectional long-short-time memory network to obtain context data;
training an attention mechanism algorithm according to the context data, and constructing an emotion analysis model.
Preferably, the key public opinion information comprises target public opinion information and target emotion type;
extracting key information from public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information, wherein the method comprises the following steps:
carrying out information extraction on the public opinion data to be detected through an information extraction model to obtain target public opinion information;
and carrying out emotion analysis on the public opinion data to be detected through an emotion analysis model to obtain a target emotion type.
Preferably, the structured public opinion data set comprises a plurality of structured public opinion data;
Before the key public opinion information is matched with the public opinion event atlas constructed in advance to obtain the corresponding public opinion event information, the method further comprises the following steps:
constructing a corresponding event sub-map according to each piece of structured public opinion data;
classifying event sub-atlas according to preset classification conditions, and constructing a public opinion event atlas set which comprises a plurality of atlas types and event sub-atlas corresponding to each atlas type.
Preferably, the structured public opinion data set comprises event trigger words, emotion types, character relationships, event relationships, public opinion entities and entity attributes;
constructing a corresponding event sub-map according to each piece of structured public opinion data, including:
and determining public opinion entities as nodes, character relationships and event relationships as edges, entity attributes as node attributes, emotion types and event trigger words as sub-spectrum attributes, and constructing an event sub-spectrum.
Preferably, matching the key public opinion information with a public opinion event atlas constructed in advance to obtain corresponding public opinion event information includes:
extracting corresponding public opinion types from the key public opinion information according to the classification conditions;
matching the public opinion type with the map type in the public opinion event map set to obtain a corresponding target event sub-map;
And obtaining corresponding public opinion event information according to the target event sub-map.
The invention also discloses a public opinion information detection device, which comprises:
the first acquisition unit is used for acquiring public opinion data to be detected;
the information extraction unit is used for extracting key information from the public opinion data to be detected through the pre-constructed information extraction model and emotion analysis model to obtain key public opinion information;
and the information matching unit is used for matching the key public opinion information with a public opinion event atlas constructed in advance to obtain corresponding public opinion event information.
The invention also discloses a computer readable medium having stored thereon a computer program which when executed by a processor implements a method as described above.
The invention also discloses a computer device comprising a memory for storing information comprising program instructions and a processor for controlling the execution of the program instructions, the processor implementing the method as described above when executing the program.
The invention also discloses a computer program product comprising a computer program/instruction which, when executed by a processor, implements a method as described above.
The method comprises the steps of obtaining public opinion data to be detected; carrying out key information extraction on public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information; the key public opinion information is matched with a public opinion event map set constructed in advance to obtain corresponding public opinion event information, and the key public opinion information of the current public opinion event is matched with the public opinion event map set constructed in advance to match a plurality of associated public opinion events, so that the association relation among the plurality of public opinion events is excavated, the public opinion events are responded rapidly and accurately, the completeness of data and the accuracy of an emotion analysis method are not required to be completely relied on, the time consumption and the manpower resource cost are reduced, and the detection accuracy and efficiency are improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a public opinion information detection method according to an embodiment of the present application;
FIG. 2 is a flowchart of another method for detecting public opinion information according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a public opinion information detection device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be noted that the public opinion information detection method and device disclosed by the application can be used in the technical field of artificial intelligence, and can also be used in any field except the technical field of artificial intelligence, and the application field of the public opinion information detection method and device disclosed by the application is not limited.
In order to facilitate understanding of the technical scheme provided by the application, the following description will explain relevant contents of the technical scheme of the application. Public opinion events have strong burst and unpredictability, and for commercial financial institutions to effectively prevent public opinion risks, public opinion risk management must be incorporated into daily management work, and a perfect public opinion risk management system must be established to ensure standardization and effectiveness of public opinion risk management. Currently, public opinion event analysis mainly involves two problems: (1) how to obtain effective public opinion event information; (2) And analyzing, processing, early warning and the like on the obtained public opinion time information. In addition, china is a multi-ethnic and multi-language country, and needs to analyze the public opinion events comprehensively and analyze the public opinion events of the minority ethnic language so as to improve the comprehensiveness and universality of the public opinion event analysis.
The invention acquires social public opinion event data of a large number of financial industries on a network platform in advance, and acquires work order data and feedback data specific to financial institutions as an analyzed data source; obtaining keywords and entities (event elements) of the event by means of natural language processing technologies such as word segmentation, entity recognition, information extraction and the like, distinguishing positive and negative public opinion events by means of emotion recognition, and storing the keywords and the event elements in a relational database in a structured information form; generating an event sub-map according to each public opinion event or work order data or feedback data record, wherein the event sub-map is centered on a specific event main body, and the event elements are connected with the event main body in an attribute relationship; event sub-maps of different map types are formed through one type of event, so that the user situation of the enterprise can be closely concerned, even the large environment of the enterprise public opinion can be effectively prevented from spreading, and enterprise public opinion monitoring can be well performed.
When sudden public opinion events exist, on one hand, event key information and emotion positive and negative of public opinion comments are extracted from sudden public opinion event information, and the sudden public opinion event information is matched with a pre-constructed public opinion event map set, so that the incidence relation and the similarity between the historical sudden public opinion events and the current public opinion events are rapidly mined; on the other hand, the method provides time for public opinion processing personnel to know public opinion information so as to accurately position the public opinion direction, improve the efficiency of analyzing sudden public opinion events and quickly find out a corresponding public opinion event processing method.
The deep learning-based information extraction model and emotion analysis model are used for public opinion event analysis of the financial industry, help is provided for quickly knowing public opinion events and quickly responding, a public opinion event map set of the financial industry is constructed, and the public opinion information detection mode is suitable for multilingual public opinion data analysis and has high comprehensiveness and universality.
The implementation process of the public opinion information detection method provided by the embodiment of the invention is described below by taking the public opinion information detection device as an execution subject. It can be understood that the execution body of the public opinion information detection method provided by the embodiment of the invention includes, but is not limited to, a public opinion information detection device.
Fig. 1 is a flowchart of a public opinion information detection method according to an embodiment of the present invention, as shown in fig. 1, where the method includes:
and step 101, obtaining public opinion data to be detected.
And 102, extracting key information from the public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information.
In the embodiment of the invention, the key public opinion information comprises target public opinion information and target emotion type.
And step 103, matching the key public opinion information with a public opinion event atlas constructed in advance to obtain corresponding public opinion event information.
In the technical scheme provided by the embodiment of the invention, the public opinion data to be detected is obtained; carrying out key information extraction on public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information; the key public opinion information is matched with a public opinion event map set constructed in advance to obtain corresponding public opinion event information, and the key public opinion information of the current public opinion event is matched with the public opinion event map set constructed in advance to match a plurality of associated public opinion events, so that the association relation among the plurality of public opinion events is excavated, the public opinion events are responded rapidly and accurately, the completeness of data and the accuracy of an emotion analysis method are not required to be completely relied on, the time consumption and the manpower resource cost are reduced, and the detection accuracy and efficiency are improved.
Fig. 2 is a flowchart of another public opinion information detection method according to an embodiment of the present invention, as shown in fig. 2, the method includes:
step 201, obtaining historical public opinion data.
In the embodiment of the invention, each step is executed by the public opinion information detection device.
In the embodiment of the invention, the historical public opinion data comprises free text data, work order data and feedback data. Specifically, free text data is obtained through a web crawler technology; and acquiring work order data and feedback data through a system database.
The free text data is data related to internet public opinion events of multilingual homonymy financial institutions and home financial institutions, and the free text data comprises but is not limited to news stories, social media events and official notices published on the internet platform.
The system database is a local database of the gold thawing mechanism, and is internally stored with full-quantity work order data and feedback data, wherein the work order data are of a work order type, and the work order type comprises but is not limited to complaints, consultations, suggestions, counseling and recourse; the feedback data is data of questionnaires and service evaluations issued by the financial institutions at each channel.
In the embodiment of the invention, data acquisition is the basis of network public opinion event analysis, and provides a data source for public opinion analysis. According to the invention, the historical public opinion data is obtained in an online-offline combined mode, so that the required basic data is provided for the subsequent construction model.
And 202, carrying out structural preprocessing on the historical public opinion data, and structuring a public opinion data set.
In the embodiment of the invention, the historical public opinion data is multilingual data, and the structured preprocessing comprises cleaning, labeling and structured conversion of the historical public opinion data of multilingual homonymy financial institutions and principal financial institutions.
In the embodiment of the present invention, step 202 specifically includes:
and 2021, performing word segmentation on the historical public opinion data to obtain word segmentation data.
In the embodiment of the invention, word segmentation processing on the Chinese historical public opinion data specifically comprises the following steps: segmenting the historical public opinion data of the Chinese by using an open source tool according to sentence granularity to obtain word segmentation data; the word segmentation processing for the historical public opinion data of the minority nationality language specifically comprises the following steps: and segmenting the historical public opinion data of the minority language by a word granularity through a word segmentation tool to obtain word segmentation data.
Taking minority nationality language as a vitamin language as an example, through a Moses word segmentation tool, word segmentation processing is carried out on the historical public opinion data of the vitamin language by using spaces to obtain word segmentation data.
Through word segmentation on the multi-language data, the information extraction model and the emotion analysis model which are constructed later are not only suitable for Chinese, but also suitable for multi-language data such as Uygur language, and the like, support cross-language and have language universality.
And 2022, carrying out attribute labeling and emotion labeling on the segmented data to obtain a structured public opinion data set.
In the embodiment of the invention, the attribute labels comprise but are not limited to event trigger words, character relations, event relations, public opinion entities and entity attributes. Public opinion entities include, but are not limited to, event names, events, places, people, bank cards, and accounts, and entity attributes include, but are not limited to, people attributes and event attributes.
The event trigger words are keywords capable of triggering public opinion events, and the event trigger words can be set according to event topics for free text data and feedback data; for work order data, the work order type may be set as a trigger word for the work order data.
Persona relationships include, but are not limited to, couples, parents, siblings, superlow, teachers and students, friends, classmates, collaboration, co-persons, lovers, grandchild, colleagues, relatives, unknowns.
Event relationships including, but not limited to, causal, compliance, conditional, concurrency, and turning, event-related word templates may be pre-constructed from event relationships, such that event relationships may be extracted from the constructed templates. As an alternative, the event keyword template includes a plurality of event relationships and keywords corresponding to each event relationship.
The keywords corresponding to the causal relationship include, but are not limited to, "because", "because, thereby", "because, so", "then", "so", "cause", "therefore", "thereby", "thus", "cause", "by virtue", "due", "only", "root", "dependent".
Keywords corresponding to a cis-bearing relationship include, but are not limited to, "next," "then," "next," "further," "on the one hand," "on the other hand"; conditions.
Keywords corresponding to concurrency relationships include, but are not limited to, "not only, but also," "and," "simultaneously," also, "" still, "" all, "" always, "" not just.
Keywords corresponding to the disjunctive relationship include, but are not limited to, "but," not as much as, "" rather, "" neither, nor, "" even if not.
In the embodiment of the invention, the emotion marking comprises emotion types, the emotion marking is emotion classification marking, and the emotion types comprise positive and negative directions. For free text data and feedback data, emotion types can be set according to the specific content of an event; for the worksheet data, emotion marks of worksheet data can be set according to worksheet types, as an alternative scheme, emotion marks of worksheet data with worksheet types of consultation, suggestion and listing are determined to be positive, and emotion marks of worksheet data with worksheet types of complaints and recourse are determined to be negative.
In the embodiment of the invention, the word segmentation data marked with the attribute marks and the emotion marks is determined to be structured public opinion data; and merging the plurality of pieces of structured public opinion data into a structured public opinion data set, wherein the structured public opinion data set comprises the plurality of pieces of structured public opinion data.
Further, the structured public opinion data set is stored in a graphic database (Neo 4 j), and according to emotion analysis results, event attributes are used for labeling emotion classification. Neo4j is an open source NoSQL graph database implemented by Java. It stores structured data on the network, with all the features of a mature database, and can be seen as a high-performance graph engine. Neo4j implements the storage of a professional database-level graph data model.
And 203, training a bidirectional long-short-term memory network by combining a double-layer Attention mechanism (BiLSTM+2 Attention) algorithm according to the structured public opinion data set, and constructing an information extraction model.
In the embodiment of the invention, a Bi-directional Long Short-Term Memory network (BiLSTM) is formed by combining a forward direction LSTM (Forward LSTM) and a reverse direction LSTM (Backward LSTM), the BiLSTM is often used for context information modeling, and the BiLSTM is jointly determined by a plurality of inputs in the front and a plurality of inputs in the back, so that Bi-directional semantic dependence can be better captured, and the result is more accurate.
In the embodiment of the invention, a double-layer Attention mechanism (2 Attention) is adopted, the word-level Attention mechanism highlights keywords in sentences, the keywords are given higher weight, and the accuracy of the representation learning of example sentences is improved by capturing key information. The sentence-level attention mechanism assigns different weights to each instance sentence in the same instance set to highlight its importance to the relationship classification result.
Specifically, the structured public opinion data set is divided into a training set and a testing set according to a specified proportion, the BiLSTM+2Attention algorithm is trained according to the training set, and entity identification and information extraction tasks are carried out on the training set, so that an initial information extraction model is obtained; and optimizing the initial information extraction model according to the test set, and continuously adjusting the parameter training model to obtain a final information extraction model.
It is worth to say that the invention adopts the accuracy rate-recall rate-harmonic mean value (P-R-F1) as the model evaluation index, when the P-R-F1 accords with the preset model requirement, the model is shown to reach the optimal state, and the information extraction model is output.
It should be noted that the information extraction model trained by the present invention is an information extraction model suitable for the financial field.
In the embodiment of the invention, the relation extraction accuracy is higher through the deep learning neural network model, the efficiency and accuracy of manually knowing and checking the public opinion events are improved, and the financial institutions can know the general view of the public opinion events more quickly.
And 204, encoding the structured public opinion data set through a sentence embedding model to obtain encoded data.
In the embodiment of the invention, the sentence embedding model is a multi-language BERT-based embedding model, namely: the Language agnostic BERT sentence embedding (LaBSE) model can generate cross-Language sentence embedding for 109 languages, and has great contribution to increasing the model and vocabulary coverage rate.
Specifically, the structured public opinion data set is input into a LaBSE model for encoding, and encoded data is output.
And 205, performing context information learning on the coded data through a bidirectional long and short time memory network (BiLSTM) to obtain context data.
Specifically, the encoded data is input to the BiLSTM, the context information learning is performed on the encoded data, and the context data is output.
And 206, training an Attention mechanism (Attention) algorithm according to the context data, and constructing an emotion analysis model.
In the embodiment of the invention, an Attention mechanism (Attention) algorithm is widely applied to the field of Natural Language Processing (NLP). Attention mechanism refers to the thought of human brain attention, and the attention mechanism provides a visual attention mechanism from human, and attention weight learned by the attention mechanism reflects the correlation between data needing to be output currently and data at certain specific positions of an input sequence.
Specifically, an Attention mechanism (Attention) algorithm is trained through context data, emotion information in a text is more fully learned, and an emotion analysis model is generated.
Step 207, constructing a corresponding event sub-map according to each piece of structured public opinion data.
In the embodiment of the invention, the structured public opinion data set comprises event trigger words, emotion types, character relations, event relations, public opinion entities and entity attributes.
Specifically, public opinion entities are determined as nodes, character relationships and event relationships are determined as edges, entity attributes are determined as node attributes, emotion types and event trigger words are determined as sub-spectrum attributes, and an event sub-spectrum is constructed.
It should be noted that, for the structured public opinion data generated by the free text data, an event sub-map is constructed by taking a piece of network public opinion event as a unit, for example: constructing an event sub-map by taking a news report as a unit; constructing an event sub-map by taking a work order as a unit for the structured public opinion data generated by the work order data; and constructing an event sub-map for the structured public opinion data generated by the feedback data by using a questionnaire or a service evaluation.
Step 208, classifying event sub-atlas according to preset classification conditions, and constructing a public opinion event atlas set, wherein the public opinion event atlas set comprises a plurality of atlas types and event sub-atlas corresponding to each atlas type.
In the embodiment of the invention, the classification condition can be set according to the actual requirement, and the embodiment of the invention is not limited to the above. As an alternative, the classification condition is a combined value of the event trigger word, emotion type and event relationship.
Specifically, according to the combined value of the event trigger word, the emotion type and the event relation, the event sub-atlas is classified, and a public opinion event atlas set is constructed. Namely: and classifying event sub-atlases with the same event trigger word, emotion type and event relation into the same atlas type to obtain a plurality of atlas types and event sub-atlases corresponding to each atlas type.
Further, the plurality of pattern types and the event sub-pattern corresponding to each pattern type are stored in a Neo4j pattern database.
Step 209, obtaining public opinion data to be detected.
In the embodiment of the present invention, the public opinion data to be detected may be real-time data.
And 210, extracting information from the public opinion data to be detected through an information extraction model to obtain target public opinion information.
Specifically, the public opinion data to be detected is input into an information extraction model to extract information, and target public opinion information is output. The target public opinion information includes event trigger words, personage relationships, event relationships, public opinion entities and entity attributes.
In the embodiment of the invention, the relation extraction accuracy is higher through the deep learning neural network model, the efficiency and accuracy of manually knowing and checking the public opinion events are improved, and the financial institutions can know the general view of the public opinion events more quickly.
And 211, carrying out emotion analysis on the public opinion data to be detected through an emotion analysis model to obtain a target emotion type.
Specifically, the public opinion data to be detected is input into an emotion analysis model to carry out emotion positive and negative analysis, and a target emotion type is output. The target emotion type includes positive or negative.
Further, according to the target public opinion information and the target emotion type corresponding to the public opinion data to be detected, key public opinion information corresponding to the public opinion data to be detected is generated.
And 212, extracting the corresponding public opinion types from the key public opinion information according to the classification conditions.
In the embodiment of the invention, the key public opinion information comprises target public opinion information and target emotion type.
Taking a classification condition as a combined value of an event trigger word, an emotion type and an event relation as an example, determining the event trigger word, the emotion type and the event relation of current public opinion data to be detected from key public opinion information, wherein the public opinion type of the public opinion data to be detected is the combined value of the event trigger word, the emotion type and the event relation.
And step 213, matching the public opinion type with the type of the public opinion event atlas to obtain a corresponding target event sub-atlas.
Specifically, according to the public opinion types, searching and matching the types of the public opinion event atlas, and screening out corresponding event sub-atlas; and determining the screened map as a target event sub-map.
And 214, obtaining corresponding public opinion event information according to the target event sub-map.
In the embodiment of the invention, the target event sub-graph comprises event trigger words, emotion types, character relationships, event relationships, public opinion entities and entity attributes, and the event trigger words, emotion types, character relationships, event relationships, public opinion entities and entity attributes are determined to be public opinion event information.
Further, the public opinion event information is sent to a business personnel terminal, so that the business personnel can distinguish whether the public opinion event information of the historical public opinion event is associated with the current public opinion event, and whether the thought of the solution can be obtained from the historical public opinion event.
Further, according to the preset time interval, updating the information extraction model and the emotion analysis model according to the multiple target event sub-maps. The time interval may be set according to actual requirements, which is not limited in the embodiment of the present application.
In the embodiment of the application, the internal user opinion and user work order data are effectively utilized, and the association relationship between the internal user opinion and the public opinion event is mined by combining the social public opinion information, so that help is provided for processing the public opinion event and finding a break. Through the continuous increase of knowledge acquisition, the knowledge base becomes larger and larger, the knowledge graph becomes more perfect, and the system is more sound. The data are arranged in advance so that when the public opinion event occurs, the solution thought can be quickly and accurately obtained from the knowledge base, and the association relation of the public opinion event is discovered.
It is worth to be noted that, in the technical scheme of the application, the acquisition, storage, use, processing and the like of the data all conform to the relevant regulations of laws and regulations. The user information in the embodiment of the application is obtained through legal compliance approaches, and the user information is obtained, stored, used, processed and the like through user authorization consent.
In the technical scheme of the public opinion information detection method provided by the embodiment of the application, the public opinion data to be detected is obtained; carrying out key information extraction on public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information; the key public opinion information is matched with a public opinion event map set constructed in advance to obtain corresponding public opinion event information, and the key public opinion information of the current public opinion event is matched with the public opinion event map set constructed in advance to match a plurality of associated public opinion events, so that the association relation among the plurality of public opinion events is excavated, the public opinion events are responded rapidly and accurately, the completeness of data and the accuracy of an emotion analysis method are not required to be completely relied on, the time consumption and the manpower resource cost are reduced, and the detection accuracy and efficiency are improved.
Fig. 3 is a schematic structural diagram of a public opinion information detection apparatus according to an embodiment of the present invention, where the apparatus is configured to execute the public opinion information detection method, as shown in fig. 3, and the apparatus includes: a first acquisition unit 11, an information extraction unit 12, and an information matching unit 13.
The first obtaining unit 11 is configured to obtain public opinion data to be detected.
The information extraction unit 12 is configured to extract key information from the public opinion data to be detected by using a pre-constructed information extraction model and emotion analysis model, so as to obtain key public opinion information.
The information matching unit 13 is configured to match the key public opinion information with a public opinion event atlas constructed in advance, so as to obtain corresponding public opinion event information.
In the embodiment of the invention, the device further comprises: a second acquisition unit 14, a preprocessing unit 15, and an information extraction model construction unit 16.
The second obtaining unit 14 is used for obtaining historical public opinion data.
The preprocessing unit 15 is configured to perform a structured preprocessing on the historical public opinion data, and structure a public opinion data set.
The information extraction model construction unit 16 is configured to train the bidirectional long-short-term memory network in combination with the double-layer attention mechanism algorithm according to the structured public opinion data set, and construct an information extraction model.
In the embodiment of the invention, the historical public opinion data comprises free text data, work order data and feedback data; the second obtaining unit 14 is specifically configured to obtain free text data through web crawler technology; and acquiring work order data and feedback data through a system database.
In the embodiment of the present invention, the preprocessing unit 15 is specifically configured to perform word segmentation processing on the historical public opinion data to obtain word segmentation data; and carrying out attribute labeling and emotion labeling on the word segmentation data to obtain a structured public opinion data set.
In the embodiment of the invention, the device further comprises: an encoding unit 17, a context learning unit 18, and an emotion analysis model construction unit 19.
The encoding unit 17 is configured to encode the structured public opinion dataset through a sentence embedding model to obtain encoded data.
The context learning unit 18 is configured to learn context information of the encoded data through a bidirectional long-short-time memory network, so as to obtain context data.
The emotion analysis model construction unit 19 is configured to train an attention mechanism algorithm according to the context data, and construct an emotion analysis model.
In the embodiment of the invention, the key public opinion information comprises target public opinion information and target emotion type; the information extraction unit 12 is specifically configured to extract information from the public opinion data to be detected through an information extraction model, so as to obtain target public opinion information; and carrying out emotion analysis on the public opinion data to be detected through an emotion analysis model to obtain a target emotion type.
In the embodiment of the invention, the structured public opinion data set comprises a plurality of structured public opinion data; the apparatus further comprises: an event sub-map construction unit 20 and a classification unit 21.
The event sub-map construction unit 20 is configured to construct a corresponding event sub-map according to each piece of structured public opinion data.
The classifying unit 21 is configured to classify event sub-atlas according to a preset classifying condition, and construct a public opinion event atlas set, where the public opinion event atlas set includes a plurality of atlas types and event sub-atlas corresponding to each atlas type.
In the embodiment of the invention, the structured public opinion data set comprises event trigger words, emotion types, character relations, event relations, public opinion entities and entity attributes; the event sub-spectrum construction unit 20 is specifically configured to determine public opinion entities as nodes, character relationships and event relationships as edges, entity attributes as node attributes, emotion types and event trigger words as sub-spectrum attributes, and construct an event sub-spectrum.
In the embodiment of the present invention, the event sub-graph construction unit 20 is specifically configured to extract a corresponding public opinion type from the key public opinion information according to the classification condition; matching the public opinion type with the map type in the public opinion event map set to obtain a corresponding target event sub-map; and obtaining corresponding public opinion event information according to the target event sub-map.
In the scheme of the embodiment of the invention, the public opinion data to be detected is obtained; carrying out key information extraction on public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information; the key public opinion information is matched with a public opinion event map set constructed in advance to obtain corresponding public opinion event information, and the key public opinion information of the current public opinion event is matched with the public opinion event map set constructed in advance to match a plurality of associated public opinion events, so that the association relation among the plurality of public opinion events is excavated, the public opinion events are responded rapidly and accurately, the completeness of data and the accuracy of an emotion analysis method are not required to be completely relied on, the time consumption and the manpower resource cost are reduced, and the detection accuracy and efficiency are improved.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer device, which may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
The embodiment of the application provides a computer device, which comprises a memory and a processor, wherein the memory is used for storing information comprising program instructions, the processor is used for controlling the execution of the program instructions, and the program instructions realize the steps of the embodiment of the public opinion information detection method when being loaded and executed by the processor.
Referring now to FIG. 4, there is illustrated a schematic diagram of a computer device 600 suitable for use in implementing embodiments of the present application.
As shown in fig. 4, the computer apparatus 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate works and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the computer device 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a liquid crystal feedback device (LCD), and the like, and a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on drive 610 as needed, so that a computer program read therefrom is mounted as needed as storage section 608.
In particular, according to embodiments of the present invention, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The technical scheme of the application obtains, stores, uses, processes and the like the data, which all meet the relevant regulations of national laws and regulations.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.

Claims (13)

1. A public opinion information detection method, the method comprising:
obtaining public opinion data to be detected;
extracting key information from the public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information;
and matching the key public opinion information with a public opinion event atlas constructed in advance to obtain corresponding public opinion event information.
2. The public opinion information detection method according to claim 1, wherein before the extracting key information from the public opinion data to be detected by the pre-constructed information extraction model and emotion analysis model, the method further comprises:
acquiring historical public opinion data;
carrying out structural pretreatment on the historical public opinion data, and structuring a public opinion data set;
And training a bidirectional long-short-term memory network in combination with a double-layer attention mechanism algorithm according to the structured public opinion data set, and constructing the information extraction model.
3. The public opinion information detection method of claim 2, wherein the historical public opinion data includes free text data, work order data, and feedback data;
the obtaining the historical public opinion data includes:
acquiring the free text data through a web crawler technology;
and acquiring the work order data and the feedback data through a system database.
4. The public opinion information detection method of claim 2, wherein the structuring preprocessing of the historical public opinion data, structuring public opinion data sets, includes:
performing word segmentation processing on the historical public opinion data to obtain word segmentation data;
and carrying out attribute labeling and emotion labeling on the word segmentation data to obtain the structured public opinion data set.
5. The public opinion information detection method of claim 2, wherein the method further comprises:
encoding the structured public opinion data set through a sentence embedding model to obtain encoded data;
performing context information learning on the coded data through a bidirectional long-short-time memory network to obtain context data;
Training an attention mechanism algorithm according to the context data, and constructing the emotion analysis model.
6. The public opinion information detection method of claim 1, wherein the key public opinion information includes target public opinion information and target emotion type;
the key information extraction is carried out on the public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information, and the method comprises the following steps:
extracting information from the public opinion data to be detected through the information extraction model to obtain target public opinion information;
and carrying out emotion analysis on the public opinion data to be detected through the emotion analysis model to obtain a target emotion type.
7. The public opinion information detection method of claim 2, wherein the structured public opinion dataset comprises a plurality of structured public opinion data;
before the key public opinion information is matched with the public opinion event atlas constructed in advance to obtain the corresponding public opinion event information, the method further comprises the following steps:
constructing a corresponding event sub-map according to each piece of structured public opinion data;
classifying the event sub-atlas according to preset classification conditions, and constructing a public opinion event atlas set, wherein the public opinion event atlas set comprises a plurality of atlas types and event sub-atlas corresponding to each atlas type.
8. The public opinion information detection method of claim 7, wherein the structured public opinion dataset includes event trigger words, emotion types, persona relationships, event relationships, public opinion entities, and entity attributes;
the constructing a corresponding event sub-map according to each piece of the structured public opinion data comprises the following steps:
and determining the public opinion entity as a node, determining the character relationship and the event relationship as edges, determining the entity attribute as a node attribute, determining the emotion type and the event trigger word as sub-spectrum attributes, and constructing the event sub-spectrum.
9. The public opinion information detection method of claim 8, wherein the matching the key public opinion information with a pre-constructed public opinion event atlas to obtain corresponding public opinion event information includes:
extracting corresponding public opinion types from the key public opinion information according to the classification conditions;
matching the public opinion type with the map type in the public opinion event map set to obtain a corresponding target event sub-map;
and obtaining corresponding public opinion event information according to the target event sub-map.
10. A public opinion information detection apparatus, the apparatus comprising:
The first acquisition unit is used for acquiring public opinion data to be detected;
the information extraction unit is used for extracting key information from the public opinion data to be detected through a pre-constructed information extraction model and an emotion analysis model to obtain key public opinion information;
and the information matching unit is used for matching the key public opinion information with a public opinion event atlas constructed in advance to obtain corresponding public opinion event information.
11. A computer-readable medium on which a computer program is stored, characterized in that the program, when executed by a processor, implements the public opinion information detection method of any one of claims 1 to 9.
12. A computer device comprising a memory for storing information including program instructions and a processor for controlling execution of the program instructions, wherein the program instructions when loaded and executed by the processor implement the public opinion information detection method of any of claims 1 to 9.
13. A computer program product comprising computer programs/instructions which when executed by a processor implement the public opinion information detection method of any of claims 1 to 9.
CN202310736949.7A 2023-06-20 2023-06-20 Public opinion information detection method and device Pending CN116881395A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310736949.7A CN116881395A (en) 2023-06-20 2023-06-20 Public opinion information detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310736949.7A CN116881395A (en) 2023-06-20 2023-06-20 Public opinion information detection method and device

Publications (1)

Publication Number Publication Date
CN116881395A true CN116881395A (en) 2023-10-13

Family

ID=88261282

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310736949.7A Pending CN116881395A (en) 2023-06-20 2023-06-20 Public opinion information detection method and device

Country Status (1)

Country Link
CN (1) CN116881395A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117494068A (en) * 2023-11-17 2024-02-02 之江实验室 Network public opinion analysis method and device combining deep learning and causal inference

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117494068A (en) * 2023-11-17 2024-02-02 之江实验室 Network public opinion analysis method and device combining deep learning and causal inference
CN117494068B (en) * 2023-11-17 2024-04-19 之江实验室 Network public opinion analysis method and device combining deep learning and causal inference

Similar Documents

Publication Publication Date Title
Resch et al. Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment
Zhou et al. VictimFinder: Harvesting rescue requests in disaster response from social media with BERT
US9535980B2 (en) NLP duration and duration range comparison methodology using similarity weighting
WO2018184518A1 (en) Microblog data processing method and device, computer device and storage medium
CN108153729B (en) Knowledge extraction method for financial field
CN111967761A (en) Monitoring and early warning method and device based on knowledge graph and electronic equipment
CN108241867B (en) Classification method and device
CN112686022A (en) Method and device for detecting illegal corpus, computer equipment and storage medium
CN111309910A (en) Text information mining method and device
CN111783016A (en) Website classification method, device and equipment
CN113986864A (en) Log data processing method and device, electronic equipment and storage medium
CN113342972A (en) Public opinion recognition model training method and system and public opinion risk monitoring method and system
CN116881395A (en) Public opinion information detection method and device
CN111858903A (en) Method and device for negative news early warning
CN112363996B (en) Method, system and medium for establishing physical model of power grid knowledge graph
CN116305257A (en) Privacy information monitoring device and privacy information monitoring method
Thanos et al. Combined deep learning and traditional NLP approaches for fire burst detection based on twitter posts
KR20230059364A (en) Public opinion poll system using language model and method thereof
CN114238641A (en) Method, device, equipment, storage medium and program product for mining operation and maintenance knowledge
CN113191137A (en) Operation risk obtaining method and system, electronic equipment and storage medium
CN111581533A (en) State recognition method and device of target object, electronic equipment and storage medium
Dashdorj et al. High‐level event identification in social media
CN112699245A (en) Construction method and device and application method and device of budget management knowledge graph
Kumaresh et al. Mining software repositories for defect categorization
Orellana et al. Evaluating named entities recognition (NER) tools vs algorithms adapted to the extraction of locations

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination