KR102096066B1 - Social topics extraction system - Google Patents

Abstract

It provides a social topic extraction system capable of extracting high-quality social topics. The social topic extraction system according to the present invention is a model building sub-system that generates a topic extraction model based on social data collected on a social network through a network, and extracts social topics from social data collected using a topic extraction model. Topic extraction subsystem, user interface to deliver user's request through the network to the topic extraction subsystem, and provide the extracted social topics to the user, and perform accuracy evaluation on the extracted social topics and It includes a model feedback module that aggregates user feedback and requests optimization or regeneration of the topic extraction model to the model building subsystem.

Description

Social topics extraction system

The present invention relates to a social topic extraction system, and more particularly, to a social topic extraction system that enables extraction of a social topic from a large amount of social data on a social network.

The present invention was derived from a study conducted by Metasoft Co., Ltd. as part of the 2017 Industry-Academic Cooperation Technology Development Project (First Step, Leap).

[Research period: 2017. 09. 01 ~ 2018. 08. 31, Research institute: Seoil University Industrial-Academic Cooperation Group, Research project name: Development of social context extraction and classification framework based on media context using deep learning technology, task identification number : C0532698]

As the number of social channels providing various services online increases and becomes active, the amount of social data generated in these social channels increases exponentially and becomes very vast. In order to utilize such social data, it is necessary to analyze a large amount of social data and extract social topics, which are the subjects of social data.

Existing social topic extraction systems allow users to search social content including keywords based on a set of related keywords according to the application of a statistical analysis model. In social networks where various social channels are provided, new words are created and trends are continuously changed according to openness and autonomy, but existing social topic extraction systems build statistical analysis models at a certain point in time, so additional meanings or keywords There is a problem that is not considered.

In addition, the social content is a natural language level unstructured data type without a formalized form, and simply determining the association based on whether keywords are included has a problem in that the quality of extracting social topics decreases.

In order to solve the above problems, the technical problem of the present invention is to provide a social topic extraction system capable of extracting high-quality social topics.

In order to solve the above technical problem, there is provided a social topic extraction system capable of extracting high-quality social topics.

The social topic extraction system according to the present invention includes a model building sub-system for generating a topic extraction model based on social data collected on a social network through a network; A topic extraction subsystem that extracts a social topic from the collected social data using the topic extraction model; A user interface that delivers a user's request through the network to the topic extraction subsystem, and provides the extracted social topic to the user; And a model feedback module that performs an accuracy evaluation on the extracted social topic, synthesizes the user's feedback on the extracted social topic, and requests the model building subsystem to optimize or regenerate the topic extraction model; It includes.

The social topic extraction system according to the present invention is capable of constructing a topic extraction model by mixing statistical analysis techniques and machine learning techniques, and regenerating and optimizing the topic extraction model, thereby enabling high-quality social topic extraction and maintenance. You can do it.

1 is a schematic diagram showing the configuration of a social topic extraction system according to an embodiment of the present invention.
2 is a schematic diagram showing the configuration of a model building subsystem of a social topic extraction system according to an embodiment of the present invention.
3 is a schematic diagram showing the configuration of a topic extraction sub-system of a social topic extraction system according to an embodiment of the present invention.
4 is a conceptual diagram illustrating a process of selecting an input variable in a model building subsystem of a social topic extraction system according to an embodiment of the present invention.
5 is a conceptual diagram illustrating a process of constructing a topic extraction model in a model building subsystem of a social topic extraction system according to an embodiment of the present invention.
6 and 7 exemplarily show results of providing a social topic extracted from a social topic extraction system according to an embodiment of the present invention.

Hereinafter, the social topic extraction system according to the embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the present invention is not limited to the following embodiments, and those skilled in the art will recognize the present invention. The present invention may be implemented in various other forms without departing from the technical spirit of the. That is, specific structural or functional descriptions are exemplified only for the purpose of describing the embodiments of the present invention, and the embodiments of the present invention may be implemented in various forms and should be interpreted as being limited to the embodiments described in the text. No. It is not limited by the embodiments described in the text, it should be understood to include all modifications, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprises" or "includes" are intended to indicate that a feature, number, step, action, component, or combination thereof described is present, one or more other features or numbers, It will be understood that the existence or addition possibilities of steps, actions, components, or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms, such as those defined in a commonly used dictionary, are interpreted to have meanings consistent with meanings in the context of related technologies, and are not to be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. .

In this specification, social data refers to data generated and transmitted in social channels serviced on a social network. For example, the conventional web is made of a static file, but social data generated on a social network such as Twitter has a dynamic flow of transmitting very small information to the world in real time. In other words, social data can have a dynamic flow on social networks.

1 is a schematic diagram showing the configuration of a social topic extraction system according to an embodiment of the present invention.

Referring to FIG. 1, the social topic extraction system 1 may include a model building subsystem 200 and a topic extraction subsystem 300. The social topic extraction system 1 collects and stores social data such as the social channel 21, the social content 22, and the writing record 23 from the social network 20 through the network 50. 510) may be further included.

The network 50 includes wired Internet service, local area network (LAN), broadband communication network (WAN), intranet, wireless Internet service, mobile computing service, wireless data communication service, wireless Internet access service, satellite communication service, wireless LAN, Bluetooth, etc. It can include anything that can send and receive data over wired or wireless. When the network 50 is connected to a smartphone or tablet, the network 50 may be wireless data communication services such as 3G, 4G, 5G, wireless LAN such as Wi-Fi, Bluetooth, or the like.

The social channel 21 may be each service provided on the social network 20, the social content 22 may be data created in each social channel 21, and the creation record 23 may be created date and time, classification, It may be additional information that the social channel 21 has for the social content 22 such as the author.

The social data DB 510 may store or permanently store the social data for a relatively long time, but may also temporarily store the social data. The social data DB 510 may be a space that can store data in any form, for example, NoSQL, relational database, or file system. The social data DB 510 may be one storage device that is logically divided, a division unit that logically divides one or a plurality of storage devices, or one storage unit that is physically divided or one of the logical divisions. You can.

The social topic extraction system 1 may generate a topic extraction model in the model building subsystem 200 and store it in the topic extraction model DB 520 based on the collected and stored social data in the social data DB 510. . The topic extraction model DB 520 may be a space for storing data separate from the social data DB 510, but the social data DB 510 and the topic extraction model DB 520 may be one space for storing data. It may be.

The model building subsystem 200 may include an input variable selection module 210, an extraction model building module 220, and a model optimization module 230.

The input variable selection module 210 receives social data and analyzes semantic associations and topic associations between social contents to select input variables in order of high influence. The input variable selection module 210 may select an input variable by comparing it based on a calculation speed and an extraction accuracy among possible combinations of features extracted from social data.

The extraction model building module 220 measures the accuracy of semantic association and subject association of social data through the placement of the learning layer based on the selected input variable, and time series analysis technique among statistical techniques such as moving average method and exponential smoothing method. A topic extraction model optimized for topic extraction from social data based on semantic relevance and topic relevance by mixing statistical analysis techniques such as Random Forest and Support Vector Machine, etc. Build an ensemble model. The topic extraction model can be selected based on the characteristics of the social data and input variables to be analyzed.

The model optimization module 230 trains the selected topic extraction model using a training data set selected from social data, and a test data set selected from social data other than the training data set. The extraction result can be evaluated using. The model optimization module 230 may be composed of input variable optimization and parameter optimization. The selected topic extraction model can be optimized based on characteristics such as operation speed and accuracy. Optimization of the selected topic extraction model can be performed for input parameters and hyperparameters of the model.

The evaluation result obtained from the model optimization module 230 is fed back, reflected in the input variable selection in the input variable selection module 210, and can be reflected in the optimization of the topic extraction model selected in the extraction model construction module 220, You can build an optimized topic extraction model. In some embodiments, the evaluation result in the model optimization module 230 may be reflected in the construction of a topic extraction model of another combination as a feedback input.

The social topic extraction system 1 uses the topic extraction model stored in the topic extraction model DB 520 to extract social topics from the topic extraction subsystem 300 from the social data collected and stored in the social data DB 510. You can.

The topic extraction sub-system 300 receives a request through the network 50 of the user 10 from the user interface (UI, 100), and receives the social topic extracted according to the request through the user interface 100 through the user 10 ).

The user interface 100 may provide an interface for accessing the social topic extraction system 1 through a terminal or the like used by the user 10. The user 10 may transmit a query such as a keyword to the social topic extraction system 1 through the user interface 100, and extract the extracted topics provided by the social topic extraction system 1 through the user interface 100. I can receive it.

The topic extraction sub-system 300 may include a semantic association extraction module 310, a topic association extraction module 320, and a topic prediction module 330.

Each of the semantic association extraction module 310 and the topic association extraction module 320 may extract semantic association and topic association from meanings and topics of each of the social data stored in the social data DB 510. The topic prediction module 330 uses the topic extraction model stored in the topic extraction model DB 520 to determine the semantic association and topic association of social data extracted from each of the semantic association extraction module 310 and the topic association extraction module 320. By analyzing, social topics can be extracted from social data. The topic prediction module 330 may collect the social data stored in the social data DB 510 and have similar meanings and topics, and extract the social topics. When the social topic extraction method in the topic prediction module 330 is described with reference to a graph (right graph) of the principal component analysis result illustrated in FIG. 4, each of the first principal component PC1 and the second principal component PC2 is used as a coordinate axis. When reconstructing the social data, it can be seen that the social data, each of which is similar to the first main component PC1 and the second main component PC2, is divided into several groups. For example, the topic prediction module 330 may extract social topics by selecting meanings and / or topics representing each group.

The social topic extraction system 1 may further include a model feedback module 400. The model feedback module 400 receives and analyzes one feedback through the user interface 100 on the social topics provided by the user 10, and analyzes the social feedback extracted from the user feedback unit 410 and the topic extraction subsystem 300 It may include a prediction result evaluation unit 420 to evaluate the.

The model feedback module 400 is the accuracy resulting from the user 10 evaluation obtained by analyzing feedback on the social topic according to the feedback of the user 10 received from the user feedback unit 410, and the prediction result evaluation unit 420 ), If any one of the accuracy of the social topics extracted from the topic extraction sub-system 300 evaluated in) is lower than the predicted accuracy in the process of constructing the topic extraction model stored in the topic extraction model DB 520, the model building sub The system 200 may be requested to regenerate the topic extraction model.

The model feedback module 400 may synthesize feedback from the user 10 and evaluations of social topics extracted from the topic extraction sub-system 300, and request the model construction sub-system 200 to regenerate the topic extraction model. . For example, the accuracy resulting from the evaluation of the user 10 obtained by analyzing feedback on the social topic according to the feedback of the user 10 received from the user feedback unit 410 is evaluated by the prediction result evaluation unit 420 If the accuracy of the social topic extracted from one topic extraction sub-system 300 is lower than the accuracy, the model feedback module 400 may request the model building subsystem 200 to regenerate the topic extraction model.

The model building sub-system 200, which is requested to regenerate the topic extraction model from the model feedback module 400, re-optimizes the topic extraction model through the model optimization module 230, or extracts the model building module 220 Through this, it is possible to construct a new topic extraction model.

In some embodiments, the social topic extraction system 1 may be configured such that each of the model building subsystem 200 and the topic extraction subsystem 300 separately forms a system.

For example, the model building system and user interface 100 including the model building subsystem 200, the social data DB 510, and the topic extraction model DB 520, the topic extraction subsystem 300, and model feedback The topic extraction system including the module 400, the social data DB 510, and the topic extraction model DB 520 may form a separate system. In this case, the model feedback module 400 of the topic extraction system may provide feedback to the model building system. The topic extraction system and the model building system may each independently have a social data DB 510, or may share a single social data DB 510. The topic extraction system and the model building system may each independently have a topic extraction model DB 520, but may be linked to each other or share a single topic extraction model 520.

The social topic extraction system 1 according to the present invention can build a topic extraction model by mixing statistical analysis techniques and machine learning techniques, and regenerate and optimize the topic extraction model, so that high-quality social topic extraction and maintenance Can make it possible.

2 is a schematic diagram showing the configuration of a model building subsystem of a social topic extraction system according to an embodiment of the present invention.

Referring to FIG. 2, the model building subsystem 200 generates and builds a topic extraction model using social data collected from the social network 20 through the network 50 and stored in the social data DB 510. It can be stored in the topic extraction model DB 520.

The model building subsystem 200 may include an input variable selection module 210, an extraction model building module 220, a model optimization module 230, and a natural language understanding module 240.

The natural language understanding module 240 performs analysis processes such as semantic role labeling, morpheme analysis, syntax analysis, object name analysis, intention classification, and domain analysis on the social data stored in the social data DB 510. By performing, it can be provided to the input variable selection module 210.

In some embodiments, the natural language understanding module 240 may provide the natural language analysis results for social data to the input variable selection module 210 as structured data having a table structure or a tree structure. Social data consists of file formats that can contain various types of information, such as CSV, XLS, XLSX, HTML, WMS, PDF, XML, RDF, ZIP, Open API, DOC, DOCX, HWP, PPT, PPTX, etc. It can have various types of structures within each file format. The structured data that can be provided by the natural language understanding module 240 may be, for example, Relation DateBase (RDB), Comma-Seperated Variables (CSV), eXtensible Markup Language (XML), JavaScript Object Notation (JSON), etc. It is not limited to this.

The natural language understanding module 240 includes a semantic roll labeling unit (SRL, 241), a morphological analysis unit 242, a syntax analysis unit 243, an entity name analysis unit 244, an intention classification unit 245, and a domain analysis unit ( 246). The semantic roll labeling unit 241 may give a semantic role to social data. The morpheme analysis unit 242 may separate social data to which a semantic role is assigned in morpheme units. The syntax analysis unit 243 and the entity name analysis unit 244 may perform syntax analysis and entity name analysis on social data separated into morpheme units. The intention classification unit 245 and the domain analysis unit 2560 may perform intention classification and domain analysis of social data to which a semantic role is assigned.

In some embodiments, the natural language understanding module 240 may assign a shoulder number to a morpheme separated from the morpheme analysis unit 242 with reference to the language dictionary information 600. Specifically, the intention classifying unit 256 may assign the shoulder number corresponding to the meaning of the homonym to a separate morpheme with reference to the language dictionary information 600.

The input variable selection module 210 may receive the social data analyzed in the natural language from the natural language understanding module 240 and analyze the semantic association and the subject association between the social data to select the input variables in the order of high influence.

The input variable selection module 210 may include a feature extraction unit 212 for extracting features corresponding to input variable candidates from social data, and a variable analysis unit 214 for analyzing the extracted features. The method of selecting the input variable will be described in detail through FIG. 4.

The extraction model building module 220 measures the accuracy of semantic association and subject association through the placement of the learning layer with respect to the social topic extraction results, and is optimized for topic extraction from social data by mixing statistical analysis techniques and machine learning techniques. A built-in topic extraction model.

The extraction model building module 220 may include a model building unit 222 for building a topic extraction model and a model evaluation unit 224 for evaluating the constructed topic extraction model. The method of constructing the topic extraction model will be described in detail with reference to FIG. 5.

The model optimization module 230 may learn and evaluate the selected topic extraction model. The model optimization module 230 may include an input variable optimization unit 232 that performs optimization for the selected input variable, and a parameter optimization unit 234 that performs optimization for the parameters.

For example, the parameter optimization unit 234 may optimize a model parameter that is a parameter that the machine learning technique learns from a training data set, or a hyperparameter that is a tuning parameter of the machine learning technique. The model parameters can be, for example, weight coefficients (or slopes) and biases (or intercepts of coordinate axes) of linear regression, and hyperparameters can be, for example, mean squared error of regression analysis, cost function ( cost function), or the regulatory intensity of the L2 penalty, or the maximum depth setting of the decision tree.

3 is a schematic diagram showing the configuration of a topic extraction sub-system of a social topic extraction system according to an embodiment of the present invention.

Referring to FIG. 3, the topic extraction subsystem 300 receives a request through the network 50 of the user 10 from the user interface 100 and collects it from the social network 20 through the network 50 For social data stored in the social data DB 510, a social topic can be extracted using the topic extraction model stored in the topic extraction model DB 520 and provided to the user 10 through the user interface 100.

have. In some embodiments, social topics extracted from the topic extraction service system 300 may be stored in the topic extraction result DB 530. The topic extraction result DB 530 may be a space for storing data separate from the social data DB 510 and the topic extraction model DB 520, but the topic extraction result DB 530 may be a social data DB 510. And one space for storing data together with at least one of the topic extraction model DB 520.

The topic extraction sub-system 300 may include a semantic association extraction module 310, a topic association extraction module 320, a topic prediction module 330, and a natural language understanding module 340.

The natural language understanding module 340 performs an analysis process such as semantic roll labeling, morpheme analysis, syntax analysis, object name analysis, intention classification, and domain analysis on the social data stored in the social data DB 510, thereby semantic association The extraction module 310 and the subject association extraction module 320 may be provided.

The natural language understanding module 340 includes a semantic roll labeling unit 341, a morpheme analysis unit 342, a syntax analysis unit 343, an entity name analysis unit 344, an intention classification unit 345, and a domain analysis unit 346 Each of these may include, each of the semantic roll labeling unit (SRL, 241), morphological analysis unit 242, syntax analysis unit 243, entity name analysis unit included in the natural language understanding module 240 described in Figure 2 (244), the intention classification unit 245 and the domain analysis unit 246 are generally similar to each, detailed description thereof will be omitted.

In some embodiments, the social topic extraction system 1 shown in FIG. 1 may provide one natural language understanding module shared by the model building subsystem 200 and the topic extraction subsystem 300.

The semantic relevance extraction module 310 receives the social data that has been analyzed by the natural language understanding module 240 and constructs a semantic relevance dictionary based on the semantic relevance analysis unit 312 for analyzing semantic relevance and the analyzed semantic relevance A semantic association dictionary construction unit 314 is included. The semantic association analysis unit 312 may analyze the semantic lightness of social data by referring to the semantic association dictionary constructed by the semantic association dictionary construction unit 314.

The topic association extraction module 320 receives the social data that has been analyzed in the natural language from the natural language understanding module 240 and constructs a topic association dictionary based on the topic association analysis unit 322 for analyzing the topic association, and the analyzed topic association. And a topic association dictionary construction unit 324. The topic association analysis unit 322 may analyze the topic lightness of social data by referring to the topic association dictionary constructed by the topic association dictionary construction unit 324.

The topic prediction module 330 extracts the topic association extracted from the semantic association topic extraction unit 332 and the topic association extraction module 320 for extracting the meaning association topic based on the semantic association extracted from the semantic association extraction module 310. Social based on the semantic associative topic and the topic associative topic extracted from the topic associative topic extractor 334 and semantic associative topic extractor 332 and the topic associative topic extractor 334 for extracting the topic associative topic based And a social topic extraction unit 336 for extracting topics.

4 is a conceptual diagram illustrating a process of selecting an input variable in a model building subsystem of a social topic extraction system according to an embodiment of the present invention.

Referring to FIG. 4, social data may be high-dimensional data, such as a graph on the left. That is, the components of the features corresponding to the input variable candidates of each dimension may be very diverse. Principal Component Analysis, a data dimensionality reduction method, is used to extract and use factors that greatly affect social topics among various components.

Principal component analysis can transform high-dimensional data, such as the left graph, into low-dimensional data, such as the right graph, and transform social data sets into new coordinate axes. Therefore, the principal component analysis can reconstruct each of the social data with independent coordinate axes while minimizing data loss, thereby obtaining a maximum effect in a minimum dimension. Through this, while minimizing information loss, it is possible to find principal components representing social data and reduce the dimension (number) of variables to reduce data overlap due to variables.

For example, through the principal component analysis, the first principal component PC1 and the second principal component PC2 may be extracted as factors that greatly affect the social topic, and social data may be converted into two-dimensional data.

In some embodiments, the principal component analysis may extract the principal components with reduced dimensions, for example, the first principal component (PC1) and the second principal component (PC2) using the genetic algorithm. Genetic algorithms are computational models based on the genetic and evolutionary mechanisms of the natural world, which represent possible solutions to the problem to be solved in a defined data structure (get_fitness), then gradually transform them (_generate_parent and _mutate) to become more and more. You can generate good solutions (get_best). Each possible solution is viewed as one organism or individual, and a collection of these is called a population.

For example, a modification algorithm for a genetic algorithm can be configured as follows.

def _mutate (parent, geneSet, get_fitness):

childGenes = parent.Genes [:]

index = random.randrange (0, len (parent.Genes))

newGene, alternate = random.sample (geneSet, 2)

childGenes [index] = alternate \

if newGene == childGenes [index] \

else newGene

fitness = get_fitness (childGenes)

return Chromosome (childGenes, fitness)

def _generate_parent (length, geneSet, get_fitness):

genes = []

while len (genes) <length:

sampleSize = min (length-len (genes), len (geneSet))

genes.extend (random.sample (geneSet, sampleSize))

fitness = get_fitness (genes)

return Chromosome (genes, fitness)

def get_best (get_fitness, targetLen, optimalFitness, geneSet, display):

random.seed ()

bestParent = _generate_parent (targetLen, geneSet, get_fitness)

display (bestParent)

if bestParent.Fitness> = optimalFitness:

return bestParent

while True:

child = _mutate (bestParent, geneSet, get_fitness)

if bestParent.Fitness> = child.Fitness:

continue

display (child)

if child.Fitness> = optimalFitness:

return child

bestParent = child

def get_fitness (genes, target):

return sum (1 for expected, actual in zip (target, genes)

if expected == actual)

In some embodiments, each of the first main component PC1 and the second main component PC2 may be a meaning and a subject. In this case, it can be determined that social data having a similar first principal component PC1 has semantic association, and social data having a similar second principal component PC2 has subject association. Social data is largely composed of the author, the date of creation, and the content, the content is described in natural language, and the natural language is composed of several sentences and words. The first main component (PC1) is vectorized based on the occurrence position and the number of occurrences of sentences and words constituting each social data to extract a list of central words, and multidimensional based on other social data, author, creation date, and central words. Construct a matrix. At this time, vectorization for extracting the central word may calculate the distance value between words using Word2Vec, and based on this, similarity between words may be determined. The determination of semantic relevance is made through the degree of similarity based on the ranking of the central words of each social data and the list of similar words of each central word. That is, if the central word and the similar word list of the social data A and the social data B are larger than the criterion for similarity, it is determined that there is a semantic association. The criterion for similarity is made by real numbers from 0 to 1, and the closer it is to 1, the higher the similarity, starting with an initial value of 0.5 and gradually increasing with the number of training sets by machine learning and the number of semantic association sets according to the training results. And down to find the optimal value. The second principal component PC2 vectorizes the similarity between social data based on the social data itself based on topics determined as semantic association in the first principal component PC1. At this time, the commonality of similar social data sets is identified as a subject, and a distance value between social data is calculated using Doc2Vec. Accordingly, a social topic can be extracted from social data having substantially the first principal component PC1 and the second principal component PC2, that is, social data having semantic association and topic association.

5 is a conceptual diagram illustrating a process of constructing a topic extraction model in a model building subsystem of a social topic extraction system according to an embodiment of the present invention.

Referring to FIG. 5, the learning layer of the topic extraction model may be composed of three levels, level 1, level 2, and level 3.

이다. At level 1 (L1), statistical analysis techniques such as moving average method (M11), exponential smoothing method (M12), autocorrelation (M13), and Arima (ARIMA, M14) can be performed by time series analysis, respectively. The results analyzed through each statistical analysis technique undergo an iterative analysis process, and weights can be applied to the results. The weight is continuously updated according to the social topic extraction result and the extraction accuracy, and the sum of the weights is 1. That is, n statistical analysis techniques applied to the time series analysis Level 1 (L1) are TA ₁ , TA ₂ , TA ₃ ,… , TAn, each weight is WS ₁ , WS ₂ , WS ₃ ,… , WS _n , and the sum of the weights

to be.

이 된다. Therefore, the result of Level 1 (L1), which is a time series analysis delivered to Level 2 (L2),

It becomes.

이다. At level 2 (L2), machine learning techniques such as random forest (M21), support vector regression (M22), and neural net (M23) can be performed for topic extraction. The results analyzed through each machine learning technique undergo an iterative analysis process, and weights can be applied to the results. The weight is continuously updated according to the social topic extraction result and the extraction accuracy, and the sum of the weights is 1. That is, if k machine learning techniques applied to level 2 (L2) are defined as MA ₁ , MA ₂ , MA ₃ , ..., MA _k for topic extraction, each weight is WM ₁ , WM ₂ , WM ₃ , ..., WM _k , and the sum of the weights

to be.

이 된다. Level 3 (L3) builds an ensemble model (M31) that integrates the extraction results from level 2 (L2), and the topic extraction result value that is integrated in the level 3 (L3) ensemble model (M31) is

It becomes.

6 and 7 exemplarily show results of providing a social topic extracted from a social topic extraction system according to an embodiment of the present invention.

1, 6 and 7 together, the social topic extraction system 1 receives a request through the network 50 of the user 10 from the user interface 100, and extracts the social topic according to the request Can be provided to the user 10 through the user interface 100.

For example, when the user 10 inputs a request for 'economy', as shown in FIG. 6, the extracted social topics related to the economy 'enterprise', 'entrepreneur', 'fund', 'stock price', 'tax Social topics such as', 'exchange rate' may be provided to the user 10.

In some embodiments, when the user 10 selects 'enterprise' from the provided social topics, as shown in FIG. 7, the extracted social topics related to the enterprise, 'Samsung', 'Modern', 'LG', and 'SK' , 'POSCO', 'Hanhwa', 'GS', 'Kookmin Bank', 'Kookmin Bank', '100 Top Companies', 'CJ', etc. can also provide detailed social topics to the user 10.

When the user 10 selects a specific social topic, the social topic extraction system 1 may provide the user 10 with social data corresponding to the specific social topic selected by the user.

Embodiments of the present invention can be written as a program executable on a computer system. In addition, the corresponding program read from a computer-readable recording medium containing the program can be executed in a digital computer system.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, DVD-ROM, magnetic tapes, floppy disks, optical data storage devices, etc., and can also be used for carrier waves (for example, transmission over the Internet). It also includes those implemented in the form. The computer-readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

1: Social topic extraction system, 10: user, 20: social network, 50: network, 100: user interface, 200: model building subsystem, 210: input variable selection module, 220: extraction model building module, 230: model optimization Module, 300: topic extraction subsystem, 310: semantic association extraction module, 320: topic association extraction module, 330: topic prediction module, 510: social data DB, 520: topic extraction model DB, 530: topic extraction result DB, 600 : Language Dictionary Information

Claims (10)

A model building subsystem that generates a topic extraction model based on the social data collected on the social network through the network;
A user interface that receives a user's request through the network and provides the user with a social topic extracted according to the user's request;
A topic extraction subsystem that receives the user's request from the user interface and extracts the social topic according to the user's request from the collected social data using the topic extraction model; And
A model feedback module that performs an accuracy evaluation on the extracted social topic and aggregates the user's feedback on the extracted social topic, requesting the model building subsystem to optimize or regenerate the topic extraction model; It includes,
The topic extraction subsystem,
The center word list is extracted through the calculation of the distance value between words of each of the collected social data, and the similarity between the central word list of each of the collected social data is determined to mean the collected social data. A semantic association extraction module for determining association;
A topic association extraction module that analyzes a topic association of social data determined as semantic association by calculating a distance value between the social data determined as semantic association among the collected social data; And
And a topic prediction module that extracts the social topic according to the user's request by analyzing the analyzed semantic association and the analyzed topic association. According to claim 1,
The model building subsystem,
Principal component analysis using genetic algorithm is performed on the collected social data to convert the input variables by converting the collected social data, which is high-dimensional data, into low-dimensional data having a meaning and a subject as a first principal component and a second principal component. An input variable selection module to select;
Based on the selected input variable, the semantic association, which determines similarity through calculation of a distance value between words of each of the collected social data, and analysis through calculation of a distance value between the collected social data An extraction model building module that builds the topic extraction model by measuring the accuracy of the subject association; And
A model optimization module for training the topic extraction model using a training data set selected from the collected social data, and evaluating it using a test data set selected from the collected social data other than the training data set ; Social topic extraction system comprising a. delete delete According to claim 2,
The extraction model building module,
Has a learning layer consisting of level 1, level 2, and level 3,
In the level 1, a plurality of statistical analysis techniques are performed by time series analysis, and weights are applied to the results.
The level 2 receives the result of the level 1, performs a plurality of machine learning techniques, and applies weights to the result.
The level 3 is a social topic extraction system characterized by constructing an ensemble model by integrating the results of the level 2 applied with weights. The method of claim 5,
The model optimization module,
An input variable optimization unit for optimizing the input variable; And
And a parameter optimizer for optimizing the parameters of the topic extraction model. The method of claim 6,
The parameter optimization unit, the social topic extraction system, characterized in that to optimize the model parameters that are parameters learned by the plurality of machine learning techniques, or hyperparameters that are tuning parameters of the plurality of machine learning techniques. delete According to claim 1,
The model feedback module, when the accuracy evaluation result of analyzing the user's feedback on the extracted social topic is lower than the accuracy evaluation result on the extracted social topic, the model construction sub-system of the topic extraction model A social topic extraction system characterized by requesting optimization or regeneration. According to claim 1,
The model feedback module analyzes the predicted accuracy when constructing the topic extraction model in the topic extraction subsystem, the accuracy evaluation result for the extracted social topic, and the user's feedback for the extracted social topic. If it is lower than at least one of the accuracy evaluation results, the social topic extraction system, characterized in that to request the optimization or regeneration of the topic extraction model to the model building subsystem.

Images