KR102224684B1 - System and method for generating prediction of technology transfer base on machine learning - Google Patents

Abstract

The method for generating a technology transfer prediction model based on machine learning according to the present invention includes the steps of: (a) collecting patent data necessary for a data collection unit to generate a technology transfer prediction model; (b) classifying the patent data into numerical data and text data by a data classification unit; (c) a pre-processing step of allowing the data pre-processing module to analyze text data, which is unstructured data among the patent data; And (d) the technology transfer prediction model configuration module receives numerical data and preprocessed text data, performs topic modeling in which a technology topic is extracted from the text data through an ensemble model, and ensembles modeling with the technology topic and numerical data. It has the effect of objectively and easily evaluating the possibility of technology transfer to the population through the collected patent data, including the step of generating a technology transfer prediction model by performing it.

Description

Machine learning-based technology transfer prediction model generation system and generation method {SYSTEM AND METHOD FOR GENERATING PREDICTION OF TECHNOLOGY TRANSFER BASE ON MACHINE LEARNING}

The present invention relates to a machine learning-based technology transfer prediction system, and more specifically, collects patent data in a technical field, divides the collected patent data into numerical data and text data, and preprocesses them for double and text data. And a technology transfer prediction system based on machine learning that derives a technology topic of the patent data to be analyzed using a topic modeling technique, and uses the derived technology topic information and numerical data.

In the recent situation where the speed of knowledge growth is increasing exponentially, it is becoming more and more difficult to keep up with this speed of change with traditional R&D methods. As an alternative to this, technology transfer, one of the methods of technology management, exists.

Moreover, technology transfer is one of the important factors necessary for technology commercialization and has the advantage of being able to contribute sufficiently to job creation and economic revitalization. Although the KIPO-affiliated offices and the Technology Guarantee Fund are conducting various technology commercialization support projects in line with international trends, there is a limit to increasing the technology transfer rate.

This is because objective indicators that can confirm the excellence of numerous patents for technology commercialization are lacking, and there is no model suitable for the current system.

Therefore, there is a limitation in that most of the selection of patents for technology commercialization is mainly dependent on the opinions of experts.

Korean Registered Patent Publication No. 10-1806669 (2017.12.01)

In order to solve the above-described limitations, the present invention collects patent data in the technical field, divides the collected patent data into numerical data and text data, and uses pre-processing and topic modeling techniques for text data to analyze target patents. The purpose is to derive technical topics of data and to provide a machine learning-based technology transfer prediction system using the derived technical topic information and numerical data.

A method for generating a technology transfer prediction model based on machine learning according to the present invention for achieving the above object comprises: (a) collecting patent data required by a data collection unit to generate a technology transfer prediction model; (b) classifying the patent data into numerical data and text data by a data classification unit; (c) a pre-processing step of allowing the data pre-processing module to analyze text data, which is unstructured data among the patent data; And (d) the technology transfer prediction model configuration module receives numerical data and preprocessed text data, performs topic modeling in which a technology topic is extracted from the text data through an ensemble model, and ensembles modeling with the technology topic and numerical data. And generating a technology transfer prediction model by performing it.

The method for generating a technology transfer prediction model based on machine learning according to the present invention for achieving the above object is effective in removing unnecessary noise patents or duplicate patents from the patent data collected by the data preprocessing module in step (c). It characterized in that it further comprises a; of the patent, extracting the transferred patent case as numerical data.

Step (c) of the method for generating a technology transfer prediction model based on machine learning according to the present invention for achieving the above object includes: (c-1) removing stop words from sentences constituting the text data by the data preprocessing module; (c-2) selecting, by the data preprocessing module, an important word from the text data using a frequency-inverse document frequency (TF-IDF) method; (c-3) unifying, by the data preprocessing module, the shape of words constituting the text data through morpheme analysis; And (c-4) removing, by the data preprocessing module, numbers, punctuation marks, and symbols that may distort the analysis process.

모델로 문서내 기술토픽을 추출하는 것을 특징으로 한다.In step (d) of the method for generating a technology transfer prediction model based on machine learning according to the present invention for achieving the above object, the technology transfer prediction model construction module

It is characterized by extracting a technical topic in a document with a model.

앙상블 모델을 이용한 기술이전 예측모형을 생성하는 것을 특징으로 한다.In step (d) of the method for generating a technology transfer prediction model based on machine learning according to the present invention for achieving the above object, the technology transfer prediction model configuration module is

It is characterized by generating a technology transfer prediction model using an ensemble model.

In another embodiment, a system for generating a technology transfer prediction model based on machine learning according to the present invention for achieving the above object comprises: a data collection unit for collecting patent data required to generate a technology transfer prediction model; A data classification unit for classifying the patent data into numerical data and text data; A data pre-processing module for pre-processing text data, which is unstructured data, among the patent data; And technology to derive a technology topic from the preprocessed unstructured data through technology topic modeling, and apply the technology topic and numerical data previously extracted by the numerical data processing unit of the data preprocessing module to the ensemble model to generate a technology transfer prediction model. It characterized in that it comprises a; previous prediction model construction module.

The machine learning-based technology transfer prediction system according to the present invention has the effect of being able to objectively and easily evaluate the possibility of technology transfer to a population through the collected patent data.

1 is a block diagram of a system for generating a technology transfer prediction model based on machine learning according to the present invention.
2 is a flowchart of a method for predicting technology transfer based on machine learning according to the present invention.
3 is a diagram illustrating a process of finding a technology subject by the system for generating a technology transfer prediction model based on machine learning according to the present invention.
4 is a graphic model of a Latent Dirichlet Allocation (LDA) graphic model and an AdaBoost algorithm used in the present invention.
5 is a modeling process for a technology transfer prediction model in the present invention.
6 is a graph of the results of analyzing the optimal number of subjects.
7 is a graph of the accuracy and specificity of the model according to the present invention and other models.
8 is a graph of the sensitivity of the model according to the present invention and other models.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. The term and/or includes a combination of a plurality of related items or any of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it should be understood that it is directly connected to or may be connected to the other component, but other components may exist in the middle. something to do. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Unless otherwise defined, all terms used herein including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessive formal meaning unless explicitly defined in this application. Does not.

Throughout the specification and claims, when a certain part includes a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated.

Hereinafter, a machine learning-based technology transfer prediction model generation system and a generation method according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a machine learning-based technology transfer prediction system according to the present invention.

As shown in Figure 1, the machine learning-based technology transfer prediction system according to the present invention includes a data preprocessing module 300 composed of a data collection unit 100, a data classification unit 200, and a numerical data processing unit 310, and A technology transfer prediction model construction module 400 is included.

The data collection unit 100 accesses a World Intellectual Property Service (WIPS) server, a service provider in Korea, or a Keyprise server, a service provided by the Korean Intellectual Property Office, and provides patent data such as patent specifications required to generate a technology transfer prediction model. To collect.

The data classification unit 200 classifies the patent data collected by the data collection unit 100 into numerical data and text data.

The data preprocessor module 300 processes the text data, which is unstructured data, among the data classified by the data classification unit 200 into a form that can be analyzed through a preprocessing process of structuring the data through text mining.

The technology transfer prediction model construction module 400 receives text data processed by the data preprocessing module 300 as input and performs topic modeling through an ensemble model to derive a technology topic.

In addition, the technology transfer prediction model configuration module 400 performs topic modeling using text data processed by the data preprocessing module 300 to derive technology topic information and numerical data processing unit 310. Ensemble modeling based on the ensemble model is performed using the generated numerical data as input to generate a technology transfer prediction model.

As another embodiment, a method of generating a prediction model using a machine learning-based technology transfer prediction model generation system according to the present invention having the configuration as described above will be described.

First, as shown in FIG. 2, the data collection unit 100 performs a step of collecting patent data such as a patent specification from a Korean patent information service provider (S100).

More specifically, the patent data may be collected by WIPS (World Intellectual Property Service), a service provider in Korea that provides patent information, or Cypris, a service provided by the Korean Intellectual Property Office.

The data classification unit 200 performs a step of classifying the patent data collected by the data collection unit 100 into numerical data and text data (S200).

In addition, the data preprocessing module 300 includes a numerical data processing unit 310, and performs a preprocessing step of allowing text data, which is unstructured data, among the patent data to be analyzed (S300).

The technology transfer prediction model construction module 500 performs a step of deriving a technology topic by performing topic modeling through an ensemble model by inputting text data processed by the data preprocessing module 300 (S400), and deriving A step of generating a technology transfer prediction model by performing ensemble modeling based on an ensemble model by inputting the technology topic information and the numerical data generated by the numerical data processing unit 310 is performed (S500).

A detailed description of the steps S300 to S500 is as described below.

Since the data preprocessing module 300 includes unnecessary noise patents or duplicate patents that are not relevant in the collected data, it is preferable that the noise patent or duplicate patents are deleted.

The patent examiner classifies the patent using IPC, cooperative patent classification (CPC), or file index (FI) according to the contents of the patent specification so that the searcher can easily find the patent.

However, since the classification code does not contain detailed technical details, the ability to check technical data based on this is limited.

Therefore, in the present invention, the description of the target domain is first classified through subject modeling.

3 shows a process of finding a technology subject in the present invention.

In order to grasp the technical subject of the collected data, the text of the patent document obtained by merging the title and abstract was used as a corpus, a set of linguistic texts.

Under a given grammatical structure, word classes are usually written according to their position in predicates, objects, etc.

Among them, stop terms such as "the", "a", "by", "as", and "is" are necessary to compose general sentences with respect to commonly used terms, but are unnecessary in the present invention.

These words do not provide any special meaning in LDA, and when included they unnecessarily increase the complexity of the computation, so they must be removed appropriately for efficient information processing.

Therefore, the data preprocessing module 300 removes stop words in the preprocessing step.

In addition, the data preprocessing module 300 selects important words using a frequency-inverse document frequency (TF-IDF) because all words in the document are not important for later analysis.

The shape of the word is determined by the position of each word in the sentence. Therefore, if the word shapes are not the same, the machine learning-based technology transfer prediction model generation system according to the present invention cannot grasp whether some words have the same meaning or different meanings, and as a result, there is a possibility that data distortion may result. have. Therefore, the data preprocessing module 300 unifies the form of words having the same meaning.

In the present invention, morpheme analysis is used to unify the shape of a word.

In addition, the data preprocessing module 300 removes numbers, punctuation marks, and symbols that may distort the analysis process.

The data refined through the data preprocessing module 300 is used to fit the LDA (Latent Dirichlet Allocation) model.

To fit the LDA model, the Gibbs sampling method was adopted, and the K was increased from 2 to 10 to find the optimal K. Also, the number of repetitions is limited to 500, 1000 and 2000 so that the subject can be displayed well.

For reference, in an embodiment of the present invention, K and the number of iterations are limited, and as a process for deriving an optimal model, the number of iterations, K, etc. are variable according to data.

The Dirichlet parameter α is set to the estimated value from the document and the parameter β is set to 0.1. Table 1 shows the optimal parameters suitable for the LDA model found through iteration.

Component Candidates Inference Algorithm
The number of K
Gibbs sampling iteration
Parameter α,β Gibbs sampling
From 2 to 10
1000
A=T/50,β=0.1

Thematic model is one of the unmanaged methods. That is, it is a text mining technology that can identify the subject or subject of a document in a larger collection of collected documents. The LDA (Latent Dirichlet allocation), one of the most common subject modeling techniques, expresses the corpus based on the Bayesian model. It is a probabilistic model for and is considered a probabilistic extension of Latent Semantic Analysis (LSA).

As shown in Fig. 4, the basic idea of LDA is that each document has a subject, and the subject can be defined as a word distribution.

으로 표시된다.

은 Dirichlet 분포에 하이퍼-파라미터 α가 뒤따르는 다항식 분포이다.For example, when the second document m D _m, D _m is a document distribution contained in the latent topic z _{m, n} is

It is indicated as.

Is a polynomial distribution followed by the hyper-parameter α in the Dirichlet distribution.

Number of subjects k = 1,. . . , K can be statistically estimated, or the experimenter can determine a fixed value.

으로 표시되며, 이는 다 변수 분포가 Dirichlet 분포를 따르는 다항 분포이기도 하다. 확률 w_m,_n은 p (w_m,_n |z_m,_n, β)에 의해 결정된다.The distribution of words over k topics is

It is denoted by, which is also a polynomial distribution whose multivariate distribution follows the Dirichlet distribution. The probabilities w _m , _n are determined by p (w _m , _n |z _m , _n , β).

In Equation 1, the limit distribution of the document can be derived from the integration with respect to θ and the summation with respect to z as shown in Equation 2.

Finally, the corpus probability can be expressed as the product of the limit probability of each single document as shown in Equation 3 below.

The subject model described above is a useful method for identifying hidden subjects in a series of documents. As a result of the above-described advantages, research for analyzing technical documents using subject models has been actively conducted in recent years.

To implement the technology transfer prediction model, we use the AdaBoost algorithm, which is a typical ensemble model.

The ensemble method is an algorithm that improves accuracy by combining several weak learners, and is generally achieved through boosting and bagging.

The most important feature of the AdaBoost algorithm is that AdaBoost sequentially generates weak learners, whereas weak learners generated in the bag are generated in parallel.

The ensemble algorithm will be described with reference to the graphic model of the AdaBoost algorithm corresponding to the latter part of FIG. 4.

The main idea is to adjust the weight of the weak classification criteria for each distribution and finally combine these weak classification criteria to get one strong classification criteria.

In other words, it generates a hypothesis with high accuracy.

로 주어진다고 가정하면, 여기서

및 y_i는 클래스 또는 레이블을 나타낸다.Training example

Assuming that it is given by

And y _i represents a class or label.

The initial distribution D ₁ (i) is defined as a uniform distribution D ₁ (i) = 1/n.

The distribution D _t is updated with the weighted error ε _t to minimize the weak hypothesis h _{t in each round.}

The weight of the weak hypothesis (h _t ) according to the error ε _t is determined according to Equation 4, and the distribution (D _t+1 (i)) is updated by the weight through Equation 5 below.

As can be seen from Equation 5, if ε _t >0.5, α _t >0, and ε _t <0.5, then α _t >0.

That is, as the error ε _{t of the weak hypothesis (h t} ) decreases, the weight α _t increases. If the error ε _t is greater than 0.5, the performance of the weak separator is lower than that of random guesses. Therefore, the hypothesis for the weak classification criterion is not considered and proceeds to the next round.

이고, h_t(i)≠y_i이면 이때

이며, 여기서 Z_t는 정규화 계수를 나타낸다. 이것은

을 만족한다. 결과적으로 강한 분류기는 아래의 수학식 7과 같이 정의된다. _{If h t} (i) = y _i in Equation 6, then,

And if h _t (i)≠y _i then

Where Z _t represents the normalization coefficient. this is

Is satisfied. As a result, the strong classifier is defined as in Equation 7 below.

5 is a proposed model. Refers to variables such as patent references and non-patent references. The technical subject represents the technical description of the patent using the LDA. Among the variables in the data set, "transfer" indicates whether the patent has been transferred from the original holder to another holder.

In the patent search database for collecting patent data in the present invention, information on the applicant and the current assignee can be checked.

If the assignee changes, the patent right is deemed to have been transferred. In the present invention, "previous" is used as the _{output variable y i.}

Therefore, if a patent is transferred to another patent, it has a value of +1, otherwise it has a value of -1. The parameters used in the AdaBoost algorithm were estimated through iterations.

To verify the performance of the proposed model, we consider the accuracy, specificity and sensitivity measurements that can be used to evaluate the classification performance. The measurements are detailed in Table 2.

Accuracy represents the overall performance of the classifier. Consider true positive (TP) and true negative (TN) together.

However, excessive learning of noisy training data by the classifier can lead to excessive results.

Therefore, accuracy cannot measure the exact performance of the classifier in such a situation.

In order to overcome this problem, the present invention evaluates the performance of the proposed model by considering both the sensitivity considering only true positives and the specificity considering only true negatives.

For the experiment of the present invention, data on the technical fields of inference and machine learning were collected according to the above-described conditions.

I want to know the trend according to a specific technology, but there is no technical classification information in the patent data. Therefore, in the present invention, technical classification was performed using LDA as described above.

In addition, the result of selecting the optimal number of subjects is as shown in FIG. 6.

According to the data, the optimal number of subjects may change later, and Cao et al (2009)-“density-based method for adaptive LDA model selection” methodology was applied to determine the optimal number of inventions in the present invention.

Based on these results, the technical topics of the collected data were classified. The results are shown in Table 3.

The technical topics were defined using the top 10 keywords included in each topic.

As a result of the technology classification, the "natural language understanding" technology accounted for the largest share of about 22.4% in inference and machine learning. Next, "expert system" technology accounted for 21.4%, followed by signal processing, image processing, and artificial neural network technologies.

Table 4 shows the number of technology transfers for the above technology fields.

The transfer rate of the collected data is an average of 29%. As a result of confirming the rate of technology transfer according to technology classification using LDA, the technology transfer rate in Topic 3 (Natural Language Understanding) is 38%, which is relatively higher than that of other technology areas.

In order to generate the technology transfer prediction model proposed in the present invention, the result of the previous subject model and the quantitative index of the patent using the AdaBoost algorithm are merged.

To compare the performance of the proposed method, a performance comparison test was performed using K-NN (K-nearest neighbor classifier), SVM (Support Vector Machine), and Neural Network algorithms, which are representative classification algorithms. Performance measurements use the accuracy, sensitivity and specificity already mentioned. We also discuss models with and without technical topics. The experimental results are as shown in Tables 5 and 6.

In FIGS. 7 and 9, NT means a model that does not include a technology subject, and YT indicates a model that includes a technology subject.

In the present invention, a model with a technical subject and a model without a technical subject was compared.

As a result, the classification performance of the technology transfer prediction model including the technical information of the document is superior to that of the model not including the document information.

The sensitivity of a model that does not contain technical content is significantly lower than that of a model that includes technical information because over-fitting occurs in the model.

Therefore, it can be assumed that technical information is an important factor in forecasting technology transfer.

Next, a model was created using the proposed method and other models based on classifiers such as K-nearest neighbor classifier (K-NN), SVM (Support Vector Machine), and Neural Network.

K-NN has a simple structure but excellent performance. For this reason, it is used in many classification problems. Support vector machines and artificial neural networks are well known to have excellent classification performance and are applied in various fields.

It can be seen that the accuracy of the model is generally similar according to the result of comparison with the model presented in the present invention and the other classification criteria mentioned above.

However, in terms of sensitivity and specificity, there was a significant difference between the proposed model and the other models by indicating true positive and true negative, respectively.

In particular, the sensitivity and specificity of the other models compared in the present invention were lower than the sensitivity and specificity of the proposed model. This is due to excessive application of the model.

This result shows that the proposed model outperforms other models in the case of technology transfer prediction.

Therefore, it can be concluded that the proposed model based on the patent data in the present invention is suitable for technology transfer prediction.

The above description is merely illustrative of the technical idea of the present invention, and a person of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments implemented in the present invention are not intended to limit the technical idea of the present invention, but to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: data collection unit
200: data classification unit
300: data preprocessing module
310: Numerical data processing unit
400: Technology transfer prediction model configuration module

Claims (6)

In the machine learning-based technology transfer prediction model generation method by the machine learning-based technology transfer prediction model generation system,
(a) collecting patent data necessary for the data collection unit to generate a technology transfer prediction model;
(b) classifying the patent data into numerical data and text data by a data classification unit;
(c) a pre-processing step of allowing the data pre-processing module to analyze text data, which is unstructured data among the patent data; And
(d) The technology transfer prediction model configuration module receives numerical data and preprocessed text data, performs topic modeling to extract a technology topic from the text data through an ensemble model, and performs ensemble modeling with the technology topic and numerical data. And generating a technology transfer prediction model by doing so,
In the step (d), the technology transfer prediction model construction module

A method of generating a technology transfer prediction model based on machine learning, characterized by generating a technology transfer prediction model using an ensemble model.
H(x): refers to a strong hypothesis, and _{a collection of generated h t} (weak learners)
T: total number of _{h t}
α _t : the weight of the weak classifier
h _t : t-th weak learner
The method of claim 1,
In step (c) above
And extracting, as numerical data, transferred patents among valid patents from which unnecessary noise patents or duplicate patents have been deleted from the collected patent data by the data preprocessing module. How to create an old predictive model.
The method of claim 2,
Step (c) is
(c-1) removing stop words from sentences constituting the text data by the data preprocessing module;
(c-2) selecting, by the data preprocessing module, an important word from the text data using a frequency-inverse document frequency (TF-IDF) method;
(c-3) unifying, by the data preprocessing module, the shape of words constituting the text data through morpheme analysis; And
(c-4) removing, by the data preprocessing module, numbers, punctuation marks, and symbols that may distort the analysis process.
The method of claim 3,
In step (d) above
The technology transfer prediction model configuration module

A method of generating a technology transfer prediction model based on machine learning, characterized in that a technology topic is extracted from a document with a model.
α: Hyper-paramter following Dirichlet Distribution to determine the distribution of documents in LDA
β: hyper-paramter following Dirichlet Distribution to determine the distribution of words in LDA
θ _m : document
z _m , _n : hidden topic
w _m : Distribution to be included in latent topics z _m and _{n (polynomial distribution)}
n: order of words
N _m : The entire set of words contained in the m th document
w _m , _n : the nth word out of M words
delete A data collection unit that collects patent data necessary to generate a technology transfer prediction model;
A data classification unit for classifying the patent data into numerical data and text data;
A data pre-processing module for pre-processing text data, which is unstructured data, among the patent data; And
Technology for receiving the numerical data and preprocessed text data, performing topic modeling to extract a technology topic from the text data through an ensemble model, and performing ensemble modeling with the technology topic and numerical data to generate a technology transfer prediction model A previous prediction model construction module; Including,
The technology transfer prediction model configuration module

A machine learning-based technology transfer prediction model generation system, characterized in that it generates a technology transfer prediction model using an ensemble model.
H(x): means a strong hypothesis, and _{a collection of generated h t} (weak learners)
T: total number of _{h t}
α _t : the weight of the weak classifier
h _t : t-th weak learner

Images