KR20200014510A - Method for providing prediction service based on mahcine-learning and apparatus thereof - Google Patents

Abstract

Provided is a method for providing a prediction service based on machine learning. The method for providing a prediction service based on machine learning, performed in a computing device, according to an embodiment of the present invention comprises the steps of: receiving data from a user terminal; selecting an optimal model to perform prediction on the data from a plurality of prediction models by using a feedback learning model; obtaining a prediction result for the data by using the selected optimal model; and providing the obtained prediction result to the user terminal, wherein the feedback learning model is a machine learning model that learns user feedback information on satisfaction for the prediction result.

Description

METHOD FOR PROVIDING PREDICTION SERVICE BASED ON MAHCINE-LEARNING AND APPARATUS THEREOF}

The present invention relates to a method and apparatus for providing predictive service based on machine learning. More specifically, the present invention relates to a method for performing a method of selecting a best predictive model among a plurality of predictive models constructed through machine learning, and providing a predetermined predictive service using the selected predictive model.

Deep learning refers to a machine learning technique that uses deep neural network structure to discover and learn useful and representative features from learning data. Deep learning techniques are currently in the limelight in many fields by overcoming the performance limitations of traditional machine learning techniques such as decision trees, support vector machines, and the like.

However, since deep learning techniques require a large amount of data for learning and it is difficult to obtain such data in reality, there are very limited fields where deep learning techniques can be applied.

If the training data is not well prepared, traditional machine learning techniques can perform better than deep learning techniques. For example, as shown in FIG. 1, if the amount of data is sufficient, the deep learning technique 1 may outperform the performance of the traditional machine learning technique 3, and vice versa. ) Can be rather effective.

However, when building a predictive model using traditional machine learning techniques, the performance varies greatly depending on the type of data, the method of extracting features, and the type of machine learning algorithms. It is a problem.

In order to solve the above problems, conventionally, various types of prediction models have been built and cross validation has been mainly used as a method of determining the most accurate model as an optimal model. However, since the verification data used for cross-validation is only a part of the original training data, the optimal model determined by the conventional method rarely shows good performance in a real environment.

Therefore, when the machine learning technique is applied to a specific field where there is not enough training data, a method for accurately determining an optimal model of the corresponding field is required.

Korean Patent Publication No. 10-2018-0049277 (published May 11, 2018)

Another technical problem to be solved by the present invention is to provide a method for selecting an optimal prediction model among a plurality of prediction models constructed through machine learning, and an apparatus for performing the method.

The technical problem to be solved by the present invention relates to a method for providing a predetermined prediction service using a plurality of prediction models constructed through machine learning, and an apparatus for performing the method.

Another technical problem to be solved by the present invention is to provide a method and an apparatus for performing the method that can ensure the accuracy of the prediction result by using a plurality of prediction models, even in a field or environment where there is not enough learning data. will be.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, a machine learning based prediction service providing method according to an embodiment of the present invention, in the method of providing a machine learning based prediction service performed in a computing device, receiving data from a user terminal Selecting an optimal model to predict the data from among a plurality of prediction models using a feedback learning model, obtaining a prediction result for the data using the selected optimal model, and obtaining the obtained prediction Providing a result to the user terminal may include. In this case, the feedback learning model may be a machine learning model that learns user feedback information on satisfaction of a prediction result.

In an embodiment, the feedback learning model may be a model built through deep learning, and each of the plurality of prediction models may be a model built through machine learning other than deep learning.

In an embodiment, the user feedback information may include a satisfaction score for a prediction result, and the feedback learning model may be a model constructed through reinforcement learning on the satisfaction score.

In one embodiment, selecting the optimal model may include selecting the optimal model in response to determining that the learning degree of the feedback learning model is greater than or equal to a threshold.

The method may further include receiving first user feedback information on the obtained prediction result from the user terminal and updating the feedback learning model by using the first user feedback information.

In an embodiment, in response to determining that user feedback information about the obtained prediction result is not provided, predicting a satisfaction score for the obtained prediction result and using the predicted satisfaction score, the feedback learning model It may further comprise the step of updating.

In an embodiment, accumulating a plurality of user feedback information provided from the user terminal, classifying the plurality of user feedback information into a first type and a second type, and assigning the feedback learning model to the first type. The method may further include subdividing into a first feedback learning model predicting an optimal model and a second feedback learning model predicting an optimal model with respect to the second type.

In one embodiment, each of the plurality of prediction models is a first prediction model built through a machine learning technique other than deep learning, accumulating a plurality of prediction results provided to the user terminal and deepen the plurality of prediction results Learning through a learning technique may further include constructing a second prediction model.

An apparatus for providing prediction service according to another embodiment of the present invention for solving the above technical problem includes a processor and a memory for storing at least one program executed by the processor, wherein the at least one program is a user. Operation of receiving data from a terminal, selecting an optimal model to predict the data from among a plurality of prediction models by using a feedback learning model, and obtaining a prediction result of the data using the selected optimal model And instructions for performing an operation of providing the user terminal with the obtained prediction result. In this case, the feedback learning model may be a machine learning model that learns user feedback information on satisfaction of a prediction result.

A computer program according to another embodiment of the present invention for solving the above technical problem is combined with a computing device to receive data from a user terminal, by using a feedback learning model, and among the plurality of prediction models. Selecting an optimal model to perform a prediction on, obtaining a prediction result for the data using the selected optimal model, and providing the obtained prediction result to the user terminal. It may be stored in a readable recording medium. In this case, the feedback learning model may be a machine learning model that learns user feedback information on satisfaction of a prediction result.

1 is a diagram illustrating a comparison between a deep learning technique and a conventional machine learning technique.
2 is a block diagram showing a prediction service providing system according to an embodiment of the present invention.
3 is a block diagram illustrating an apparatus for providing prediction service according to an embodiment of the present invention.
4 is a detailed block diagram illustrating a selection unit among components of a prediction service providing apparatus according to an embodiment of the present invention.
5 is an exemplary diagram for describing a structure of a deep learning model that may be referred to in some embodiments of the present disclosure.
6 is a detailed block diagram illustrating a result generator among components of a prediction service providing apparatus according to an exemplary embodiment.
7 is a hardware block diagram showing an apparatus for providing prediction service according to an embodiment of the present invention.
8 and 9 are flowcharts illustrating a method for providing predictive service based on machine learning according to an embodiment of the present invention.
10 and 11 are exemplary diagrams for describing a method for constructing a prediction model according to some embodiments of the present invention.
12 is an exemplary diagram for describing a method of selecting an optimal prediction model using a feedback learning model based on reinforcement learning according to an embodiment of the present invention.
13 is a flowchart illustrating a method of updating a feedback learning model according to an embodiment of the present invention.
14 and 15 are exemplary diagrams for describing a feedback learning model segmentation method according to an embodiment of the present invention.
16 and 17 are diagrams for describing a method of further utilizing a deep learning based prediction model according to an embodiment of the present invention.
18 to 20 are diagrams for explaining the configuration and operation of the intelligent agent-based query / response system according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be embodied in various different forms, and the present embodiments merely make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the present invention, if it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used as meanings that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are specifically defined clearly. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to that other component, but there may be another configuration between each component. It is to be understood that the elements may be "connected", "coupled" or "connected".

As used herein, “comprises” and / or “comprising” refers to a component, step, operation and / or element that is mentioned in the presence of one or more other components, steps, operations and / or elements. Or does not exclude additions.

Prior to the description herein, some terms used herein will be clarified.

In the present specification, an instruction is a series of instructions grouped based on a function and refers to a component of a computer program and executed by a processor.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram of a prediction service providing system according to an embodiment of the present invention.

Referring to FIG. 2, the prediction service providing system may be configured to include a prediction service providing server 100 and the like. However, this is only a preferred embodiment for achieving the object of the present invention, of course, some components may be added or deleted as necessary. In addition, each component of the prediction service providing system illustrated in FIG. 2 illustrates functionally divided functional elements, and it is noted that a plurality of components may be implemented in a form in which they are integrated with each other in an actual physical environment.

In the prediction service providing system, the prediction service providing server 100 is a computing device that provides a machine learning based prediction service to a plurality of users. Here, the computing device may be a laptop, a desktop, a laptop, and the like, but is not limited thereto and may include all kinds of devices equipped with a computing function and a communication function. However, when providing a prediction service to a plurality of users, the prediction service providing server 100 may be implemented as a high performance server computing device.

The prediction service means any service provided based on a machine learning model. For example, the prediction service may provide any kind of prediction service that can be provided through a machine learning model, such as a service for predicting a class of an object included in an image, a service for predicting a topic or emotion included in a user's spoken sentence. It may include.

According to an exemplary embodiment of the present invention, the prediction service providing server 100 receives data from the user terminal 30 and selects an optimal model selected from a plurality of prediction models 51-1, 51-2,?, And 51-n. Using may provide a prediction result for the input data. Here, the prediction service providing server 100 may select an optimal model using a feedback learning model. The feedback learning model is a machine learning model that learns user feedback about satisfaction of prediction results. According to the present exemplary embodiment, an optimal model for performing the actual prediction is selected based on the user's satisfaction, and thus user satisfaction with the prediction service may be improved. Detailed description of the embodiment will be described in more detail with reference to the drawings below with reference to FIG. 3.

In the prediction service providing system, the plurality of prediction models 51-1, 51-2,..., 51-n are machine learning models that provide prediction results with respect to input data. Each of the plurality of prediction models 51-1, 51-2,..., 51-n may be constructed through a traditional machine learning technique, which will be described later with reference to FIGS. 10 and 11.

The plurality of prediction models 51-1, 51-2,..., 51-n may be built in a computing device that is physically independent of the prediction service providing server 100, or may be built in the prediction service providing server 100. This may vary depending on the embodiment. Hereinafter, a description will be given on the assumption that a plurality of prediction models 51-1, 51-2,..., 51-n are built in the prediction service providing server 100.

In the prediction service providing system, the user terminal 30 is a terminal of a user receiving a prediction service. The user may input data through the terminal 30 and receive a prediction result for the input data. In this case, the prediction service providing server 100 may provide a predetermined user interface to induce user feedback, and the user may provide user feedback information on the satisfaction of the prediction result through the user interface. The prediction service providing server 100 may gradually learn a feedback learning model using the user feedback information.

Components of the prediction service providing system may communicate via a network. The network may be any type of wired / wireless network such as a local area network (LAN), a wide area network (WAN), a mobile radio communication network, a wireless broadband Internet (Wibro), or the like. Can be implemented.

In the above-described prediction service providing system, the device 100 is called a "prediction service providing server" in consideration of providing a prediction service to a plurality of users, but hereinafter referred to as "prediction service providing device".

Up to now, a prediction service providing system according to an embodiment of the present invention has been described with reference to FIG. 2. Hereinafter, the configuration and operation of the prediction service providing apparatus 100 according to an embodiment of the present invention will be described with reference to FIGS. 3 to 7. In the following description, the prediction service providing apparatus 100 will be abbreviated as the prediction apparatus 100 for convenience of description.

3 is a block diagram illustrating a prediction apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 3, the prediction apparatus 100 may be configured to include an interface 110, a selector 130, and a result generator 150. However, FIG. 3 shows only the components related to the embodiment of the present invention. Therefore, it will be appreciated by those skilled in the art that the present invention may further include other general purpose components in addition to the components shown in FIG. 3. In addition, each component of the prediction apparatus 100 illustrated in FIG. 3 represents functionally divided functional elements, and it is noted that a plurality of components may be implemented in a form in which they are integrated with each other in an actual physical environment. In addition, the prediction apparatus 100 may be implemented in a form in which some of the components shown in FIG. 3 are omitted.

Looking at each component, the interface unit 110 provides an interface with another device (e.g. user terminal 30). For example, the interface unit 110 may receive data from the user terminal 30, provide a prediction result with respect to the input data, and receive feedback about the prediction result from the user terminal 30.

Next, the selector 130 selects an optimal prediction model for the input data from among the plurality of prediction models 155-1, 155-2,?, And 155-n. To this end, the selector 130 may be configured to include a model selector 131, a feedback learner 133, and a feedback learning model 135 as shown in FIG. 4.

The model selector 131 selects an optimal model from among the plurality of prediction models 155-1, 155-2,?, And 155-n based on the feedback learning model 135. However, when the learning degree of the feedback learning model 135 is low, the model selector 131 may select a model to perform prediction regardless of the feedback learning model 135. For example, the model selector 131 may randomly select some models from among the plurality of prediction models 155-1, 155-2,?, And 155-n, and predict the models 155-1, 155-2,? , 155-n) may be selected.

For a detailed description of the operation of the model selector 131, refer to the description of FIG. 8.

Next, the feedback learner 133 learns user feedback information provided from the user terminal to build a feedback learning model 135. The feedback learning model 135 may be understood as a machine learning model that predicts an optimal model based on user satisfaction.

In an embodiment, the feedback learner 133 may build the feedback learning model 135 through a deep learning technique. In this case, the feedback learning model 135 may be configured as a deep neural network structure as shown in FIG. 5. Since the feedback learning model 135 according to the present embodiment discovers and learns key features used for prediction in the training data by itself, the optimal learning model can be predicted with high accuracy when sufficiently learned.

In an embodiment, the feedback learner 133 may build the feedback learning model 135 through reinforcement learning. In this case, the feedback learner 133 may perform reinforcement learning by using the satisfaction score included in the user feedback information as a reward.

For a detailed description of the operation of the feedback learner 133, refer to the description of FIGS. 9 and 13 to 15.

Referring back to FIG. 3, the result generator 150 constructs a plurality of predictive models 155-1, 155-2,? 155-n through machine learning, and selects the predictive model selected by the selector 130. Generate a prediction result for the input data using. To this end, the result generator 150 may generate a model learner 151, a predictor 153, and a plurality of predictive models 155-1, 155-2,?, And 155-n as shown in FIG. 6. It can be configured to include.

The model learner 151 builds a plurality of predictive models 155-1, 155-2,?, And 155-n by machine learning the acquired training data set. For example, the model learner 151 may build a plurality of predictive models 155-1, 155-2,?, 155-n through traditional machine learning techniques.

For more detailed description of the operation of the model learner 151, refer to the description of FIGS. 8, 10, and 11.

Next, the prediction unit 153 calculates the prediction result for the input data using the prediction model selected by the selection unit 130 from among the plurality of prediction models 155-1, 155-2,?, And 155-n. Create

For a detailed description of the operation of the predictor 153, refer to the description of FIG. 8.

Each component of FIG. 3 may refer to software or hardware such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). However, the components are not limited to software or hardware, and may be configured to be in an addressable storage medium, or may be configured to execute one or more processors. The functions provided in the above components may be implemented by more detailed components, or may be implemented as one component that performs a specific function by combining a plurality of components.

7 is a hardware diagram illustrating a prediction device 100 according to an embodiment of the present invention.

Referring to FIG. 7, the prediction apparatus 100 may include one or more processors 101, a bus 105, a network interface 107, and a memory 103 that loads a computer program executed by the processor 101. And a storage 109 for storing the computer program 109a. However, FIG. 7 shows only the components related to the embodiment of the present invention. Therefore, it will be appreciated by those skilled in the art that other general purpose components may be further included in addition to the components illustrated in FIG. 7.

The processor 101 controls the overall operation of each component of the prediction device 100. The processor 101 is configured to include a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphics processing unit (GPU), or any type of processor well known in the art. Can be. In addition, the processor 101 may perform an operation on at least one application or program for executing a method according to embodiments of the present invention. The prediction device 100 may have one or more processors.

The memory 103 stores various data, commands and / or information. Memory 103 may load one or more programs 109a from storage 109 to execute methods in accordance with some embodiments of the present invention. RAM is illustrated as an example of the memory 103 in FIG.

When the program 109a is loaded into the memory 103, the modules shown in FIG. 3 on the memory 103 may be implemented in the form of logic.

The bus 105 provides a communication function between components of the prediction device 100. The bus 105 may be implemented as various types of buses such as an address bus, a data bus, and a control bus.

The network interface 107 supports wired and wireless Internet communication of the prediction device 100. In addition, the network interface 107 may support various communication methods other than Internet communication. To this end, the network interface 107 may comprise a communication module well known in the art.

The network interface 107 may receive various data from the user terminal or transmit various data to the user terminal.

The storage 109 may non-temporarily store the one or more programs 109a. The storage 109 is well-known in the technical field to which a nonvolatile memory, hard disk, removable disk, or the like to which the present invention belongs, such as Read Only Memory (ROM), Eraseable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, and the like. And any known type of computer readable recording medium.

The computer program 109a may include instructions that, when loaded into the memory 103, cause the processor 101 to execute a method of providing machine learning based prediction services in accordance with some embodiments of the present invention.

For example, the computer program 109a may be configured to select an optimal model to perform prediction on the data from among a plurality of prediction models, using an operation of receiving data from a user terminal, a feedback learning model, and the selected optimal model. It may include instructions for performing the operation of obtaining a prediction result for the data and providing the obtained prediction result to the user terminal by using.

So far, the configuration and operation of the prediction apparatus 100 according to the embodiment of the present invention have been described with reference to FIGS. 3 to 7. Hereinafter, a machine learning based prediction service providing method according to some embodiments of the present invention will be described in detail with reference to FIGS. 8 to 17.

Each step of the method for providing predictive service based on machine learning according to an embodiment of the present invention may be performed by a computing device including a processor. For example, the computing device may be the prediction device 100. However, for convenience of description, the description of the operation subject of each step included in the prediction service providing method may be omitted. In addition, each step of the method of providing prediction service may be implemented by instructions of a computer program executed by a processor.

8 and 9 are flowcharts illustrating a method for providing predictive service based on machine learning according to an embodiment of the present invention. However, this is only a preferred embodiment for achieving the object of the present invention, of course, some steps may be added or deleted as necessary.

Referring to FIG. 8, the method of providing a prediction service starts at step S100 in which the prediction apparatus 100 constructs a plurality of prediction models. For example, the model learner 151 may construct a plurality of predictive models by performing machine learning on a given training data set.

ve bayes), 결정 트리(decision tree), 서포트 벡터 머신(support vector machine) 등이 될 수 있다. 이와 같은 학습 기법들은 딥러닝 기법보다 학습 파라미터(parameter)의 개수가 적고, 딥러닝 기법과는 달리 미리 정의된 특징(feature)을 학습하기 때문에, 학습 데이터셋이 많지 않더라도 효과적으로 예측 모델을 구축할 수 있다. 본 실시예에 따르면, 학습 데이터셋이 충분히 준비되지 않은 분야 또는 환경에서도 용이하게 예측 서비스가 제공될 수 있다는 장점이 있다.According to an embodiment of the present invention, the plurality of prediction models may be models built through traditional machine learning techniques. An example of a traditional machine learning technique is Naive Bayes (na

ve bayes, decision trees, support vector machines, and the like. These learning techniques have fewer learning parameters than deep learning techniques, and unlike deep learning techniques, they learn a predefined feature, which makes it possible to effectively build predictive models even if there are not many training datasets. have. According to the present embodiment, there is an advantage that the prediction service can be easily provided even in a field or environment in which the training data set is not sufficiently prepared.

In addition, according to an embodiment of the present invention, at least some of factors affecting performance, such as a kind of feature, a value of a hyper-parameter, and a kind of machine learning technique, may be set differently in each of the plurality of prediction models Can be. This is because the performance of the predictive model constructed by the traditional machine learning technique can vary greatly depending on the type of data, the type of the feature, and so on, in order to build various types of models in advance and select the optimal model therein. .

For example, as shown in FIG. 10, a plurality of prediction models 13-1, 13-2,?, And 13-n can be constructed by training the training dataset 11 with different machine learning techniques. have.

For example, as illustrated in FIG. 11, a plurality of prediction models 23-1, 23-2,?, 23-n may be constructed by training the training dataset 11 while changing hyperparameter values. It may be.

As another example, a plurality of prediction models may be constructed by extracting a plurality of features from a training dataset, and learning at least some of the plurality of features.

Referring back to FIG. 8, in step S110, the prediction apparatus 100 receives data from a user terminal. The data is input data used to generate a prediction result through the prediction model. For example, the prediction device 100 may receive data from the user terminal through the interface unit 110.

In step S120, the prediction apparatus 100 determines whether the feedback learning model has been sufficiently learned. For example, the model selector 131 may determine whether the learning degree of the feedback learning model is greater than or equal to a threshold. The degree of learning of the model may be determined based on factors such as performance of the model (e.g. accuracy, error, etc.), the number of learned user feedbacks, the number of learning, and the like. However, the technical scope of the present invention is not limited thereto.

In response to determining that the feedback learning model has not been sufficiently trained, step S130 may be performed. In operation S130, the prediction apparatus 100 obtains a prediction result for data input from at least some models of the plurality of prediction models. For example, the prediction unit 153 may obtain a prediction result for data input from the at least some models. The manner of selecting the at least some models may vary depending on the embodiment.

In one embodiment, the at least some models may be randomly selected.

In one embodiment, the at least some models may be selected based on evaluation results according to cross validation.

In one embodiment, the at least some models may be selected by a combination of the foregoing embodiments. For example, some models may be selected randomly, and some models may be selected based on the evaluation result.

In one embodiment, the at least some models may be a whole plurality of prediction models.

The at least some models may be one prediction model or two or more prediction models. When two or more prediction models are selected, the prediction apparatus 100 may obtain the final prediction result by combining the prediction results of the two or more prediction models. For example, the prediction apparatus 100 may apply an ensemble technique that is well known in the art, such as a voting technique, to obtain a final prediction result.

On the other hand, in response to determining that the feedback learning model has been sufficiently learned, step S140 may be performed. In operation S140, the prediction apparatus 100 selects an optimal model among a plurality of prediction models by using a feedback learning model. For example, the model selector 131 may select the optimal model using a feedback learning model.

As described above, the feedback learning model is a machine learning model that learns user feedback information about a prediction result. The method of building the feedback learning model and the method of selecting an optimal model through the feedback learning model may vary according to embodiments.

In one embodiment, the feedback learning model may be built through deep learning techniques. Specifically, the prediction apparatus 100, the feedback learning unit 133 is precisely composed of the data input from the user terminal, the prediction model for providing a prediction result for the input data and the user feedback information for the prediction result. The feedback learning model can be constructed by learning a training dataset through deep learning techniques. Unlike the training data set of the predictive model, the user feedback information is information that can be easily obtained by a simple click or simple selection of the user, and thus it is possible to secure a large data set within a short period of time. Also, if a large dataset is prepared, the performance of deep learning techniques can outperform traditional machine learning techniques. Therefore, when the feedback learning model is constructed through the deep learning technique, the optimal model can be predicted with high accuracy. According to the present embodiment, an optimal model based on user satisfaction may be accurately determined through a feedback learning model. Therefore, the accuracy of the prediction result can be improved, and the user's service satisfaction can be improved.

In one embodiment, the feedback learning model may be built through reinforcement learning techniques. In detail, the prediction apparatus 100 may perform reinforcement learning using the satisfaction score included in the user feedback information as a reward. The algorithm used for the reinforcement learning may be any algorithm such as Epsilon-Greedy or Upper Confidence Bound (UCB). In the present embodiment, the prediction apparatus 100 may select an optimal model in various ways. For example, in response to the determination that a predetermined condition (eg, a probability value set in epsilon) is satisfied, the prediction apparatus 100 selects a model having an expected reward greater than or equal to a threshold among the plurality of prediction models as an optimal model. (Use Procedure) In response to the determination that the predetermined condition is not satisfied, the optimal model can be randomly selected (exploration procedure). In this case, as shown in FIG. 12, as the feedback learning model is trained, the rate at which the use procedure 33 is performed may gradually increase, and the rate at which the exploration procedure 31 may be gradually decreased. According to the present embodiment, the user feedback information on the prediction model can be secured evenly through the exploration procedure, so that the learning of the feedback learning model can be prevented from being biased.

For reference, the optimal model selected in step S140 may be one or two or more. If more than one prediction model is selected as the optimal model, the final prediction result may be provided through an ensemble technique.

Referring back to FIG. 8, in step S150, the prediction apparatus 100 obtains a prediction result for data input from the optimal model. For example, the prediction unit 153 may extract a feature from the input data, apply the extracted feature to the best model, and obtain an output value of the best model as the prediction result.

In operation S160, the prediction apparatus 100 provides the prediction result to the user terminal. For example, the prediction device 100 may provide the prediction result to the user terminal through the interface unit 110. A description of the method will be continued with reference to FIG. 9.

In operation S170, the prediction apparatus 100 determines whether user feedback information on the prediction result is received. For example, the prediction apparatus 100 may determine whether to receive user feedback information based on a predetermined waiting time.

In response to determining that the user feedback information has not been received, step S180 may be further performed. In operation S180, the prediction apparatus 100 may predict a satisfaction score with respect to the prediction result. The specific method of predicting the satisfaction score may vary depending on the embodiment.

In one embodiment, in response to determining that user feedback information has not been received, the prediction device 100 may predict the satisfaction score to a relatively low value (e.g. a value below intermediate). Typically, when the user's satisfaction is low, it is because there is a high probability that feedback is not provided.

In one embodiment, the prediction apparatus 100 may predict the satisfaction score based on the feedback providing pattern of the user. The feedback providing pattern may include various types of information such as a recent feedback providing frequency, a total feedback providing frequency, a feedback providing probability, and the like. For example, when the corresponding user is a user with a high frequency of providing feedback, the prediction apparatus 100 may predict the satisfaction score to a relatively low value. On the contrary, if the user is a user with a low frequency of providing the original feedback, the prediction apparatus 100 may predict the satisfaction score to a relatively low value.

In one embodiment, a user's satisfaction score may be predicted by a combination of several embodiments described above.

On the other hand, in response to the determination that the user feedback has been received, step S190 may be immediately performed. In operation S190, the prediction apparatus 100 updates the feedback learning model by learning user feedback information. That is, whenever the user feedback information about the satisfaction of the prediction result is provided, the feedback learning model may be continuously updated.

Steps S170 to S190 may be performed by the feedback learner 133, for example.

Meanwhile, according to the embodiment of the present invention, the prediction apparatus 100 may precisely perform the preprocessing process on the user feedback information before updating the feedback learning model. Hereinafter, the pretreatment process will be described with reference to FIG. 13.

Referring to FIG. 13, in step S191, the prediction apparatus 100 adjusts a satisfaction score of user feedback information. The specific method of adjusting the satisfaction score may vary depending on the embodiment.

In one embodiment, the prediction apparatus 100 may adjust the current satisfaction score based on the average satisfaction score of the user. In this case, the average satisfaction score may be determined based on past feedback information of the corresponding user. For example, if the average satisfaction score of the user is high but the current satisfaction score is low, the prediction apparatus 100 may adjust the current satisfaction score to a lower value. If a user with a high average satisfaction score gives a low score, it is because the satisfaction with the prediction result is very low. On the contrary, when the average satisfaction score of the corresponding user is low but the current satisfaction score is high, the prediction apparatus 100 may adjust the current satisfaction score to a higher value. For another example, when the average satisfaction score of the corresponding user is high and the current satisfaction score is also high, the prediction apparatus 100 may adjust the current satisfaction score to a lower value. According to the present embodiment, since the satisfaction score may be prevented from being distorted by the user's disposition, a good learning data set may be secured, and thus the performance of the feedback learning model may be improved.

In an embodiment, the prediction apparatus 100 may adjust the current satisfaction score based on the feedback providing pattern of the user. For example, when a user with a low frequency of providing feedback provides feedback, the prediction apparatus 100 may adjust a higher value if the current satisfaction score is high, and adjust a lower value if the current satisfaction score is low. This is because the user feedback provided intermittently is likely to reflect the true intention of the user.

In one embodiment, the satisfaction score may be adjusted by a combination of several embodiments described above.

In operation S193, the prediction apparatus 100 may update the feedback learning model using the adjusted satisfaction score.

So far, the overall contents of the method for providing the prediction service based on the machine learning according to the embodiment of the present invention have been described with reference to FIGS. 8 to 13. Hereinafter, some embodiments of the present invention for further improving the accuracy of the prediction result will be described with reference to FIGS. 14 to 17.

14 and 15 are exemplary diagrams for describing a feedback learning model segmentation method according to an embodiment of the present invention. The feedback learning model refinement method may be performed by the feedback learner 133, for example.

According to an embodiment of the present invention, the prediction apparatus 100 may classify user feedback information accumulated as a prediction service is provided by type, and build a detailed feedback learning model for each type by learning the classified user feedback information. have.

For example, as shown in FIG. 15, the first feedback learning model 53-1 is constructed by learning the user feedback information 51-1 for the first type, and the user feedback information for the second type. The second feedback learning model 53-2 may be constructed by learning 51-2. In this case, the first feedback learning model 53-1 is a model for predicting an optimal model for the first type, and the second feedback learning model 53-2 is a model for predicting an optimal model for the second type. .

The type may vary from type of user, type of data, type determined by a specific context, and the like. When the feedback learning model is subdivided for each user type, a prediction model suitable for the user may be determined as an optimal model by the feedback learning model corresponding to the user type.

According to an embodiment of the present invention, as shown in FIG. 14, the prediction apparatus 100 may determine whether to refine the feedback learning model based on the average 41 and the variance 43 of the number of user feedbacks by type. have. For example, the prediction apparatus 100 may refine the feedback learning model in response to determining that the average 41 of the number of user feedbacks per type is greater than or equal to the threshold and the variance 43 is less than the threshold. Since the average 41 is greater than or equal to the threshold and the variance 43 is less than the threshold, a plurality of user feedback information is accumulated evenly by type, and thus the prediction apparatus 100 may subdivide the feedback learning model according to such a condition. .

So far, the feedback learning model segmentation method according to an embodiment of the present invention has been described with reference to FIGS. 14 and 15. According to the above-described method, the optimal prediction model will be selected for each type, and thus the accuracy of the prediction result and the user satisfaction with the prediction service can be further improved.

On the other hand, if a process of receiving data from a plurality of users and providing a prediction result is repeated, a large data set composed of the data and the prediction result may be secured. According to an embodiment of the present invention, a deep learning based prediction model may be constructed by using the data set as the training data. Hereinafter, this embodiment will be described with reference to FIGS. 16 and 17. In this embodiment, it is assumed that a plurality of prediction models are constructed through traditional machine learning techniques.

First, referring to FIG. 16, in operation S200, the prediction apparatus 100 accumulates a data set composed of input data and a prediction result in a predetermined storage.

In operation S220, the prediction apparatus 100 builds a new prediction model (hereinafter, referred to as a “deep learning model”) by learning a data set accumulated by the deep learning technique. For example, the model learner 151 may build the deep learning model by learning the training data set and the accumulated data set used to build a plurality of prediction models.

According to an embodiment of the present invention, the prediction apparatus 100 may filter the data set whose satisfaction score for the prediction result is greater than or equal to the threshold among the accumulated data sets, and build the deep learning model by learning the filtered data set. . According to the present embodiment, since the deep learning model is constructed by learning a high quality dataset with high reliability, the performance of the deep learning model may be improved.

In operations S240 and S260, the prediction apparatus 100 provides a prediction service using a plurality of prediction models and a deep learning model. For example, the prediction unit 153 may generate a prediction result by utilizing the optimal model selected from the plurality of prediction models together with the deep learning model. Of course, if the deep learning model is sufficiently learned, the prediction unit 153 may generate the prediction result using only the deep learning model. The method of using a plurality of prediction models (hereinafter, referred to as “existing models”) and deep learning models may vary according to embodiments.

In an embodiment, as shown in FIG. 17, the prediction apparatus 100 may utilize two types of models in a manner of gradually increasing the utilization rate of the deep learning model and gradually decreasing the utilization rate of the existing model. have. This is because the learning maturity of the deep learning model is improved over time, and the performance of the deep learning model will exceed that of the existing model. That is, in this embodiment, the utilization cost of the deep learning model may be updated based on the learning degree of the deep learning model.

In an embodiment, the utilization rate of the deep learning model may be updated based on user feedback information on a prediction result of the deep learning model. For example, when the satisfaction level of the deep learning model is low, the utilization rate of the deep learning model may be decreased, and the utilization rate of the existing model may be increased.

For reference, the utilization ratio means the degree to which the model is utilized to generate the prediction result. For example, if the utilization ratio between the deep learning model and the existing model is 1: 1, the existing model may operate to generate one prediction result whenever the deep learning model generates one prediction result. Of course, when the ensemble technique is applied, the utilization ratio may be applied even when generating one prediction result. For example, when the utilization ratio between the deep learning model and the existing model is 2: 1, the final prediction result may be generated by considering the prediction result of the deep learning model as two tables and the prediction result of the existing model as one table. .

Up to now, a method of providing a prediction service by further utilizing a deep learning model according to an embodiment of the present invention has been described with reference to FIGS. 15 and 17. According to the above-described method, as the deep learning model is built, an effect of evolving an existing prediction service into a higher quality prediction service may be achieved. If the training dataset is well prepared, the performance of deep learning techniques will outperform traditional machine learning techniques.

So far, a method of providing a predictive service based on machine learning according to an embodiment of the present invention has been described with reference to FIGS. 8 to 17. According to the above-described method, various types of prediction models may be constructed through traditional machine learning techniques, and an optimal prediction model may be selected through a feedback learning model obtained by machine learning user feedback information. Accordingly, the accuracy of the prediction result and the user's satisfaction with the prediction service may be improved. In addition, when a prediction result is accumulated, a deep learning model may be built and a more accurate prediction service may be provided through the deep learning model. Accordingly, the user's satisfaction with the prediction service can be improved.

Lastly, in order to provide a more convenient understanding, one application of the present invention will be described with reference to FIGS. 18 to 20. In particular, FIGS. 18 to 20 illustrate, by way of example, that the technical idea of the present invention is utilized in an intelligent conversation agent based question / answer system. The question / answer system may be an example of the prediction service providing system illustrated in FIG. 2.

18 illustrates a process of building a plurality of classification models. The plurality of classification models 69 correspond to the aforementioned prediction model. In this application, it is assumed that the plurality of classification models 69 are machine learning models that predict intention in the user's query sentence. Hereinafter, with reference to FIG. 18, it demonstrates in detail.

The plurality of classification models 69 may be constructed through the feature extraction process 61 and the learning process 65. The feature extraction process 61 is preceded by the traditional machine learning technique, rather than the deep learning technique, as described above.

Feature extraction process 61 may include tokenization 62, feature selection 63, and feature generation process 64.

Tokenization process 62 is a process of extracting a plurality of features by tagging parts of speech in a natural language sentence through a morpheme analyzer or by applying N-gram techniques to bind N tokens of a specific unit (eg word or syllable). to be.

The feature selection process 63 is performed based on metrics such as Term Frequency (TF), Term Frequency-Inverse Document Frequency (TF-IDF), Information Gain (IG), Mutual Information (MI), and chi-square. It is a process of selecting features to learn from among the plurality of features. Since the feature selection technique is well known in the art, a detailed description thereof will be omitted.

The feature generation process 64 is a process of generating a feature value matrix such as a document-feature matrix by combining the selected features. Since the document characteristic matrix is a well-known concept in the art, a description thereof will be omitted.

Next, in the learning process 65, a plurality of classification models 69 may be constructed by learning the feature value matrix through different machine learning techniques 66, 67, and 68.

Next, a process of providing a response service to a user query will be described with reference to FIG. 19. In particular, FIG. 19 illustrates a response service providing process when the feedback learning model 79 is not sufficiently learned.

When the query sentence is received from the user terminals 70 and 71, the feature extraction process 73 for the query sentence is performed in the same manner as the learning process.

In the classification process 74, the extracted features (eg document characteristic matrix) are input to the classification models 75-1 and 75-2, and the classification models 75-1 and 75-2 are users included in the query sentence. To predict the intention of. At this time, the classification models 75-1 and 75-2 may predict user intention according to an ensemble technique such as voting.

In the response process 76, a response sentence is generated and provided based on the predicted user intention.

When the user terminals 70 and 71 receiving the response provide satisfaction with the response as the user feedback 77, in the feedback learning process 78, the feedback learning model 79 may learn the user feedback 77. . The feedback learning model 79 may learn a dataset including a prediction model that provides a query sentence, satisfaction, and a response through a deep learning technique. In this case, in order to convert the query sentence into an input value format of the feedback learning model 79, a preprocessing process such as tokenization and word embedding may be performed.

Next, a process of providing a response service using the feedback learning model 89 will be described with reference to FIG. 20.

When a query sentence is received from the user terminals 80 and 81, in the selection process 82, an optimal prediction model 85 for the query sentence is selected by the feedback learning model 89. For example, if the query sentence is converted into an appropriate input value and the converted input value is applied to the feedback learning model 89, the optimal prediction model 85 may be output as the output value of the feedback learning model 89. have.

Subsequent processes 84, 84, 86, and 89 are the same as those described with reference to FIG. 19, and further description thereof will be omitted.

So far, the use example of the present invention has been described with reference to FIGS. 18 to 20. The technical idea of the present invention can be easily applied to all fields to which machine learning is applied in addition to the above-described application examples.

So far, reference has been made to some embodiments of the invention and effects in accordance with the embodiments with reference to FIGS. 2 to 20. Effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

The concepts of the present invention described so far with reference to FIGS. 2 through 20 may be implemented in computer readable code on a computer readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disc, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer equipped hard disk). Can be. The computer program recorded on the computer-readable recording medium may be transmitted to another computing device and installed in the other computing device through a network such as the Internet, thereby being used in the other computing device.

In the above description, all the components constituting the embodiments of the present invention are described as being combined or operating in combination, but the present invention is not necessarily limited to the embodiments. That is, within the scope of the present invention, all of the components may be selectively operated in combination with one or more.

Although the operations are shown in a specific order in the drawings, it should not be understood that the operations must be performed in the specific order or sequential order shown or that all the illustrated operations must be executed to achieve the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of the various configurations in the embodiments described above should not be understood as such separation being necessary, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. Should be understood.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it is to be understood that the embodiments described above are exemplary in all respects and not restrictive. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (18)

In the machine learning based prediction service providing method performed in a computing device,
Receiving data from a user terminal;
Selecting an optimal model to predict the data from among a plurality of prediction models using a feedback learning model;
Obtaining a prediction result for the data using the selected optimal model; And
Providing the obtained prediction result to the user terminal,
The feedback learning model is characterized in that the machine learning model learning the user feedback information on the satisfaction of the prediction result,
How to provide predictive service based on machine learning. According to claim 1,
The feedback learning model is a model built through a deep learning technique,
Each of the plurality of prediction models is a model built through machine learning techniques other than deep learning,
How to provide predictive service based on machine learning. According to claim 1,
The user feedback information includes a satisfaction score for the prediction result,
The feedback learning model is characterized in that the model built through reinforcement learning (reinforcement learning) on the satisfaction score,
How to provide predictive service based on machine learning. The method of claim 3, wherein
Selecting the optimal model,
In response to determining that a predetermined condition is satisfied, selecting a model having an expected reward for the data above a threshold from the plurality of prediction models as the optimal model; And
And in response to determining that the predetermined condition is not satisfied, determining a model randomly selected from the plurality of prediction models as the optimal model.
How to provide predictive service based on machine learning. According to claim 1,
Selecting the optimal model,
In response to determining that the learning degree of the feedback learning model is greater than or equal to a threshold, selecting the optimal model.
How to provide predictive service based on machine learning. According to claim 1,
Receiving first user feedback information on the obtained prediction result from the user terminal; And
The method may further include updating the feedback learning model using the first user feedback information.
How to provide predictive service based on machine learning. The method of claim 6,
The first user feedback information includes a satisfaction score for the obtained prediction result,
Updating the feedback learning model,
Adjusting a satisfaction score of the first user feedback information based on an average satisfaction score of the user; And
And updating the feedback learning model using the adjusted satisfaction score.
How to provide predictive service based on machine learning. According to claim 1,
Predicting a satisfaction score for the obtained prediction result in response to a determination that user feedback information for the obtained prediction result is not provided; And
And updating the feedback learning model by using the predicted satisfaction score.
How to provide predictive service based on machine learning. The method of claim 8,
Predicting the satisfaction score,
Characterized in that it comprises the step of predicting the satisfaction score based on a feedback providing pattern of the user,
How to provide predictive service based on machine learning. According to claim 1,
Accumulating a plurality of user feedback information provided from the user terminal;
Classifying the plurality of user feedback information into a first type and a second type; And
And subdividing the feedback learning model into a first feedback learning model that predicts an optimal model for the first type and a second feedback learning model that predicts an optimal model for the second type. ,
How to provide predictive service based on machine learning. The method of claim 10,
The subdividing step,
In response to determining that a predetermined condition is satisfied, subdividing the feedback learning model,
The predetermined condition is characterized in that the condition based on the average and variance of the number of user feedback by type,
How to provide predictive service based on machine learning. According to claim 1,
Each of the plurality of prediction models is a first prediction model constructed through machine learning techniques other than deep learning,
Accumulating a plurality of prediction results provided to the user terminal; And
The method may further include constructing a second prediction model by learning the plurality of prediction results through a deep learning technique.
How to provide predictive service based on machine learning. The method of claim 12,
The user feedback information includes a satisfaction score for the prediction result,
Building the second prediction model,
Filtering prediction results having a satisfaction score greater than or equal to a threshold among the plurality of prediction results; And
And learning the filtered prediction result to build the second prediction model.
How to provide predictive service based on machine learning. The method of claim 12,
Receiving a plurality of data from the user terminal;
Using the first prediction model and the second prediction model together to obtain a plurality of prediction results for the plurality of data; And
And providing the obtained plurality of prediction results to the user terminal.
How to provide predictive service based on machine learning. The method of claim 14,
The utilization ratio of the second prediction model is updated based on the learning degree of the second prediction model.
How to provide predictive service based on machine learning. The method of claim 14,
The utilization ratio of the second prediction model is updated based on user feedback information on the prediction result of the second prediction model.
How to provide predictive service based on machine learning. A processor; And
A memory for storing at least one program executed by the processor,
The at least one program,
Receiving data from a user terminal;
Selecting an optimal model to perform prediction on the data from among a plurality of prediction models using a feedback learning model;
Obtaining a prediction result for the data using the selected optimal model; And
Instructions for performing the operation of providing the obtained prediction result to the user terminal,
The feedback learning model is characterized in that the machine learning model learning the user feedback information on the satisfaction of the prediction result,
Predictive service providing device. In conjunction with computing devices,
Receiving data from a user terminal;
Selecting an optimal model to predict the data from among a plurality of prediction models using a feedback learning model;
Obtaining a prediction result for the data using the selected optimal model; And
Providing the obtained prediction result to the user terminal;
The feedback learning model is stored in a computer-readable recording medium, which is a machine learning model that learns user feedback information on satisfaction of prediction results.
Computer program.

Images