KR20190067717A - Apparatus and system for machine learning using coding technique - Google Patents

Abstract

The present invention discloses an apparatus for machine learning. The apparatus comprises: a coding data reception unit receiving a coded data set composed of a characteristic value and label from a client; a gradient correction unit calculating a gradient through the encoded data set and removing bias from the gradient; a model generating unit determining a model and parameter minimizing the loss by performing learning with a probabilistic gradient descent method based on the gradient from which the bias is removed; and a model transferring unit transferring the model and parameter in which the determines loss is minimized to the client. Through the present invention, the security and performance of a conventional machine learning algorithm optimization technique may be significantly improved.

Description

[0001] APPARATUS AND SYSTEM FOR MACHINE LEARNING USING CODING TECHNIQUE [0002]

The present invention relates to a machine learning apparatus and a system using an encoding technique, and more particularly, to a machine learning apparatus and a system that are performed in an environment such as cloud computing and distributed computing.

Recently, machine learning has evolved to a level that allows accurate prediction in various fields. The data required for machine learning can be classified into data requiring general privacy protection and general data. Medical data and cloud-based personal data correspond to data requiring privacy protection.

One of the techniques used to optimize machine learning algorithms is stochastic approximation of the slope (slope) of the objective function to be optimized and repeats the shift to the lower slope until it reaches the pole . This stochastic descent method requires data in order to approximate the slope. When the data is difficult to be taken out of the hospital, such as the patient's personal information, for example, There is a problem that the personal data of the patient can not be leaked out to the outside according to the hospital regulations when using the cloud service in which the learning model possessed by the third party exists to predict the risk.

In this way, even though a large amount of information is kept in the case of performing a machine learning at a medical institution that contains patient information, due to factors such as lack of computing equipment and lack of understanding of training algorithms for machine learning There was a difficulty in data analysis.

Recently, efforts have been made to provide machine learning-based services in third-party cloud environments such as Amazon, Microsoft, and Google, but due to legal requirements and ethical issues, such as patient data in hospitals, In applications where information security is to be maintained, such as limited services, the accessibility of data is very difficult and the utility of the technique is very limited.

Recently, it has been proved that encoding technology, which is a technique of transferring information into codes in order to protect information or to increase efficiency of information transmission, can be applied not only to linear algorithms but also to nonlinear algorithms such as deep learning. In order to create an optimization technique that maintains information security based on information security, it is necessary to have a systematic algorithm that can approximate the slope of an accurate function while maintaining security of information.

SUMMARY OF THE INVENTION It is an object of the present invention to improve security and performance by performing machine learning based on randomly mixed data without sharing original data.

According to an aspect of the present invention, there is provided a machine learning apparatus comprising: a coded data receiving unit for receiving a coded data set composed of a characteristic value and a label from a client, calculating a tilt through the coded data set, A model generating unit for determining a model and a parameter for minimizing loss by learning by a stochastic descent method on the basis of the slope from which the deflection is removed, and a model and parameters for minimizing the determined loss, To the client.

The encoded data set may further include: (a) extracting a data set including n number of characteristic values and n number of labels corresponding to the n number of characteristic values; and (b) And generating a coded data set by encoding the data set to be constructed.

Also, a process of encoding a data set having n characteristic values and n labels is performed by Equation (1).

Also, the process of removing the deflection from the tilt is performed by the following equation (2).

Also, a process of determining a parameter by which a loss is minimized by learning by a stochastic gradient descent method based on a slope from which the deflection is removed is performed by Equation (3).

The present invention also discloses a machine learning system for extracting a data set consisting of n characteristic values and n labels corresponding to the n characteristic values and generating a data set including n characteristic values and n labels, A coded data receiver for receiving a coded data set composed of a characteristic value and a label from the client, a tilt calculating unit for calculating a tilt through the coded data set, A model generating unit for determining a model and a parameter for minimizing a loss by learning by a stochastic descent method on the basis of the inclination removed from the deflection, and a model and a parameter for minimizing the determined loss to the client And a machine learning server that includes a model transfer unit for transferring the model to the system.

In addition, the client performs a process of learning the model using original data to the model delivered from the model delivery unit.

Effects of the machine learning apparatus and system using the encoding technique according to the embodiments of the present invention will be described as follows.

Through the present invention, the security and performance of the existing machine learning algorithm optimization technique can be significantly improved.

It also allows clients to achieve high machine learning performance without sharing the original data with sensitive customer information.

However, the effects that can be achieved by the machine learning apparatus and system using the encoding technique according to the embodiments of the present invention are not limited to those mentioned above, and other effects not mentioned can be obtained from the following description And will be apparent to one of ordinary skill in the art.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
1 schematically illustrates how a service provider provides an artificial intelligence service based on machine learning.
2 is a schematic representation of a machine learning system 1000 according to the present invention.
3 is a flowchart illustrating a process of deriving a machine learning model through the machine learning system 1000 according to the present invention.
FIG. 4 illustrates original data and encoded data according to an embodiment of the present invention.
5 is a block diagram showing a configuration of the machine learning server 100 according to the present invention.
6 is a schematic diagram illustrating a process of correcting a tilt in the tilt correction unit 120 according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments will be described in detail below with reference to the accompanying drawings.

The following examples are provided to aid in a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely an example and the present invention is not limited thereto.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification. The terms used in the detailed description are intended only to describe embodiments of the invention and should in no way be limiting. Unless specifically stated otherwise, the singular form of a term includes plural forms of meaning. In this description, the expressions "comprising" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, Should not be construed to preclude the presence or possibility of other features, numbers, steps, operations, elements, portions or combinations thereof.

It is also to be understood that the terms first, second, etc. may be used to describe various components, but the components are not limited by the terms, and the terms may be used to distinguish one component from another . Hereinafter, a machine learning apparatus and a system using an encoding technique according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 schematically illustrates how a service provider provides an artificial intelligence service based on machine learning.

The present invention relates to a machine learning apparatus and system using an encoding technique, and provides a machine learning model that does not share original data so that a user can use a desired service.

Machine learning is one of the research fields of artificial intelligence. It refers to technologies and techniques that realize functions such as human learning ability on a computer. Machine learning includes various approaches such as clustering, deep learning, artificial neural networks, expressive learning, reinforcement learning, and Bayesian network. In general, machine learning models are learned using training data, which means already trained models.

The machine learning server 100 is an object that receives data from a plurality of clients 200, that is, a data provider and provides a predetermined service. Here, the machine learning server 100 may be understood to mean a machine learning apparatus possessed by a service provider, and the service provider may be, for example, a Google cloud, an Amazon Web service, or the like.

The service provider can analyze the data collected from the client 200 through a machine learning apparatus possessed by the service provider (or a third party), and provide the service in a manner of presenting an optimal model to the client.

Examples of such services include medical analysis services, learning analysis services, and the like. As an example of a medical analysis service, data from a plurality of patients may be analyzed to provide a medical analysis service that predicts the likelihood of re-admission or the risk of developing a particular disease.

In order to receive such a medical analysis service, a user must send data including personal information to the service provider, and the privacy of the user may be a problem.

2 is a schematic representation of a machine learning system 1000 according to the present invention.

The machine learning system 1000 according to the present invention includes a client 200 and a machine learning server 100. The client 200 may be, for example, a hospital that holds patient data.

및 상기 n 개의 특성값에 대응되는 n 개의 레이블

로 구성되는 데이터 세트

를 추출하고, 상기 n 개의 특성값 및 n 개의 레이블로 구성되는 데이터 세트를 부호화하여 부호화된 데이터 세트

를 생성한다.The client 200 has n property values

And n labels corresponding to the n characteristic values

≪ / RTI >

And encodes a data set composed of the n characteristic values and n labels to generate a coded data set

.

를 수신하고, 이러한 부호화된 데이터 세트를 통하여 경사를 계산하여, 상기 경사에서 편향을 제거하는 보정을 수행한 후, 편향이 제거된 경사를 기초로 확률적 경사 하강법으로 학습하여 손실이 최소화되는 모델 및 파라미터를 결정한다. 이후 손실이 최소화되는 모델(

) 및 파라미터(

)를 클라이언트(200)로 전달한다.The machine learning server 100 receives from the client 200 an encoded data set

Calculates a slope through the coded data set, performs correction to remove the slope from the slope, and then learns by a stochastic slope descent method based on the slope from which the slope is removed to minimize the loss And parameters. Then the model with minimal loss (

) And parameters (

) To the client (200).

)에 원본 데이터

를 이용하여, 재차 학습함으로써 모델의 정확성을 높일 수 있다.The client 200 receives the model (s)

) To the original data

The accuracy of the model can be improved by learning again.

3 is a flowchart illustrating a process of deriving a machine learning model through the machine learning system 1000 according to the present invention.

Referring to FIG. 3, a machine learning system 1000 according to the present invention includes a step S100 of generating an encoded data set, a step S200 of calculating a gradient using a coded data set, (Step S300) of learning an optimal model and parameters by learning by a slope descent method, and learning a model using an original data set (step S400).

In operation S100, the client 200 generates a coded data set. As described above, the coded data set includes a step of extracting a data set composed of n characteristic values and n labels corresponding to the n characteristic values, and extracting the data consisting of n characteristic values and n labels And encoding the set.

를 부호화하는 과정은, 하기 (수학식 1)과 같이 수행될 수 있다.Here, a data set composed of the n characteristic values and n labels

May be performed as Equation (1). &Quot; (1) "

(1)

는 무작위로 추출되는 부호화 계수로

로 표현되며

는 베르누이 분포,

은 부호화 폭,

은 정규분포,

는 특성값,

는 레이블,

는

,

는

로

는 항등 행렬(identity matrix),

,

는 행렬의 크기(size)이다.here

Is a randomly extracted coding coefficient

And

The Bernoulli distribution,

The coding width,

Is a normal distribution,

Is a characteristic value,

The label,

The

,

The

in

Is an identity matrix,

,

Is the size of the matrix.

The encoding process means random mixing of the original data, thereby anonymizing the personal information containing the privacy.

,

가 가우시안 노이즈에 해당한다. 가우시안 노이즈는 가우시안 분포를 이용한 렌덤 프로세스의 결과물로, 정규분포를 가지는 잡음을 의미한다.The present invention further uses Gaussian noise in the process of encoding the original data set. In Equation (1)

,

Corresponds to Gaussian noise. Gaussian noise is a result of a random process using Gaussian distribution, which means noise having a normal distribution.

FIG. 4 illustrates original data and encoded data according to an embodiment of the present invention. The left side shows the original data before encoding, and the right side is the data in which the encoding process is performed from the original data.

Referring to FIG. 4, it can be seen that the contents of the original data on the left side are completely hidden from the data on the right side. As described above, according to the present invention, original data can not be derived from coded data by adding Gaussian noise in the encoding process, thereby effectively protecting personal information.

In the client 200, the original data is encoded and transferred to the machine learning server 100. Steps S200 and S300 are performed through the machine learning server 100, which will be described in detail with reference to FIG.

5 is a block diagram showing a configuration of the machine learning server 100 according to the present invention.

The machine learning server 100 according to the present invention includes a coded data receiving unit 110 for receiving a coded data set composed of a characteristic value and a label from a client and calculating a tilt through the coded data set, A model generating unit 130 for determining a model and parameters for which loss is minimized by learning by a stochastic descent method based on the slope from which the deflection is removed, and a model for minimizing the determined loss And a model transferring unit 140 for transferring the parameters to the client.

를 수신하여 경사를 계산하고, 이러한 경사에서 편향을 제거하는 단계이다.In step S200, the encoded data receiving unit 110 of the machine learning server 100 transmits the encoded data set

To calculate the slope, and to remove the slope at this slope.

The process of removing the deflection from the tilt can be performed by the following equation (2).

(2)

는 경사로

은 손실함수,

는 부호화된 특성,

는 부호화된 레이블,

는 심층 선형 신경망에서

번째 레이어의 파라미터,

는 편향이다. 그리고

이고,

이다.here

The ramp

Is the loss function,

Lt; RTI ID = 0.0 >

Lt; RTI ID = 0.0 > label,

In the deep linear neural network

Th layer,

Is a bias. And

ego,

to be.

Since the present invention generates a model with randomly mixed data, that is, a coded data set, rather than generating a model with a set of original data, the accuracy is inevitably lower than that of a model using an original. Accordingly, in order to appropriately interpolate the difference, the slope correction unit 120 performs an operation of removing the slope from the slope value calculated through the encoded data set.

이 최종적으로 수렴하고자 하는 모델의 최적 파라미터이고,

가 현재 파라미터이다. 6 is a schematic diagram illustrating a process of correcting a tilt in the tilt correction unit 120 according to an embodiment of the present invention. In the diagram

Is an optimal parameter of the model to be finally converged,

Is the current parameter.

If the green arrow is an inclination vector when original data that is not encoded is used and the red arrow is an inclination vector when the encoded data is used, a blue arrow which is a difference between the two vectors corresponds to the deflection. Gaussian noise In the case of using the coded data added with the coded data. Therefore, by removing the deflection, a numerical value close to the slope value using the original data can be derived, so that the learning of the model can be performed more accurately.

Referring to FIGS. 4 and 5 again, step S300 will be described. Step S300 is a step of determining a model and parameters in which the loss is minimized by learning by the stochastic gradient descent method based on the slope value obtained by removing the deflection calculated in step S200. This is performed through the model generation unit 130 of the machine learning server 100. The stochastic gradient descent method used here is a known algorithm, and a detailed description thereof will be omitted.

The process in which the model generating unit 130 learns the parameters by the stochastic gradient descent method based on the slope from which the deflection is removed can be performed as Equation (3).

(3)

이고,

는 학습률이다.here

ego,

Is the learning rate.

에 반복하여 최적의 모델

및 이 경우의 파라미터를 탐색한다.Thereafter, the model generation unit 130 repeats step S200 and the stochastic descent method using m models

The optimal model

And the parameters in this case are searched.

This process can be expressed by the following equation (4).

(4)

는 업데이트의 의미이다. 즉, 가장 적합한 모델이 도출하기 위해 계산된 파라미터를 계속하여 업데이트하고 궁극적으로 최적의 모델

및 파라미터

를 결정하여 모델 전달부(140)를 통해 이를 클라이언트(200)에게 제공한다.here

Is the meaning of the update. That is, the calculated parameters are continuously updated to derive the most suitable model, and ultimately the optimal model

And parameters

And provides it to the client 200 through the model transfer unit 140. [

In operation S400, the client 200 learns the model using the original data set to the model received from the model delivery unit 140. [ Since the client 200 receives the learned model from the machine learning server 100 through the encoded data set, the client 200 has lower accuracy than the model learned through the original data. Therefore, the performance of the model can be improved by learning the model again as the original data.

) 및 파라미터(

)를 모두 전달받아 원본 데이터 세트를 이용한 학습을 수행할 수 도 있고, 모델(

)만을 전달받아 원본 데이터 세트를 이용한 학습을 수행하여 파라미터를 다시 도출하는 방식으로 수행할 수 도 있다. 이는 서비스 제공의 목적, 환경, 비용에 따라 적절히 정해질 수 있다.In this case, the client 200 receives from the machine learning server 100 a model

) And parameters (

), And can perform learning using the original data set, and the model (

) Is received and learning is performed using the original data set to derive the parameters again. This can be properly determined according to the purpose, environment and cost of service provision.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers of the technical field to which the present invention belongs.

In addition, the above-described method and apparatus using the same may not be limited in configuration and method of the embodiments described above, but the embodiments may be modified so that all or some of the embodiments are selectively As shown in FIG.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, . Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the scope of the appended claims, as well as the appended claims.

1000: Machine learning system
100: machine learning server
110: encoded data receiving unit 120: gradient correction unit
130: model generation unit 140: model delivery unit
200: Client

Claims (11)

A coded data receiver for receiving a coded data set composed of a characteristic value and a label from a client;
A gradient correction unit for calculating gradient through the encoded data set and removing bias from the gradient;
A model generating unit that learns a model and a parameter whose loss is minimized by learning by a stochastic descent method based on the slope from which the deflection is removed; And
A model transferring unit for transferring the model and the parameter with which the determined loss is minimized to the client;
The machine learning apparatus comprising:
The method according to claim 1,
Wherein the encoded data set comprises:
(a) extracting a data set composed of n characteristic values and n labels corresponding to the n characteristic values; And
(b) generating a coded data set by coding a data set including the n characteristic values and n labels;
Is generated.
3. The method of claim 2,
Wherein the encoding of the data set including the n characteristic values and the n labels is performed by the following equation (1).
(1)

(here

Is a randomly extracted coding coefficient

And

The Bernoulli distribution,

The coding width,

Is a normal distribution,

Is a characteristic value,

The label,

The

,

The

in

Is an identity matrix,

,

Is the size of the matrix.
The method of claim 3,
Wherein the step of removing the deflection from the tilt is performed by the following equation (2).
(2)

(here

The ramp

Is the loss function,

Lt; RTI ID = 0.0 >

Lt; RTI ID = 0.0 > label,

In the deep linear neural network

Th layer,

Lt; / RTI >

ego,

being.)
5. The method of claim 4,
Wherein learning a parameter by minimizing a loss by learning a stochastic descent method based on a slope from which the deflection is removed is performed by the following equation (3).
(3)

(here

ego,

Is the learning rate.)
a data set having n characteristic values and n labels corresponding to the n characteristic values, and generating a coded data set by encoding a data set composed of the n characteristic values and n labels, ; And
A coded data receiver for receiving a coded data set composed of a characteristic value and a label from the client, a tilt correction unit for calculating a tilt through the coded data set and removing the tilt from the tilt, A model generating unit that learns a model and a parameter that minimizes a loss by learning with a stochastic gradient descent method based on the model; and a model transferring unit that transmits a model and parameters that minimize the determined loss to the client.
The machine learning system comprising:
The method according to claim 6,
Wherein the encoded data set comprises:
(a) extracting a data set composed of n characteristic values and n labels corresponding to the n characteristic values; And
(b) generating a coded data set by coding a data set including the n characteristic values and n labels;
Of the machine learning system.
8. The method of claim 7,
Wherein the encoding of the data set including the n characteristic values and the n labels is performed by the following equation (1).
(1)

(here

Is a randomly extracted coding coefficient

And

The Bernoulli distribution,

The coding width, Is a normal distribution,

Is a characteristic value,

The label,

The

,

The

in

Is an identity matrix,

,

Is the size of the matrix.
9. The method of claim 8,
Wherein the step of removing deflection from the tilt is performed by the following equation (2).
(2)

(here

The ramp

Is the loss function,

Lt; RTI ID = 0.0 >

Lt; RTI ID = 0.0 > label,

In the deep linear neural network

Th layer,

Lt; / RTI >

ego,

being.)
10. The method of claim 9,
Wherein learning a parameter by minimizing a loss by learning by a stochastic descent method based on a slope on which the deflection is removed is performed by the following equation (3).
(3)

(here

ego,

Is the learning rate.)
The method according to claim 6,
Wherein the client performs a process of learning the model using original data to the model delivered from the model delivery unit.

Images