KR20220083630A - Deep learning using embedding vectors of heterogeneous data-sets in multi-distributed database environments - Google Patents

Abstract

According to an embodiment, a deep learning learning method performed by a deep learning learning apparatus in a multi-distributed database environment includes the steps of merging heterogeneous datasets of a multi-distributed database on a time series basis to generate an input stream for model training; Generating an embedding vector for an input stream, and adaptively reflecting the embedding vector to at least one of learning of a deep learning model and inference through a pre-learned deep learning model.

Description

DEEP LEARNING USING EMBEDDING VECTORS OF HETEROGENEOUS DATA-SETS IN MULTI-DISTRIBUTED DATABASE ENVIRONMENTS}

The present invention relates to deep learning learning using embedding vectors of heterogeneous datasets in a multi-distributed database environment.

In recent years, with the development of information technology, the explosive increase in data size, diversification of data types, and the emergence of big data with faster data cycles, the importance of creating new values through the use of big data is increasing.

Accordingly, the concept of data economy, a new ecosystem based on data distribution, has emerged as data has been attracting attention as a new type of asset. , medical, environmental, energy, etc.) data producers and providers, consumers who collect and utilize shared data to provide data services through AI learning, and members who play different roles, such as managers who provide and manage shared infrastructure is made of

In order to technically support the open data ecosystem, the data synchronization technology of the data sharing management function has been developed, which duplicates the data set of the supplier database to the consumer database, or the data set of the consumer database according to the data set update through database change detection. It supports the distribution of open data through the real-time synchronization function.

In this open data ecosystem environment, through data synchronization technology, data consumers ultimately receive the necessary data among different domain data such as public and private energy, environment, medical care, and culture from the data provider in real time, store the data in the database, and store the data. Through the set, deep learning-based artificial neural network model training can be performed, and services can be provided when the model's performance is verified.

General deep learning-based artificial neural network models are designed to solve various problems without being limited to specific domains such as life/finance/engineering, and are trained for a wide range of purposes such as image and text classification, object detection, speech recognition, and natural language processing. In most cases, the data to be learned is learned from a large amount of unchanging datasets stored in a database.

However, if the learning target data in the open data ecosystem environment where the dataset is updated in real time is data with time series characteristics, the accuracy of the model will be improved if the newly added data is inferred using the deep learning model trained on the existing dataset. lower, and the model must be retrained to reflect real-time changing data characteristics.

Among the learning-based models for real-time change data characteristics, a common method is the incremental learning method in which retraining and model update are performed using a newly added dataset. Its purpose is to form a robust and flexible deep learning model from data characteristics.

In addition, in recent studies, effective techniques for learning new data or classes have been studied. When a gradient based on regularization is used, good performance is shown in learning existing data and new data in an environment in which data is increased. In order to apply the above method to time-series data, which has a characteristic that the meaning greatly changes according to time and sequence, such as a language change environment, there is a method of extracting a representative embedding vector.

Embedding means expressing a multi-dimensional vector having the meaning of specific data, and among the feature representations, which is a method of expressing the properties of an object as a multi-dimensional vector, a dense representation or a distributed representation ( distributed representation)). The embedding technique is often used for natural language processing, and unlike the sparse representation (one-hot encoding) technique, which expresses as many independent dimensions as the number of all possible instances of the property, the user Data properties can be expressed with an assumed number of dimensions, and each property is not expressed as an independent dimension.

(Document 1) R. Polikar, L. Upda, S. S. Upda, and V. Honavar, "Learn++: An incremental learning algorithm for supervised neural networks," IEEE transactions on systems, man, and cybernetics, part C (applications and reviews) , vol. 31, no. 4, pp. 497-508, 2001. (Document 2) O. Levy and Y. Goldberg, "Neural word embedding as implicit matrix factorization," in Advances in neural information processing systems, 2014, pp. 2177-2185. (Document 3) Y.-Y. Chang, F.-Y. Sun, Y.-H. Wu, and S.-D. Lin, “A memory-network based solution for multivariate time-series forecasting,” arXiv preprint arXiv:1809.02105, 2018.

An embodiment of the present invention provides a deep learning learning apparatus and method that is adaptively reflected in model learning or model inference after generating an embedding vector for an input stream generated by merging heterogeneous datasets on a time series basis in a multi-distributed database environment. to provide.

The problems to be solved of the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description.

A deep learning learning method performed by a deep learning learning apparatus in a multi-distributed database environment according to the first aspect includes the steps of merging heterogeneous datasets of a multi-distributed database on a time series basis to generate an input stream for model training; Generating an embedding vector for the stream, and adaptively reflecting the embedding vector in at least one of learning of a deep learning model and inference through a pre-learned deep learning model.

A deep learning learning apparatus according to a second aspect includes an input unit that receives a heterogeneous dataset of a multi-distributed database, and a processor unit that performs learning or inference of a deep learning model using the heterogeneous dataset, wherein the processor unit comprises: By merging heterogeneous datasets on a time series basis, an input stream for model training is generated, an embedding vector is generated for the input stream, and the embedding vector is used for learning of the deep learning model and inferring through a pre-trained deep learning model. adaptively reflected in at least one of

The computer-readable recording medium storing the computer program according to the third aspect, when the computer program is executed by a processor, merges heterogeneous datasets of a multi-distributed database on a time-series basis to generate an input stream for model training Deep comprising the steps of: generating an embedding vector for the input stream; and adaptively reflecting the embedding vector in at least one of learning of a deep learning model and inference through a pre-learned deep learning model. A computer-readable recording medium comprising instructions for causing the processor to perform a learning learning method.

The computer program stored in the computer-readable recording medium according to the fourth aspect, when the computer program is executed by a processor, merges heterogeneous datasets of a multi-distributed database on a time-series basis to generate an input stream for model training Deep comprising the steps of: generating an embedding vector for the input stream; and adaptively reflecting the embedding vector in at least one of learning of a deep learning model and inference through a pre-learned deep learning model. and instructions for causing the processor to perform a learning learning method.

According to an embodiment of the present invention, a deep learning learning method using an embedding vector of heterogeneous datasets in a multi-distributed database environment is a time change in the distribution of real-time shared data and data service utilization through deep learning pursued in an open data ecosystem. By applying an incremental learning method that additionally learns time series data characteristics according to

In addition, when heterogeneous datasets updated in real time from multiple distributed databases have different arrival rates and are given as input for deep learning learning, the dimensions are reduced through embedding the data to form a representative value in the form of a vector, and for new data later Re-learning is performed by composing a normalized gradient, and when there is an inference request, the inference accuracy can be improved by re-learning data closely related to the current inference data through the embedding vector in advance.

1 is a diagram illustrating a multi-distributed database environment to which a deep learning learning apparatus and method according to an embodiment of the present invention can be applied.
2 is a block diagram of a deep learning learning apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a deep learning learning method according to an embodiment of the present invention.
4 is an exemplary diagram for explaining a deep learning scheduling method from a dataset update stream generated from a multi-distributed data hub according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a process of generating a prediction stream by generating a weighted gradient and an embedding vector based on a learning task data stream in deep learning model learning according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully inform the person of the scope of the invention, and the scope of the invention is only defined by the claims.

In describing the embodiments of the present invention, detailed descriptions of well-known functions or configurations will be omitted except when it is actually necessary to describe the embodiments of the present invention. In addition, the terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification.

In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. In this application, terms such as 'comprise' or 'comprise' are intended to designate that a feature, number, step, operation, component, part, or a combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Also, in an embodiment of the present invention, when it is said that a part is connected to another part, this includes not only direct connection but also indirect connection through another medium. In addition, the meaning that a certain part includes a certain component means that other components may be further included, rather than excluding other components, unless specifically stated to the contrary.

When time series processing of stream data input from multiple distributed databases is performed through machine learning or deep learning techniques (regressor or predictor), information that has occurred in the past is generally collectively collected, data set formatted, and stored, and then the corresponding Training is performed on the dataset. However, when data following a non-stationary probability process having a time-varying characteristic is given as an input, a problem arises in that an inference error gradually increases in proportion to the delay time between learning and inference due to the changing characteristic. The most common method for learning-based models for real-time changing data characteristics is to retrain and update the model with a new dataset. In the prior art, in order to learn the characteristics of data that change in real time, it was attempted to solve the problem through re-learning of data input in real time through incremental learning. Incremental learning is an online learning method that gradually and continuously trains the model with stream data with new data properties, rather than training the entire dataset once. However, it is effective only when the distribution of data does not change significantly due to excessively learning the characteristics of the latest data. vulnerable to That is, the learning convergence speed and accuracy performance may be greatly affected by the degree of change in the time series distribution from the database.

Also, when time series data input from multiple distributed databases is input with different arrival rates, it causes fluctuations in the input data due to skewed data distribution, which reduces the learning accuracy. there is When data input from multiple random processes is chronologically sorted and collected as an input of a deep learning model, the degree of difference between adjacent data increases significantly compared to the original dataset. In the case of over-fitting to the recent time-varying characteristics due to the characteristic of gradual learning of partial data, there is a problem in that performance and stability are further reduced in a multi-distributed database environment. In the prior art, it was shown that a small amount of already learned data is stored, and good performance is shown in learning the existing data and new data by reflecting the gradient to be used in the previously learned model optimization process through the gradient method based on regularization during real-time learning. . The prior art was mainly applied to the classification problem, but it did not consider the environment in which learning and reasoning occur at the same time.

In the present invention, real-time learning that is adaptive to the degree of data time-varying characteristic change in an environment where time series learning and inference for a complex dataset input from a multi-distributed database occur at the same time, but is not overfitted for short-term input even for multi-distributed database input; We propose an inference model.

In the present invention, in order to solve the problems of the prior art, in consideration of an environment in which time series learning and inference for a complex dataset input from a multi-distributed database occur at the same time, a normalization-based basis for a biased data distribution that changes greatly through an embedded vector of data A method of effectively reflecting the value of a vector based on a pattern when the previously learned information through the gradient method is reflected in the information currently being learned, and an online inference based on model re-learning that is suitable for the data characteristics to be inferred through an embedding vector including methods.

1 is a diagram illustrating a multi-distributed database environment to which a deep learning learning apparatus and method according to an embodiment of the present invention can be applied. As shown in FIG. 1 , the deep learning learning apparatus and method according to an embodiment of the present invention can be applied to a multi-distributed database environment in which a plurality of databases are distributedly arranged on a network.

2 is a block diagram of a deep learning learning apparatus according to an embodiment of the present invention.

Referring to FIG. 2 , the deep learning learning apparatus 100 according to the embodiment includes an input unit 110 and a processor unit 120 . In addition, the deep learning learning apparatus 100 may further include a storage unit 130 and/or an output unit 140 .

The input unit 110 receives a heterogeneous dataset of a multi-distributed database. In addition, the input unit 110 may be requested to infer data. The input unit 110 may include a communication module connected to the multi-distributed database through a communication network. Alternatively, the input unit 110 may include an interface connected to a separate communication module connected to the multi-distributed database through a communication network.

The processor unit 120 performs learning or inference of a deep learning model using a heterogeneous dataset. Here, the processor unit 120 generates an input stream for model learning by merging heterogeneous datasets on a time-series basis, generates an embedding vector for the input stream, and uses the embedding vector to learn the deep learning model and pre-learned deep. It is adaptively reflected in at least one of the inferences through the learning model.

For example, the processor unit 120 may calculate a gradient through the embedding vector, and may learn a parameter of the deep learning model using the gradient. For example, when calculating the gradient, the processor 120 may calculate the gradient by applying pattern-based regularization, and may learn the parameters of the deep learning model by using the gradient to which the normalization is applied.

For example, the processor unit 120 may select data whose data characteristics meet the inference processing requirements based on the embedding vector, and may re-learn the parameters of the deep learning model previously learned on the selected data, , data inference can be performed through the retrained deep learning model.

The storage unit 130 may store a computer program including instructions for enabling the processor unit 120 to perform the deep learning learning method according to an embodiment of the present invention. In addition, the storage unit 130 may store various processing results by the processor unit 120 .

The output unit 140 may output various processing results by the processor unit 120 . For example, the output unit 140 may include a communication module connected to the multi-distributed database through a communication network. Alternatively, the output unit 140 may include an interface connected to a separate communication module connected to the multi-distributed database through a communication network.

3 is a flowchart illustrating a deep learning learning method according to an embodiment of the present invention. 3 will be referred to when looking in detail below for a deep learning learning method.

4 is an exemplary diagram for explaining a deep learning scheduling method from a dataset update stream generated from a multi-distributed data hub according to an embodiment of the present invention. As shown in FIG. 4 , it is shown that updates generated from multiple databases may have different arrival rates due to data generation or transmission deviation. The big data learning system generates an embedding vector from the dataset stream to remove the deviation caused by the arrival rate from the online update data, and performs pattern-based gradient optimization through the generated embedding vector, and then the embedding vector The updated dataset is trained on the deep learning service model through the online inference process through

5 is an exemplary diagram for explaining a process of generating a prediction stream by generating a weighted gradient and an embedding vector based on a learning task data stream in deep learning model learning according to an embodiment of the present invention. When an input stream with different arrival rates from a multi-distributed database is given as an input for deep learning learning, the dimensions are reduced through embedding data to form a representative value in the form of a vector, and then a normalized gradient is constructed for new data and re-learning and at the same time when there is an inference request, it shows the procedure for performing model re-learning by precedenting the data closely related to the current inference data through the embedding vector.

Hereinafter, a process of performing deep learning learning by the deep learning learning apparatus 100 according to an embodiment of the present invention with reference to FIGS. 1 to 5 and a process of performing data inference through the learned deep learning model. do it with

의 입력(S210)과 함께 추론 요청

가 딥 러닝 학습 모델에 주어질 때(S240), 학습 및/또는 추론 알고리즘이 동작한다. 학습 데이터세트에서 취합 된 학습 입력 데이터

의 연속 된 값으로부터 임베딩함수를 통해 스칼라(Scalar) 값을 도출하고, 분포도가 높은 입력의 대표 벡터값인 임베딩 벡터를 도출한다(S220, S230). 학습 시에는 임베딩 벡터를 통한 기울기를 계산한 후(S250), 계산된 기울기를 이용하여 딥 러닝 모델의 파라미티를 학습한다(S260. 추론 요청이 들어올 경우에는 임베딩 벡터를 기반으로 최적의 데이터를 선별한 후(S270), 딥러닝 모델을 재학습을 수행하고(S280), 재학습된 딥 러닝 모델에 대해 추론하는 과정을 수행한다(S290).The deep learning learning algorithm by applying an embedding vector to a heterogeneous dataset of the present invention considers a model in which time series learning and inference occur simultaneously, and derives representative embedding values that can increase the stability of learning as a vector for learning and inference We propose a method of adaptively reflecting. Multiple training datasets

Inference request with the input (S210) of

When is given to the deep learning learning model ( S240 ), a learning and/or inference algorithm operates. Training input data collected from training dataset

A scalar value is derived through an embedding function from the continuous values of , and an embedding vector that is a representative vector value of an input with a high distribution is derived (S220, S230). In training, after calculating the gradient through the embedding vector (S250), the parameters of the deep learning model are learned using the calculated gradient (S260. When an inference request is received, optimal data is selected based on the embedding vector) After (S270), re-learning the deep learning model is performed (S280), and a process of inferring on the re-trained deep learning model is performed (S290).

과 타겟

으로 이루어지며 n는 데이터 인스턴스의 수, p은 데이터의 차원이다. 학습을 위해 입력되는 데이터세트는 각기 다른 분산 데이터베이스 원천(

)으로부터 제공된 데이터로 도착한 순서를 가지고 정렬을 하면 다음의 수학식 1과 같은 학습 데이터세트로 구성할 수 있다.For a deep learning learning model that is updated in real time, the i-th input dataset is the input

and target

, where n is the number of data instances and p is the dimension of the data. The dataset input for training is from different distributed database sources (

) and sorting in the order of arrival with the data provided from ), it can be composed of a training dataset as shown in Equation 1 below.

[Equation 1]

)으로부터 도착하는 추론 요청은 i번째의 학습이 수행 된 후, 다음 입력 i+1에 대하여 입력 X에 대한 추론 값을 찾는 것을 목적으로 하며, 추론 요청은 다음의 수학식 2와 같이 표현할 수 있다.At the same time, different distributed database sources (

), the purpose of the inference request is to find an inference value for the input X with respect to the next input i+1 after the i-th learning is performed, and the inference request can be expressed as Equation 2 below.

[Equation 2]

의 k번째 값에 대해서 고차원의

를 적은 차원, 혹은 Scalar의 값으로 변환하는 임베딩을 적용한다 (

.

,

). 임베딩을 적용하기 위한 방법으로는 자연어 처리에서 워드 임베딩, 시계열 데이터에 대한 코사인 유사도, 다차원 데이터에 대한 차원 감소 기법을 사용할 수 있다 (Auto Encoder, PCA 등).Training input data collected from training dataset

For the k-th value of

Apply an embedding that transforms , into a smaller dimension or a value of Scalar

.

,

). As a method for applying embedding, word embedding in natural language processing, cosine similarity for time series data, and dimensionality reduction for multidimensional data can be used (Auto Encoder, PCA, etc.).

[Equation 3]

벡터 기준으로 p차원의 데이터와 코사인 연산을 통해 두 벡터의 유사도를 구할 수 있으며, 기준 벡터를 기준으로 떨어진 정도를 정량화하여 값의 대표 특성으로 분석할 수 있다.The present invention shows a cosine similarity method for time series data as a practical example. equidistant from all axes

The degree of similarity between two vectors can be obtained through cosine operation with p-dimensional data based on the vector, and the degree of separation from the reference vector can be quantified and analyzed as a representative characteristic of the value.

[Equation 4]

는 i번째 까지의 학습 과정에서 입력된 데이터를 편향되지 않는 고른 분포를 가지도록 유지하는 임베딩 값을 기준으로 분류한 대표 벡터로써 기존의

과

를 통해 구성된다. 이를 위하여 먼저는 두 벡터를 합치기 위해 벡터의 합성을 수행한다.embedding vector

is a representative vector classified based on the embedding value that maintains an unbiased, even distribution of the input data in the learning process up to the i-th.

class

is composed through To do this, first, vector synthesis is performed to combine two vectors.

[Equation 5]

의 최대 크기인 h를 넘지 않도록 (

)하는 값을 구간별로 찾는다. 임베딩의 값이 작은 (코사인 거리가

에 가까운) 순서부터 큰 순서로 올라가면서 반복적으로 대표 입력값을 다음과 같이 탐색한다. 해당 임베딩 범주의 j 번째 구간에 들어가는 입력값들을 다음과 같이 분류할 수 있다.To find a finite number of representative values

so as not to exceed h, the maximum size of (

) for each section. If the value of the embedding is small (the cosine distance

) and iteratively searches the representative input values as follows, ascending from the largest to the largest. Input values entering the j-th section of the corresponding embedding category can be classified as follows.

[Equation 6]

중에서 랜덤으로 하나의 값을 선택하는 함수를

이라고 하고 해당 값이 원래의 집합

에서의 인덱스를 찾아내는 함수를

라고 하였을 때, 임베딩 벡터는 다음과 같이 구할 수 있다.In this case, a representative value may be selected by randomly selecting among the values of the corresponding section.

A function that randomly selects a value from among

, and the corresponding value is the original set

function to find the index in

, the embedding vector can be obtained as follows.

[Equation 7]

After constructing the embedding vector, we describe how to apply the gradient to which normalization is applied using the vector in learning. The slope for the current task is calculated as

[Equation 8]

는

에 대한 타겟값으로 [수학식 7]를 계산할 때

에 대해서 구한 항목이다. Also, the gradient for the embedding vector is calculated as follows. At this time,

Is

When calculating [Equation 7] as a target value for

It is an item saved for

[Equation 9]

In this case, the normalized slope can be calculated as follows.

[Equation 10]

과 현재 기울기

를 사용자에 의해 정의된 하이퍼파라미터

비율로 새로운 일차모멘텀

를 업데이트한다.As an example for learning the parameters of a deep learning model using a normalized gradient, the parameter update process through the Adam Optimizer will be described. First, the first-order momentum of the previous optimization process (i-1) of the optimizer

and current slope

is a hyperparameter defined by the user.

New Primary Momentum in Ratio

update

[Equation 11]

의 비율로 이전 최적화과정 (i-1)의 이차모멘텀

과 현재 기울기 제곱인

에서 새로운 이차모멘텀

를 업데이트한다.Secondary momentum is defined by the user.

Secondary momentum of the previous optimization process (i-1) as the ratio of

and the current slope squared

new secondary momentum in

update

[Equation 12]

비율로 업데이트로 인해

만큼 bias생긴 momentum의 영향을 줄이기 위해 [수학식 13]과 [수학식 14] 과정처럼 현재 모멘텀들,

과

, 에서

를 나누는 보정 과정을 거침The Adam optimizer runs multiple times

due to update in percentage

In order to reduce the influence of momentum generated by bias as much as [Equation 13] and [Equation 14], the current momentums,

class

, at

go through a calibration process that divides

[Equation 13]

[Equation 14]

를 업데이트하고 있다.The final update is a method of subtracting the value obtained by dividing the value of [Equation 13] by the value of [Equation 14] as much as a ratio in order to reflect a constant slope change amount with the above values.

is updating

[Equation 15]

를 구했을 때, 동시에 입력되는

에 대한 추론 성능을 극대화하기 위해서는 정규화로 인한 최신의 데이터 특성을 딥러닝 네트워크로 다시 재학습 시킬 수 있다. 이를 위하여 임베딩 벡터

로부터 추론 입력

에 대한 임베딩 벡터로부터의 저장 기억 복구를 위해

와 가장 유사한 과거의 데이터 입력을 쿼리한다.Through updating the deep learning parameters for the input i-th dataset

, which is input at the same time

In order to maximize the inference performance for , the latest data characteristics due to regularization can be re-trained with a deep learning network. For this, the embedding vector

input inference from

To recover the save memory from the embedding vector for

Query the historical data input most similar to

[Equation 16]

를 통하여 딥러닝 모델의 파라메터를 정규화 하지 않은 상태의 재학습함을 통하여 개선 된 파라메터

값을 구한다. 최종적으로 개선된 딥러닝 모델 파라메터를 통해 추론을 진행한다. 해당 과정은 추론 데이터에 대해 반복적으로 진행할 수 있다.corresponding

The parameters improved through re-learning without normalizing the parameters of the deep learning model through

get the value Finally, inference proceeds through the improved deep learning model parameters. The process may be iteratively performed on the inference data.

[Equation 17]

The present invention has previously learned through a normalization-based gradient method for a biased data distribution that changes greatly through an embedding vector of data in consideration of an environment in which time series learning and inference for a heterogeneous complex dataset input from a multi-distributed database occur simultaneously. It relates to the inclusion of a method of effectively reflecting the value of a vector based on a pattern when the information is reflected in the information to be currently learned, and an online inference method based on model re-learning suitable for the data characteristics to be inferred through an embedding vector. It may include a step of generating an embedding vector from a distributed database input stream of Accordingly, the model can be updated by reflecting the time-varying data characteristics of heterogeneous complex datasets updated with different arrival rates, and inference accuracy can be increased.

On the other hand, each step included in the deep learning learning method according to the above-described embodiment may be implemented in a computer-readable recording medium for recording a computer program programmed to perform these steps.

In addition, each step included in the deep learning learning method according to the above-described embodiment may be implemented in the form of a computer program stored in a computer-readable recording medium programmed to perform these steps.

Combinations of each block in the block diagram attached to the present invention and each step in the flowchart may be performed by computer program instructions. These computer program instructions may be embodied in a processor of a general-purpose computer, special purpose computer, or other programmable data processing equipment, such that the instructions executed by the processor of the computer or other programmable data processing equipment may be configured in the respective blocks of the block diagram or of the flowchart. Each step creates a means for performing the described functions. These computer program instructions may also be stored in a computer-usable or computer-readable medium that may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus the computer-usable or computer-readable medium. The instructions stored in the recording medium are also possible to produce an item of manufacture including instruction means for performing the functions described in each block of the block diagram or each step of the flowchart. The computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other programmable data processing equipment. It is also possible that instructions for performing the processing equipment provide steps for carrying out the functions described in each block of the block diagram and each step of the flowchart.

Further, each block or each step may represent a module, segment, or portion of code comprising one or more executable instructions for executing specified logical function(s). It should also be noted that in some alternative embodiments it is also possible for the functions recited in blocks or steps to occur out of order. For example, it is possible that two blocks or steps shown one after another may in fact be performed substantially simultaneously, or that the blocks or steps may sometimes be performed in the reverse order according to the corresponding function.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent scope should be construed as being included in the scope of the present invention.

100: deep learning learning device
110: input unit
120: processor unit

Claims (10)

A deep learning learning method performed by a deep learning learning device in a multi-distributed database environment, comprising:
Generating an input stream for model training by merging heterogeneous datasets from a multi-distributed database on a time-series basis;
generating an embedding vector for the input stream;
Comprising the step of adaptively reflecting the embedding vector to at least one of learning of a deep learning model and inference through a pre-learned deep learning model
How to learn deep learning. The method of claim 1,
The reflecting step is
calculating a gradient through the embedding vector;
Using the gradient to learn the parameters of the deep learning model comprising the step of
How to learn deep learning. 3. The method of claim 2,
In the calculating step, pattern-based normalization is applied to calculate the slope, and in the learning step, the slope to which the normalization is applied is used.
How to learn deep learning. The method of claim 1,
The reflecting step is
selecting data whose data characteristics meet the inference processing requirements based on the embedding vector;
re-learning the parameters of the previously learned deep learning model with respect to the selected data;
Comprising the step of performing data inference through the retrained deep learning model
How to learn deep learning. an input unit for receiving heterogeneous datasets from a multi-distributed database;
A processor unit that performs learning or inference of a deep learning model using the heterogeneous dataset,
The processor unit,
Merge the heterogeneous datasets on a time series basis to generate an input stream for model training, generate an embedding vector for the input stream, and use the embedding vector to learn the deep learning model and pre-trained deep learning model through adaptively reflecting at least one of the inferences
Deep learning learning device. 6. The method of claim 5,
The processor unit,
Calculate the gradient through the embedding vector, and learn the parameters of the deep learning model using the gradient
Deep learning learning device. 7. The method of claim 6,
The processor unit,
When calculating the gradient, the gradient is calculated by applying pattern-based regularization, and the parameters of the deep learning model are learned using the gradient to which the normalization is applied.
Deep learning learning device. 6. The method of claim 5,
The processor unit,
Based on the embedding vector, data characteristics that meet the inference processing requirements are selected, the parameters of the pre-trained deep learning model are re-learned on the selected data, and data through the re-trained deep learning model making inferences
Deep learning learning device. As a computer-readable recording medium storing a computer program,
The computer program, when executed by a processor,
Generating an input stream for model training by merging heterogeneous datasets of a multi-distributed database on a time-series basis, generating an embedding vector for the input stream, and learning and prior learning of a deep learning model using the embedding vector A computer-readable recording medium comprising instructions for causing the processor to perform a deep learning learning method comprising the step of adaptively reflecting at least one of inferences through the deep learning model. As a computer program stored in a computer-readable recording medium,
The computer program, when executed by a processor,
Generating an input stream for model training by merging heterogeneous datasets of a multi-distributed database on a time-series basis, generating an embedding vector for the input stream, and learning and prior learning of a deep learning model using the embedding vector A computer program comprising instructions for causing the processor to perform a deep learning learning method comprising the step of adaptively reflecting at least one of inferences through the deep learning model.

Images