KR20220087044A - Method and system for processing missing data in artificial neural network through correction of sparsity bias of zero imputation - Google Patents

Abstract

A method and system for processing missing data that can simply and effectively handle missing data by solving a variable sparsity problem (VSP) through sparsity normalization (SN) are provided.

Description

Method and system for processing missing data in artificial neural network through sparsity bias correction of zero imputation

The following description relates to a method and system for processing missing data in an artificial neural network through sparsity bias correction of zero imputation.

Handling missing data is one of the most fundamental problems in machine learning. Of the many approaches, the simplest and most intuitive is zero imputation, which treats the value of a missing item as simply zero. Numerous studies have experimentally confirmed that zero imputation produces suboptimal performance in training neural networks. However, no existing work has been able to explain what is causing such performance degradation.

[Prior art literature]

Korean Patent Publication No. 10-2020-0115001

A method and system for processing missing data that can simply and effectively handle missing data by solving a variable sparsity problem (VSP) through sparsity normalization (SN) are provided.

A method for processing missing data in a computer device including at least one processor, the method comprising: receiving, by the at least one processor, a first dataset including a plurality of input vectors and a plurality of binary masks and a fixed constant; applying, by the at least one processor, sparsity normalization using the binary mask and the fixed constant to each of the plurality of input vectors; and providing, by the at least one processor, a second dataset including a plurality of input vectors to which the sparsity normalization is applied.

According to one side, the step of applying the sparsity normalization may include multiplying an arbitrary input vector among the plurality of input vectors by the fixed constant, dividing the arbitrary input vector by the size of a vector corresponding to a binary mask corresponding to the arbitrary input vector. It can be characterized by applying sparsity normalization to the input vector of .

According to another aspect, the coordinates of each of the plurality of input vectors may be generated by element-by-element multiplication between a feature value of the input vector and a corresponding binary mask.

According to another aspect, each of the plurality of binary masks may be Missing Completely At Random (MCAR), and may be characterized in that it is independent of a feature value of another binary mask or a corresponding input vector.

According to another aspect, each of the plurality of binary masks may follow the same distribution.

According to another aspect, the fixed constant may be calculated as an average of magnitudes of vectors corresponding to each of the plurality of binary masks.

Provided is a computer program stored in a computer-readable recording medium in combination with a computer device to execute the method on the computer device.

It provides a computer-readable recording medium in which a program for executing the method in a computer device is recorded.

at least one processor implemented to execute computer readable instructions, receiving, by the at least one processor, a first dataset comprising a plurality of input vectors and a plurality of binary masks and a fixed constant; It provides a computer device characterized by applying sparsity normalization using the binary mask and the fixed constant to each of a plurality of input vectors, and providing a second dataset including a plurality of input vectors to which the sparsity normalization is applied. .

By solving the variable sparsity problem through sparsity regularization, missing data can be handled simply and effectively.

1 to 3 are graphs related to a collaborative filtering dataset according to an embodiment of the present invention.
4 to 6 are graphs related to a Long Short Term Memory (LSTM) dataset for an electronic medical record according to an embodiment of the present invention.
7 to 9 are graphs related to a single cell ribonucleic acid (RNA) sequence dataset according to an embodiment of the present invention.
10 and 11 are diagrams illustrating a change in an output value according to a sparsity level of an input according to an embodiment of the present invention.
12 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention.
13 is a flowchart illustrating an example of a missing data processing method according to an embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1. Introduction

Many real-world datasets often contain data instances that do not have a subset of the input features. Various imputations can apply their own strengths and weaknesses, from substituting using global statistics such as mean, to substituting individually by learning an auxiliary model such as a Generative Adversarial Network (GAN), but the simplest and most natural way is to With zero imputation, we simply treat missing features as zeros. In neural networks, at first glance, zero imputation can be considered a reasonable solution because it drops missing input nodes simply by preventing the weights associated with the input nodes from being updated. Surprisingly, however, many previous studies have reported that this intuitive approach has a negative effect on model performance. And none of them investigated the reasons for such poor performance.

1 to 3 are graphs related to a collaborative filtering dataset according to an embodiment of the present invention. The graph of FIG. 1 shows the average of the target values according to the number of known items of the training set as the collaborative filtering dataset. The graph of FIG. 2 shows the predicted values of the model with zero imputation according to the known number of items for randomly selected test points with respect to the collaborative filtering dataset. To artificially control the sparsity level, the input mask can be randomly sampled. At this time, in the graph of FIG. 2 , 50 samples are drawn for each target sparsity level along the x-axis to scatter predicted values, and the average is indicated by a solid line. 3 is a diagram of a method of modifying the plot of FIG. 2 with sparsity normalization. Sparsity normalization will be described in more detail later.

4 to 6 are graphs related to a Long Short Term Memory (LSTM) dataset for an electronic medical record according to an embodiment of the present invention. The graph of Fig. 4 shows the average of the target values according to the number of known items in the training set as the electronic medical record dataset. The graph of Fig. 5 shows the predicted values of the model with zero imputation according to the known number of items for randomly selected test points with respect to the electronic medical record dataset. To artificially control the sparsity level, the input mask can be randomly sampled. At this time, in the graph of FIG. 6 , 50 samples are drawn for each target sparsity level along the x-axis to scatter predicted values, and the average is indicated by a solid line. Fig. 7 is a diagram of a method of modifying the plot of Fig. 6 with sparsity normalization.

7 to 9 are graphs related to a single cell ribonucleic acid (RNA) sequence dataset according to an embodiment of the present invention. The graph of FIG. 7 shows the average of the target values according to the number of known items of the training set as a single-cell RNA sequence dataset. The graph of Figure 8 shows the predicted values of the model with zero imputation according to the known number of items for a randomly selected test point with respect to a single cell RNA sequence dataset. To artificially control the sparsity level, the input mask can be randomly sampled. At this time, in the graph of FIG. 8 , 50 samples are drawn for each target sparsity level along the x-axis to scatter predicted values, and the average is indicated by a solid line. Fig. 9 is a diagram of a method of modifying the plot of Fig. 6 with sparsity normalization.

Hereinafter, it is explained that the output of a neural network varies greatly depending on the number of items missing from the input due to zero imputation. In embodiments of the present invention, this phenomenon is called a Variable Sparsity Problem (VSP), which should be avoided in many practical tasks. For example, consider a movie recommender system. It is not desirable to receive a different average of predicted ratings for different numbers of movies rated (regardless of actual rating values). It can be argued that people with lower ratings generally don't like movies, and it is reasonable to give people with higher ratings a higher predictive value. This may be partly true for users of some sparsity levels, but this is not the general case for a wider range of sparsity levels. This can be seen in a real Collaborative Filtering dataset (see graph in Fig. 1) where users have similar average ratings for the test data regardless of the number of known ratings. However, in a standard neural network with zero imputation, it indicates that the model's inference is correlated with the known number of items in the data instance, such as the graphs of FIGS. 2, 5 and 8 . In some safety critical applications, such as the medical field, it can be fatal. For example, you shouldn't evaluate a patient's probability of developing a disease differently based on the number of medical tests (we don't want a model that predicts a high probability of death just because some patients have been screened a lot!).

In addition, in the embodiments of the present invention, theoretically, the existence of VSP in various situations is analyzed, and the intuitive advantage of zero imputation, that is, a simple yet suggest effective means. This regularization is called "Sparity Normalization (SN)" and shows that it effectively handles VSPs, resulting in greatly improving both the performance and stability of neural network training.

In the following, we identify the causes of the side effects of zero computation, referred to as the variable sparsity problem (VSP), explain how this problem actually affects the training and inference of neural networks, and also explain how this problem is not explicitly explained using VSP. or explain a new perspective to understand a phenomenon that has been misunderstood.

Also, below, we present sparsity normalization (SN) and explain that SN can theoretically solve VSP under certain conditions. In addition, it is explained that VSP can be effectively mitigated or solved just by applying SN.

2. Variable sparsity problem

The variable sparsity problem can be defined as

VSP: A phenomenon in which the expected value of the output layer of a neural network (exceeding weights and input distribution) depends on the sparsity (number of zero values) of the input data.

10 and 11 are diagrams illustrating a change in an output value according to a sparsity level of an input according to an embodiment of the present invention. FIG. 10 shows a neural network using variable sparsity levels, and FIG. 11 shows a neural network using sparsity normalization, respectively. At this time, in FIGS. 10 and 11 , a darker color indicates a larger absolute value and a dotted circle indicates a missing node with zero imputation. Sparsity normalization can make the range of possible outputs of the network more stable compared to the sparsity level.

With VSP, the activation value of the neural network can vary significantly for exactly the same input instance, depending on the number of zero entries. This makes training more difficult and can mislead the model into making erroneous predictions.

Zero imputation is intuitive in that it drops the missing input characteristics, but we show that it causes VSP in cases 1 to 3 below. Specifically, we show VSP under the assumption of increasing generality.

(Case 1) When the activation function is an unbiased identity mapping.

(Case 2) When the activation function is an affine function

(Case 3) When the activation function is a convex function that does not decrease, such as Rectified Linear Unit (ReLU), leaky ReLU, Exponential Linear Unit (ELU), or Softplus.

을 사용하여 i-번째 계층의 가중치 행렬을 나타내고,

는 편향을 나타내며,

는 활성화 벡터를 나타낸다. 단순성을 위해

과

을 각각 사용하여 입력 및 출력 계층을 표현함으로써 다음 수학식 1을 얻을 수 있다.The notation is summarized here for clarity. For an L-layer deep network with nonlinearity,

denote the weight matrix of the i-th layer using

represents the bias,

denotes an activation vector. for simplicity

class

The following Equation 1 can be obtained by expressing the input and output layers using respectively.

(입력 x)의 희소성이 변화함에 따라

의 변화를 관찰하기 위해, 다음과 같은 가정들을 고려한다.

As the sparsity of (input x) changes

To observe the change in , consider the following assumptions.

의 모든 좌표는

과

의 두 랜덤 변수의 요소별 곱에 의해 생성될 수 있다. 여기서

은 누락값을 나타내는 이진 마스크이고

은 (관찰되지 않은) 입력 벡터의 특징값이다. 여기서 누락된 마스크

은 MCAR(missing completely at random)이며, 다른 마스크 변수 또는 해당 값

에 대한 종속성이 없다. 모든

은 동일한 분포를 따르며 기대값은

이고 (ii) 행렬

의 원소는 상호 독립적이며 모두 동일한 분포를 따르고, 각 원소의 기대값은

이평균다. 마찬가지로

와

은 각각 평균

와

를 가진 i.i.d 좌표로 구성된다. (iii)

는 모든 i에 대해 균일하게 0이 아니다.Assumption 1. (i) input vector

All coordinates of

class

It can be generated by element-wise product of two random variables of here

is the binary mask representing the missing values,

is the feature value of the (unobserved) input vector. missing mask here

is missing completely at random (MCAR), and is another mask variable or its value

There is no dependency on every

follows the same distribution and the expected value is

and (ii) the matrix

The elements of is independent of each other and all follow the same distribution, and the expected value of each element is

this is average Likewise

Wow

is the average of each

Wow

i with . i . It consists of d coordinates. (iii)

is uniformly nonzero for all i .

(i) assumes the simplest omission mechanism. (ii) is defined similarly as in the study of well-known weight initialization techniques. (iii) may not hold for some initialization strategies, but is very likely to persist as learning progresses.

의 평균값이 마스크 벡터

의 예상값과 정비례한다는 것을 보여준다.(Case 1) For simplicity, first consider a network without nonlinearity or bias. Theorem 1 is the output layer

The mean value of the mask vector

shows that it is directly proportional to the expected value of

가 항등함수이고 가정 1에서

이 균일하게 0으로 고정된다고 가정하자. 그러면

를 얻는다.Cleanup 1. Activation

is an identity function, and under assumption 1

Assume that this is uniformly fixed to zero. then

to get

는 다음과 같은 방법으로

의 영향을 받는다.(Case 2) If the activation function is affine, but there is a non-zero potential for bias,

in the following way

is affected by

는 가정 1에 따른 affine 함수라고 가정한다. 더 나아가

는

로 정의된다고 가정하자. 그러면

이다.Cleanup 2. Activation

Assume that is an affine function according to assumption 1. Furthermore

Is

Assume that it is defined as then

to be.

는

와 관련된 일부 수량에 의해 하한선을 보인다는 것을 보여줄 수 있다.(Case 3) Finally, if the activation function is non-linear but not decreasing and convex,

Is

It can be shown that there is a lower bound by some quantity related to

는 가정 1 하에서 감소하지 않는 볼록함수라고 가정한다. 더 나아가

가

및

으로 정의된다고 가정하자. 그러면

이다.Arrangement 3.

Is Assume that it is a convex function that does not decrease under assumption 1. Furthermore

go

and

Assume that it is defined as then

to be.

If the expected value of the output layer (or the lower bound of the output layer) depends on the level of sparsity/omission, as in Theorem 1-3, even similar instances of data may have different outputs depending on the level of sparsity, which is a fair and accurate reasoning of the model. may interfere with As shown in the graphs of FIGS. 2, 5 and 8 , VSP can easily occur even in the actual setting of neural network training in which the above condition is not maintained.

3. Sparsity Normalization

(

는 요소별 곱을 나타냄)의 표기법을 상기하면, 고정 상수

에 대해

을 통한 단순 정규화는 입력 희소성에 대한 의존성을 떨어뜨릴 수 있다는 것을 알 수 있다. 이미 설명한 바와 같이 이 간단한 정규화 기법의 이름을 희소성 정규화(Sparsity Normalization, SN)라 정하고 아래 표 1의 알고리즘 1에 기술한다.Below we propose a simple but surprisingly effective way to solve VSP. Since the linearity of the activation simplifies the calibration, we first revisit (Case 2) to find a way to make the expected output independent of the input sparsity level.

(

Recalling the notation of element-by-element product), the fixed constant

About

It can be seen that simple regularization through As already described, the name of this simple regularization technique is called Sparsity Normalization (SN), and it is described in Algorithm 1 of Table 1 below.

Conceptually, this method adjusts the size of each input value according to the sparsity, so that the change in the output size becomes less sensitive to the sparsity as shown in FIG. 2(b). The official explanation for the correction of sparsity bias by SN is as follows in this particular case.

가 가정 1에 따른 affine 함수라고 가정한다. 더 나아가

라고 가정하고 SN을 사용하여 입력 계층을 교체한다. 즉, 고정 상수

에 대해

이다. 그러면

을 얻는다.Arrangement 4. Activate (using sparsity regularization)

Assume that is an affine function according to assumption 1. Furthermore

, and use SN to replace the input layer. That is, a fixed constant

About

to be. then

to get

으로부터 평균 활성화가 독립되도록 만든다.

는 일반적으로 유지되지 않기 때문에 SN을 이용하여 (사례 3)의 카운터파트를 보여주는 것은 사소한 일이 아니다. 그러나 광범위한 실험을 통해 보다 일반적인 경우에도 SN이 실질적으로 효과적이라는 것을 알 수 있었다.Unlike in Theorem 2, in Theorem 4, SN determines the sparsity of the input.

make the mean activation independent from

It is not trivial to show the counterpart of (Case 3) using the SN, since in general is not maintained. However, extensive experiments have shown that SN is practically effective even in the more general case.

이 모든 데이터 인스턴스에 걸쳐 알려져 있고 고정되어 있다고 가정하지만, 실제로 이러한 가정을 완화하고 데이터 인스턴스들 간에 다양한

을 고려할 수 있다. 특히, 최대우도 원칙에 의해 각 인스턴스에 대해

으로

을 추정할 수 있다. 따라서

(알고리즘 1 참조)에서

을 얻는다. 실제로, 훈련 세트의 모든 예에 대해

를

의 평균으로 사용하는 것을 권장한다.

가 너무 작으면(예:

= 1) 다잉(dying) ReLU 현상이 나타날 수 있다. 하이퍼 파라미터

는 경사(gradient) 크기 조절을 통해 정규화 효과를 가져올 수 있으므로

를 정의하여 평균 스케일이 정규화를 전후하여 일정하게 유지되도록 하며, SN에 의해 야기되는 부작용을 최소화할 수 있다.Theorem 4

It is assumed that these assumptions are known and fixed across all data instances, but in practice we relax these assumptions and vary between data instances.

can be considered. In particular, for each instance by the maximum likelihood principle,

by

can be estimated. therefore

(see Algorithm 1)

to get In fact, for every example in the training set,

cast

It is recommended to use the average of

is too small (e.g.

= 1) Dying ReLU phenomenon may appear. hyperparameter

can bring normalization effect through gradient scaling, so

is defined so that the average scale is kept constant before and after normalization, and side effects caused by SN can be minimized.

12 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. The missing data processing system according to embodiments of the present invention may be implemented by at least one computer device 1200 or a plurality of computer devices (each corresponding to the computer device 1200 ). For example, a computer program according to an embodiment may be installed and driven in the computer device 1200 , and the computer device 1200 processes missing data according to embodiments of the present invention under the control of the driven computer program. method can be performed.

As shown in FIG. 12 , the computer device 1200 may include a memory 1210 , a processor 1220 , a communication interface 1230 , and an input/output interface 1240 . The memory 1210 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a non-volatile mass storage device such as a ROM and a disk drive may be included in the computer device 1200 as a separate permanent storage device distinct from the memory 1210 . Also, an operating system and at least one program code may be stored in the memory 1210 . These software components may be loaded into the memory 1210 from a computer-readable recording medium separate from the memory 1210 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory 1210 through the communication interface 1230 instead of a computer-readable recording medium. For example, the software components may be loaded into the memory 1210 of the computer device 1200 based on a computer program installed by files received through the network 1260 .

The processor 1220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 1220 by memory 1210 or communication interface 1230 . For example, the processor 1220 may be configured to execute a received instruction according to a program code stored in a recording device such as the memory 1210 .

The communication interface 1230 may provide a function for the computer device 1200 to communicate with other devices (eg, the storage devices described above) through the network 1260 . For example, a request, command, data, file, etc. generated by the processor 1220 of the computer device 1200 according to a program code stored in a recording device such as the memory 1210 is transmitted to the network ( 1260) to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 1200 through the communication interface 1230 of the computer device 1200 via the network 1260 . A signal, command, or data received through the communication interface 1230 may be transferred to the processor 1220 or the memory 1210 , and the file may be a storage medium (described above) that the computer device 1200 may further include. persistent storage).

The input/output interface 1240 may be a means for an interface with the input/output device 1250 . For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or a speaker. As another example, the input/output interface 1240 may be a means for an interface with a device in which functions for input and output are integrated into one, such as a touch screen. The input/output device 1250 may be configured as one device with the computer device 1200 .

Also, in other embodiments, the computer device 1200 may include fewer or more components than those of FIG. 7 . However, there is no need to clearly show most of the prior art components. For example, the computer device 1200 may be implemented to include at least a portion of the above-described input/output device 1250 or may further include other components such as a transceiver and a database.

13 is a flowchart illustrating an example of a missing data processing method according to an embodiment of the present invention. The missing data processing method according to the present embodiment may be performed by the computer device 1200 implementing the missing data processing system. In this case, the processor 1220 of the computer device 1200 may be implemented to execute a control instruction according to a code of an operating system included in the memory 1210 or a code of at least one computer program. Here, the processor 1220 causes the computer device 200 to perform steps 1310 to 1330 included in the method of FIG. 13 according to a control command provided by a code stored in the computer device 1200 . can control

에 대응할 수 있다. 이 경우, 복수의 입력 벡터 각각은

에 대응할 수 있으며, 복수의 이진 마스크 각각은

에 대응할 수 있다. 또한, 고정 상수는

에 대응할 수 있다. 여기서, 복수의 입력 벡터 각각의 좌표는 입력 벡터의 특징값 및 대응하는 이진 마스크간의 요소별 곱에 의해 생성될 수 있다. 특징값은 앞서 설명한

에 대응할 수 있으며, 이러한 특징값과 이진 마스크간의 요소별 곱은

로 표현될 수 있다. 이때, 복수의 이진 마스크 각각은 MCAR(Missing Completely At Random)이고, 다른 이진 마스크 또는 대응하는 입력 벡터의 특징값에 대해 독립적일 수 있다. 또한, 복수의 이진 마스크 각각은 복수의 이진 마스크의 평균과 동일한 분포를 따르도록 얻어질 수 있다. 또한, 고정 상수는 복수의 이진 마스크 각각에 대응하는 벡터의 크기의 평균으로 계산될 수 있다. 앞서, 고정 상수가

와 같이 정의될 수 있음을 설명한 바 있다.In operation 1310 , the computer device 200 may receive a first dataset including a plurality of input vectors and a plurality of binary masks and a fixed constant. This first dataset is the dataset shown in Algorithm 1 of Table 1 above.

can respond to In this case, each of the plurality of input vectors is

can correspond to, and each of the plurality of binary masks is

can respond to Also, the fixed constant is

can respond to Here, the coordinates of each of the plurality of input vectors may be generated by element-by-element multiplication between a feature value of the input vector and a corresponding binary mask. The feature values are

, and the element-by-element product between these feature values and the binary mask is

can be expressed as In this case, each of the plurality of binary masks is a Missing Completely At Random (MCAR), and may be independent of a feature value of another binary mask or a corresponding input vector. Also, each of the plurality of binary masks may be obtained to follow the same distribution as the average of the plurality of binary masks. Also, the fixed constant may be calculated as an average of magnitudes of vectors corresponding to each of the plurality of binary masks. Earlier, the fixed constant

It has been explained that it can be defined as

에 대응될 수 있다.In operation 1320 , the computer device 200 may apply sparsity normalization using a binary mask and a fixed constant to each of the plurality of input vectors. For example, the computer device 200 multiplies an arbitrary input vector among a plurality of input vectors by a fixed constant and divides it by the magnitude of a vector corresponding to a binary mask corresponding to the arbitrary input vector to perform sparsity normalization on the arbitrary input vector. can be applied This process is described in the algorithm in Table 1.

can correspond to

에 대응할 수 있다. 이처럼, 각 입력값의 크기를 그 희소성에 따라 조절함으로써, 출력 크기의 변화가 희소성에 덜 민감해지도록 희소성 정규화된 데이터셋을 얻을 수 있게 된다.In operation 1330 , the computer device 200 may provide a second dataset including a plurality of input vectors to which sparsity normalization is applied. This second dataset is the sparsity normalized dataset shown in Algorithm 1 of Table 1.

can respond to In this way, by adjusting the size of each input value according to its sparsity, it is possible to obtain a sparsity-normalized dataset so that changes in the output size are less sensitive to sparsity.

As such, according to embodiments of the present invention, it is possible to simply and effectively process missing data by solving the variable sparsity problem through sparsity normalization.

The system or apparatus described above may be implemented as a hardware component or a combination of a hardware component and a software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The medium may continuously store a computer executable program, or may be a temporary storage for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributedly on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store for distributing applications, sites supplying or distributing other various software, and servers. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (14)

A method for processing missing data in a computer device comprising at least one processor, the method comprising:
receiving, by the at least one processor, a first dataset including a plurality of input vectors and a plurality of binary masks and a fixed constant;
applying, by the at least one processor, sparsity normalization using the binary mask and the fixed constant to each of the plurality of input vectors; and
providing, by the at least one processor, a second dataset including a plurality of input vectors to which the sparsity normalization is applied;
A method of handling missing data, including According to claim 1,
The step of applying the sparsity normalization comprises:
Sparsity normalization is applied to the arbitrary input vector by multiplying an arbitrary input vector among the plurality of input vectors by the fixed constant and dividing by the size of a vector corresponding to a binary mask corresponding to the arbitrary input vector How to handle missing data. According to claim 1,
The method for processing missing data, characterized in that the coordinates of each of the plurality of input vectors are generated by element-by-element multiplication between a feature value of the input vector and a corresponding binary mask. According to claim 1,
Each of the plurality of binary masks is a Missing Completely At Random (MCAR), and is independent of a feature value of another binary mask or a corresponding input vector. According to claim 1,
Each of the plurality of binary masks follows the same distribution. According to claim 1,
The fixed constant is calculated as an average of magnitudes of vectors corresponding to each of the plurality of binary masks. A computer program stored in a computer-readable recording medium in combination with a computer device to cause the computer device to execute the method of any one of claims 1 to 6. A computer-readable recording medium in which a computer program for executing the method of any one of claims 1 to 6 in a computer device is recorded. at least one processor implemented to execute computer-readable instructions
including,
by the at least one processor;
receiving a first dataset including a plurality of input vectors and a plurality of binary masks and a fixed constant;
applying sparsity normalization using the binary mask and the fixed constant to each of the plurality of input vectors;
Providing a second dataset including a plurality of input vectors to which the sparsity normalization is applied
A computer device comprising a. 10. The method of claim 9,
by the at least one processor;
applying sparsity normalization to the arbitrary input vector by multiplying an arbitrary input vector among the plurality of input vectors by the fixed constant and dividing by the magnitude of a vector corresponding to a binary mask corresponding to the arbitrary input vector
A computer device comprising a. 10. The method of claim 9,
The coordinates of each of the plurality of input vectors are generated by element-by-element multiplication between the feature values of the input vectors and the corresponding binary masks.
A computer device comprising a. 10. The method of claim 9,
Each of the plurality of binary masks is a Missing Completely At Random (MCAR), and is independent of a feature value of another binary mask or a corresponding input vector.
A computer device comprising a. 10. The method of claim 9,
each of the plurality of binary masks follows the same distribution
A computer device comprising a. 10. The method of claim 9,
The fixed constant is calculated as an average of the magnitudes of vectors corresponding to each of the plurality of binary masks.
A computer device comprising a.

Images