JP2020123008A - Variable optimization device, noise elimination device, variable optimization method, noise elimination method, and program - Google Patents

Abstract

To provide a latent variable optimization technique to optimize a latent variable and a coefficient adjusting the relative magnitude of each term in a cost function.SOLUTION: A variable optimization device includes an optimization unit to optimize a latent variable x with observational data as input. f(x) is defined as a loss term of the latent variable x, and y(i=1-n) is defined as a variable (hereinafter referred to as a coefficient variable) to express the coefficient of a normalized term f(x) to the latent variable x. The optimization unit optimizes the latent variable × and the coefficient variable y by solving the optimization problem of the cost function L(x, y):R×R→ RU{-∞, +∞} (where, y=[y-y]) which is defined by the expression L(x, y)=f(x)+Σ(yf(x)+ g(y)) using the normalized term g(y) (i=1- n) to the coefficient variable y.SELECTED DRAWING: Figure 5

Description

The present invention relates to a technique for optimizing a latent variable of a model to be optimized in machine learning.

When designing a map that obtains desired information from observed information (hereinafter referred to as observation information) using machine learning, (1) Suitable for individual problems in a wide range of fields such as image processing, speech recognition, and natural language processing. Both the technique of designing the cost function and the technique (2) of optimizing the latent variable so as to minimize the cost function are required. This cost function is designed in consideration of the generation process of the latent variable (which should be optimized) included in the observation information and the cost function. For example, as in Non-Patent Document 1, x ∈ R ^m is a latent variable to be optimized, and the cost function is expressed as the sum of a plurality of terms as in the following equation.

Where f ₀ is the loss term (also called the penalty term), f _i (i=1, …, n) is the normalization term, and γ _i (>0) (i=1, …, n) is given in advance. Is the calculated coefficient. The loss term f ₀ and the normalization term f _i (i=1,..., N) are both closed true convex functions (hereinafter simply referred to as convex functions).

Then, for example, by inputting the observation information and sequentially optimizing the latent variable x by the optimization algorithm shown in FIG. 1, a latent variable that minimizes the cost function is obtained.

Robert Tibshirani, “Regression Shrinkage and Selection via the Lasso”, Journal of the Royal Statistical Society, Series B (Methodological), Vol.58, No.1, pp.267-288, 1996.

The coefficient γ _i in the equation (1-1) is a parameter for adjusting the relative size of each term, and conventionally, the coefficient γ _i is manually adjusted in advance and the optimization process as shown in FIG. 1 is performed. Was running. However, there is a problem that a desired result cannot be obtained in many cases unless the coefficient γ _i is adjusted carefully. Further, there is also a problem that it becomes difficult to manually adjust when the number of coefficients γ _i to be adjusted increases, for example, when different coefficients γ _i are set for each of the audio frequency spectrogram and the local region of the image. is there.

Therefore, it is an object of the present invention to provide a latent variable optimization technique for optimizing a coefficient for adjusting the relative size of each term of a cost function together with a latent variable.

One aspect of the present invention is a variable optimizing apparatus that includes an optimization unit that optimizes a latent variable x by inputting observation data, and f ₀ (x) is a loss term of the latent variable x, y _i (i = 1, ..., variable representing the coefficients of normalization term f _i (x) for the latent variable x to n) (hereinafter, the referred coefficients variable), the optimization unit normalization term g _i for the coefficient variable y _i Using (y _i ) (i=1, …, n), the cost function L(x, y) defined by the following equation: R ^m ×R ⁿ →R ∪{-∞, +∞}

The latent variable x and the coefficient variable y are optimized by solving the minimization problem of (where y=[y ₁ ,..., Y _n ] ^T ).

According to the present invention, it is not necessary to adjust the coefficient for adjusting the relative size of each term of the cost function in advance, and it is possible to optimize at the same time as the latent variable.

It is a figure which shows an example of the conventional optimization algorithm. It is a figure which shows an example of the optimization algorithm of this application. It is a figure which shows an example of logarithmic concave distribution v(x, y). It is a figure which shows an example of the noise removal algorithm of this application. 3 is a block diagram showing a configuration of a variable optimizing device 100. FIG. 6 is a flowchart showing the operation of the variable optimizing device 100. 3 is a block diagram showing a configuration of an optimization unit 110. FIG. 6 is a flowchart showing the operation of the optimization unit 110. 3 is a block diagram showing a configuration of a noise removing device 200. FIG. 6 is a flowchart showing the operation of the noise removal device 200. 3 is a block diagram showing the configuration of an estimated signal generator 210. FIG. 6 is a flowchart showing the operation of the estimated signal generation unit 210. It is a block diagram showing a configuration of a first variable updating unit 212. 9 is a flowchart showing the operation of the first variable updating unit 212.

Hereinafter, embodiments of the present invention will be described in detail. It should be noted that components having the same function are denoted by the same reference numeral, and redundant description will be omitted.

Prior to the description of each embodiment, the notation method in this specification will be described.

^ (Caret) represents a superscript. For example, x ^{y^z} means that y ^z is a subscript to x, and x _y^z means that y ^z is a subscript to x. Further, _ (underscore) represents a subscript. For example, x ^y_z means that y _z is a subscript to x, and x _{y_z} means that y _z is a subscript to x.

The superscript "^" or "~" such as ^x or ~x for a certain character x should be written directly above "x", but due to the restrictions of the description in the specification. , ^x and ~x.

<Technical background>
In the embodiment of the present invention, the coefficient γ _i which has been treated as a fixed parameter so far is treated as a random variable to which a stochastic assumption is imposed. That is, in the embodiment of the present invention, assuming statistical properties of coefficient gamma _i in advance, it adjusts the coefficient gamma _i automatically. Therefore, in the embodiment of the present invention, the coefficient γ _i is optimized in addition to the latent variable x.

<<Basic form of cost function>>
First, a cost function when the coefficient γ _i is treated as a random variable will be described. Since the coefficient γ _i is treated as a random variable, it will be expressed using a symbol different from γ _i . Specifically, the coefficient that adjusts the relative size of each term of the cost function before the normalization term is represented by y _i (i=1,..., N) and is called a coefficient variable.

In the embodiment of the present invention, both the latent variable x and the coefficient variable y _i (i=1,..., N) are optimized. Therefore, the cost function L is defined by the following equation.

Where L(x, y): R ^m × R ⁿ → R ∪ {-∞, +∞} is a cost function, x ∈ R ^m is a latent variable, y=[y ₁ , …, y _n ] ^T ∈ R ⁿ is a coefficient variable, f ₀ (x) is the loss term of the latent variable x, f _i (x) (i=1, …, n) is the normalization term for the latent variable x, g _i (y _i ) (i =1, …, n) is the normalization term for the coefficient variable y _i . The domain of the normalization term g _i (y _i ) is limited to the positive region. That is, y _i ∈ (0, +∞).

Further, the loss term f ₀ (x) and the normalization term f _i (x), g _i (y _i ) (i=1,..., N) are all convex functions. At this time, the cost function L(x, y) is a convex function related to x when (1) y is fixed and a convex function related to y when (2) x is fixed. That is, the cost function L(x, y) is bi-convex.

Therefore, the fixed point (that is, the solution that minimizes the cost function) is obtained by updating each of the variables {x, y} sequentially so as to minimize L(x, y) as follows. obtain.

Figure 2 shows the optimization algorithm for the variable {x, y} based on Eq. (2-2). Here, T is an integer of 1 or more indicating the number of repetitions. The optimization algorithm shown in FIG. 2 inputs the observation data and sequentially optimizes the latent variable x and the coefficient variable y.

Comparing equations (2-1) and (1-1), it can be seen that there are the following differences regarding the cost function.
(1) In equation (2-1), the coefficient multiplied by the normalization term for the latent variable x is a variable.
(2) In equation (2-1), there are two types of variables to be optimized: latent variable x and coefficient variable y.
In (3) (2-1), regularization term g _i are summed for the coefficient variable y _i. Introducing this term g _i is equivalent to presetting the stochastic assumption for the coefficient variable y _i .

<<Design of cost function related to probabilistic model>>
Next, a guideline on how to design each term constituting the cost function L(x, y) will be described. Specifically, we describe a method of designing the terms that make up the cost function L(x, y) by expressing the statistical properties of the observation data/variable generation process as a probability distribution.

As a class of probability distributions related to convex functions, there is a logarithmic concave distribution (reference non-patent document 1). If the probability distribution used for designing the cost function is restricted to the log-concave distribution, it can be guaranteed that the solution can be uniquely obtained.
(Reference Non-Patent Document 1: "Log-concave probability and its applications", [online], [January 24, 2019 search], Internet <URL: http://econ.ucsb.edu/~tedb/Theory /delta.pdf>)

Since the cost function L(x, y) includes two types of variables {x, y}, three types of log-concave distributions are required depending on the combination. Specifically, the log-concave distribution u _i (x) (i=0, …, k, where k is an integer of 0 or more) containing only the latent variable x, the log-concave containing the latent variable x and the coefficient variable y _i The distribution is v _i (x, y _i ) (i=1,..., N), and the log-concave distribution w _i (y _i ) (i=1,..., N) includes only the coefficient variable y _i .

Where ~φ _i (x) (i=0, …, k), φ _i (x) (i=1, …, n), ψ _i (x) (i=1, …, n), ξ _i (y _i ) (i=1, …, n) is a convex function. Since u _i (x), v _i (x, y _i ), and w _i (y _i ) are probability distributions, they are all normalized. That is, ∫u _i (x)dx=1, ∫v _i (x, y _i )dxdy _i =1, ∫w _i (y _i )dy _i =1.

For log-concave distributions u _i (x) and w _i (y _i ), any log-concave distribution can be used, so it is possible to use any convex function as ~φ _i (x), ξ _i (y _i ). it can. On the other hand, as for the log-concave distribution v _i (x, y _i ), an arbitrary log-concave distribution cannot be used. FIG. 3 shows an example of a logarithmic concave distribution v(x, y)=exp[-(yf(x)+φ(x)+ψ(y))] that can be specifically used. Here, Β(θ ₁ , θ ₂ ) is a beta function and Γ(θ) is a gamma function. The table of FIG. 3 shows combinations of y, f(x), φ(x), and ψ(y) when v(x, y) is in the form of the probability density function p. The second column from the left of the table shows the support of each variable.

Probability distribution u ₀ (x), v _i (x, y _i ) (i=1, …, n), w _i (y _i ) (i =1, …, n). At this time, if φ ₀ (x) is simply expressed as φ ₀ (x), the likelihood function P(x, y) is expressed by the following equation.

Replacing the terms appearing in equation (2-6) with the following equation,

Equation (2-6) is as follows.

Equation (2-7) is the maximum log-likelihood max _x,y logP(x, y) of the generation process when the minimum value min _{x, y} L(x, y) of the cost function L(x, y) is assumed above. Indicates that it matches.

Generation of observation data/variables using the log-concave distribution u ₀ (x), v _i (x, y _i ), w _i (y _i ) defined by Eqs. (2-3) to (2-5) Expressing the process, the negative log-likelihood-logP(x, y) agrees with the cost function L(x, y). Further, the expression (2-2) means a process of fixing one variable (for example, y) and minimizing the cost function with respect to the other variable (for example, x). Therefore, the process corresponds to the process of alternately maximizing the likelihood.

<Application example>
Here, the design of the cost function L(x, y) in the latent variable optimization based on LASSO is described. Consider the problem of inputting an observed signal d ∈ R ^{m in} which noise is mixed and outputting an estimated signal x ∈ R ^m from which noise has been removed. Here, the signals of interest are time domain sounds, still images, and the like. To derive the cost function in this problem, we make some probabilistic assumptions. Hereinafter, each assumption will be described.

(Assumption 1): Assumption regarding the difference between the observed signal d and the estimated signal x It is assumed that the difference between the observed signal d and the estimated signal x follows a Gaussian distribution of variance σ ² given by the following equation.

(Assumption 2): Assumptions about possible values of the estimated signal x It is assumed that the estimated signal x follows a uniform distribution having a certain value in the range c={c _min ,c _max }(c _min ,<c _max ).

Here, x=[x ₁ ,..., X _m ] ^T .

(Assumption 3): Probabilistic assumption for auxiliary variable z of estimated signal x that is a latent variable It is known that the spectrogram of speech has high sparseness in a local region. Also, the adjacent element difference of the image is often highly sparse in the local region.

Therefore, with respect to the voice, the result of performing the discrete Fourier transform (DFT) by performing windowing while frame-shifting is used as the auxiliary variable z. For the image, the result of subtracting adjacent pixels is used as the auxiliary variable z. In either case, the auxiliary variable z is obtained by multiplying the latent variable x by a predetermined matrix D.

Here, the matrix D is a matrix derived from the generation of the observation signal.

If the audio/image is divided into local regions and a total of n local regions are obtained, the auxiliary variable z is also divided into n. This is represented as {z ₁ ,..., Z _n }∈z. The local areas may be overlapped when dividing.

From the sparsity of the auxiliary variable z _i (i=1,..., N), it is assumed that the auxiliary variable z _i follows the Laplace distribution with variance 2β _i ² .

To match the form of equation (2-4), using the substitution of

Equation (2-14) is as follows.

Here, the parameter y _i (i=1,..., N) related to the variance β _i of the Laplace distribution is a coefficient variable.

(Assumption 4): Probabilistic assumption for coefficient variable y _i (Assumption 3) A Laplace distribution is used to express the sparsity of the auxiliary variable z _i , and the parameter y _i related to its variance is treated as a variable. However, it is assumed that the coefficient variables y _i also follow a probability distribution represented by a log-concave distribution.

Here, a Gaussian distribution with limited upper and lower limits is used as an example of a stochastic assumption for the coefficient variable y _i . If the lower and upper bounds of the Gaussian distribution are ζ _i ={ζ _min,i ,ζ _max,i }(ζ _min,i >0, ζ _max,i >0), the mean is μ _i , and the variance is ρ _i ² , The probability distribution w _i is as follows.

Here, erf(•) is an error function.

Therefore, the likelihood function is given by:

From equation (2-9), equation (2-11), equation (2-15), and equation (2-18) relating to the generation process of observation data/variables derived from (assume 1) to (assume 4), The cost function L(x, y), which is a negative log-likelihood, is expressed as follows.

Therefore,

Becomes Here, the latent variable x and its auxiliary variable z are related by the linear equation z=Dx.

《Optimization solver》
The optimization solver of the cost function L(x, z, y) will be described below. By optimizing the variables {x, z, y} alternately, we aim to obtain a fixed point (a solution that minimizes the cost function). Here, an algorithm based on the conventional proximity point method is applied as an optimization solver.

First, with respect to the cost function L(x, z, y), an operator T which is subdifferentiated with respect to each variable is defined.

Then, the fixed point of each variable is obtained by the following equation.

We will update the variable {x, z, y} with the resolvent operator R _T =(I+αT) ⁻¹ (α>0) associated with the subdifferential operator _T.

Since the cost function L(x, z, y) is a bi-convex function, the main variable {x, z} and the dual variable y are updated alternately. Equation (2-22) is reduced to the following two types of iterative operations for constrained minimization problems.

One method for solving the problem of equation (2-23) is the alternating direction multiplier method (ADMM). To solve equation (2-23) based on ADMM, we introduce a new dual variable λ. Therefore, finally, the ADMM-based variable optimization algorithm (optimization solver) is an algorithm that sequentially updates the four types of variables {x, z, y, λ}. The algorithm shown in FIG. 4 is a noise removal algorithm. Note that ε(>0), α(>0), η(>0) are predetermined constants, and T is an integer of 1 or more.

Although concrete experimental data are not shown here, it was confirmed that highly accurate signal estimation can be performed using the noise removal algorithm if the stochastic assumption for the coefficient variables y _i is appropriate.

<First Embodiment>
The variable optimizing device 100 optimizes the latent variable x of the model to be optimized with the observation data as input. Here, the model is a function in which input data is input and output data is output. The observation data is input data used for optimizing the latent variable.

When optimizing the latent variable x, the variable optimizing apparatus 100 sets f ₀ (x) to the loss term of the latent variable x and y _i (i=1,..., N) to the normalization term f _i for the latent variable x. The cost defined by the following equation using the normalization term g _i (y _i ) (i=1, …, n) for the coefficient variable y _i as a variable that represents the coefficient of (x) (hereinafter referred to as the coefficient variable) Optimize the latent variable x and the coefficient variable y by solving the minimization problem of the function L(x, y): R ^m ×R ⁿ →R∪{-∞, +∞}.

(However, y=[y ₁ , ..., y _n ] ^T .)

Hereinafter, the variable optimizing device 100 will be described with reference to FIGS. FIG. 5 is a block diagram showing the configuration of the variable optimization device 100. FIG. 6 is a flowchart showing the operation of the variable optimizing device 100. As shown in FIG. 5, the variable optimizing device 100 includes an optimizing unit 110 and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the variable optimization device 100. The recording unit 190 records, for example, input observation data, latent variables x and coefficient variables y _i to be optimized.

The operation of the variable optimizing device 100 will be described with reference to FIG.

In S110, the optimization unit 110 optimizes the latent variable x and the coefficient variable y by solving the minimization problem of the cost function L with the observation data as input.

Hereinafter, the optimization unit 110 will be described with reference to FIGS. 7 to 8. FIG. 7 is a block diagram showing the configuration of the optimization unit 110. FIG. 8 is a flowchart showing the operation of the optimizing unit 110. As shown in FIG. 7, the optimization unit 110 includes an initialization unit 111, a latent variable update unit 112, a coefficient variable update unit 113, a counter update unit 114, and an end condition determination unit 115.

The operation of the optimization unit 110 will be described with reference to FIG.

In S111, the initialization unit 111 initializes the counter t. Specifically, t=1. Further, the initialization unit 111 initializes the latent variable x and the coefficient variable y.

In S112, the latent variable updating unit 112 updates the latent variable x by the following formula.

In S113, the coefficient variable updating unit 113 updates the coefficient variable y by the following formula.

In S114, the counter updating unit 114 increments the counter t by 1. Specifically, t←t+1.

In S115, the end condition determination unit 115 determines that the counter t has reached the predetermined number of updates T (T is an integer of 1 or more, for example, 100,000) (that is, t>T, and the end condition is satisfied). In the case), the value x of the latent variable and the value y of the coefficient variable at that time are output, and the process ends. Otherwise, the process returns to S112. That is, the optimization unit 110 repeats the processing of S112 to S115.

According to the invention of this embodiment, it is not necessary to adjust the coefficient for adjusting the relative magnitude of each term of the cost function in advance, and it is possible to optimize the coefficient simultaneously with the latent variable. That is, it is not necessary to manually adjust the parameters (for example, coefficients) other than the parameters of the probabilistic model, and therefore the manual adjustment of the parameters can be minimized.

<Second Embodiment>
Here, a noise removal apparatus 200, which is an example in which the variable optimization apparatus 100 is applied to noise removal of an observation signal, will be described. The noise removal apparatus 200 receives an observation signal in which noise is mixed, and generates an estimated signal from which noise has been removed.

Hereinafter, the noise removing device 200 will be described with reference to FIGS. 9 to 10. FIG. 9 is a block diagram showing the configuration of the noise removing apparatus 200. FIG. 10 is a flowchart showing the operation of the noise eliminator 200. As shown in FIG. 9, the noise removal device 200 includes an estimated signal generation unit 210 and a recording unit 290. The recording unit 290 is a component that appropriately records information necessary for the processing of the noise removing apparatus 200. The recording unit 290 records, for example, latent variables that represent the observed signal and the estimated signal that are input.

The operation of the noise eliminator 200 will be described with reference to FIG.

In S210, the estimated signal generation unit 210 is obtained by optimizing the latent variable x, the auxiliary variable z, and the coefficient variable y representing the estimated signal by solving the minimization problem of the cost function L with the observed signal as an input. , A latent variable x representing the estimated signal is generated as an estimated signal from which noise is removed. Where x ∈ R ^m is a latent variable that represents the estimated signal with noise removed, and z is an auxiliary variable of the latent variable x that represents the estimated signal that satisfies z=Dx (where matrix D is the matrix from which the observed signal is generated). , {Z ₁ , …, z _n } ∈ z is the auxiliary variable obtained by dividing the auxiliary variable z into n, and y _i (i=1, …, n) is the auxiliary variable z _i with variance 2(1/y _i ) ² is a coefficient variable obtained when it is assumed to follow the Laplace distribution v _i (z _i , y _i ). Further, the cost function L(x, z, y): R ^m ×R ⁿ ×R ⁿ →R∪{-∞, +∞} is defined by the following equation.

(However, y=[y ₁ , …, y _n ] ^T , u ₀ (x|d, σ ² ) is the Gaussian distribution of variance σ ² that represents the distribution of the difference between the observed signal d and the latent variable x that represents the estimated signal. , U ₁ (x|c) is a uniform distribution in the range c that represents the distribution of the latent variable x that represents the estimated signal, and w _i (y _i |ζ _i ,μ _i ,ρ _i ² ) is the distribution of the coefficient variable y _i The lower and upper limits ζ _i ={ζ _min,i , ζ _max,i }(ζ _min,i >0, ζ _max,i >0), mean μ _i , and Gaussian distribution of variance ρ _i ² )

The estimated signal generator 210 will be described below with reference to FIGS. 11 to 12. FIG. 11 is a block diagram showing the configuration of estimated signal generation section 210. FIG. 12 is a flowchart showing the operation of the estimated signal generator 210. As shown in FIG. 11, the estimated signal generation unit 210 includes an initialization unit 211, a first variable update unit 212, a second variable update unit 213, a counter update unit 214, and an end condition determination unit 215.

The operation of the estimated signal generator 210 will be described with reference to FIG.

In S211, the initialization unit 211 initializes the counter t. Specifically, t=1. The initialization unit 111 also initializes a latent variable x, an auxiliary variable z, a dual variable λ, and a coefficient variable y that represent the estimated signal.

In S212, the first variable updating unit 212 updates the latent variable x, the auxiliary variable z, and the dual variable λ that represent the estimated signal. The first variable updating unit 212 will be described below with reference to FIGS. 13 to 14. FIG. 13 is a block diagram showing the configuration of the first variable updating unit 212. FIG. 14 is a flowchart showing the operation of the first variable updating unit 212. As shown in FIG. 13, the first variable update unit 212 includes a latent variable update unit 2121, an auxiliary variable update unit 2122, a dual variable update unit 2123, and a convergence condition determination unit 2124.

The operation of the first variable updating unit 212 will be described with reference to FIG. Note that α and η are predetermined constants that satisfy α>0 and η>0, respectively.

In S2121, the latent variable updating unit 2121 updates the latent variable x representing the estimated signal by the following equation.

In S2122, the auxiliary variable updating unit 2122 updates the auxiliary variable z by the following equation.

In S2123, the dual variable updating unit 2123 updates the dual variable λ by the following equation.

In S 2124, the convergence condition determination unit 2124, || xx _old representing the difference of the latent variable x before and after the update || ₂ ² is the predetermined value ε (> 0) between the larger repeats the processing S2121～S2124. On the other hand, the convergence condition determination unit 2124, || xx _{if old} || ₂ ² is equal to or less than a predetermined value epsilon, the latent variable x, the auxiliary variable z, and outputs the dual variable lambda, the next processing step (i.e. , S213).

In S213, the second variable updating unit 213 updates the coefficient variable y by the following equation.

In S214, the counter updating unit 214 increments the counter t by 1. Specifically, t←t+1.

In step S215, the termination condition determination unit 215 determines that the counter t has reached the predetermined number of updates T (T is an integer of 1 or more, for example, 100,000) (that is, t>T, and the termination condition is satisfied). In the case), the value x of the latent variable at that time is output as an estimated signal from which noise has been removed, and the process ends. Otherwise, the process returns to S212. That is, the estimated signal generation unit 210 repeats the processing of S212 to S215.

(Modification)
The first variable updating unit 212 that executes the following S212 may be used instead of the first variable updating unit 212. That is, in S212, the first variable updating unit 212 updates the latent variable x and the auxiliary variable z representing the estimated signal by the following equation.

According to the invention of this embodiment, it is possible to generate a highly accurate estimated signal from which noise has been removed.

<Additional notes>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Connectable communication unit, CPU (Central Processing Unit, cache memory and registers may be provided), memory RAM and ROM, hard disk external storage device and their input unit, output unit, communication unit , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged among external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

The external storage device of the hardware entity stores a program necessary for realizing the above-described functions and data necessary for the processing of this program (not limited to the external storage device, for example, the program is read). It may be stored in a ROM that is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in the RAM, the external storage device, or the like.

In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data necessary for the processing of each program are read into the memory as necessary, and interpreted and executed/processed by the CPU as appropriate. .. As a result, the CPU realizes a predetermined function (each constituent element represented by the above,... Unit,... Means, etc.).

The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. Further, the processing described in the above embodiments may be executed not only in time series according to the order described, but also in parallel or individually according to the processing capability of the device that executes the processing or the need. ..

As described above, when the processing functions of the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on the computer, the processing functions of the hardware entity are realized on the computer.

The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, or the like. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disc. Memory), CD-R (Recordable)/RW (ReWritable), etc., as magneto-optical recording medium, MO (Magneto-Optical disc), etc., as semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

The distribution of this program is performed by selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network to distribute the program.

A computer that executes such a program temporarily stores, for example, the program recorded in a portable recording medium or the program transferred from the server computer in its own storage device. Then, when executing the process, this computer reads the program stored in its own storage device and executes the process according to the read program. As another execution form of this program, a computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be sequentially executed. Further, the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing the execution instruction and the result acquisition without transferring the program from the server computer to the computer. May be Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (such as data that is not a direct command to a computer but has the property of defining computer processing).

Further, in this embodiment, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of the processing content may be implemented by hardware.

The foregoing description of the embodiments of the invention has been presented for purposes of illustration and description. I have no intention of being exhaustive or of limiting the invention to the precise form disclosed. Modifications and variations are possible from the above teachings. The embodiments are intended to provide the best illustration of the principles of the invention and to those of ordinary skill in the art in various embodiments and in various ways to suit the actual use contemplated. It has been chosen and expressed so that it can be used with additional transformations. All such variations and variations are within the scope of the invention as defined by the appended claims, which are construed in accordance with the breadth to which they are impartially and legally imparted.

Claims (7)

A variable optimizing device including an optimizing unit for optimizing a latent variable x by inputting observation data,
Let f ₀ (x) be the loss term of the latent variable x, and y _i (i=1, …, n) be a variable that represents the coefficient of the normalization term f _i (x) for the latent variable x (hereinafter referred to as the coefficient variable). ,
The optimization unit is
Using the normalization term g _i (y _i ) (i=1, …, n) for the coefficient variable y _{i, the} cost function L(x, y):R ^m ×R ⁿ →R∪ {-∞, +∞}

A variable optimizer that optimizes the latent variable x and the coefficient variable y by solving the minimization problem of (where y=[y ₁ ,..., Y _n ] ^T ). The variable optimizing device according to claim 1, wherein
The optimization unit is
With the following formula, the latent variable update part that updates the latent variable x,

A coefficient variable updating unit for updating the coefficient variable y by the following equation,

A variable optimizer including. A noise removal apparatus including an estimated signal generation unit that generates an estimated signal from which noise is removed by inputting an observation signal in which noise is mixed,
x ∈ R ^m is a latent variable representing the estimated signal from which the noise is removed, z is an auxiliary variable of the latent variable x representing an estimated signal satisfying z=Dx (where matrix D is a matrix derived from the generation of the observed signal), {z ₁ , …, z _n } ∈ z is an auxiliary variable obtained by dividing the auxiliary variable z into n, and y _i (i=1, …, n) is the auxiliary variable z _i with variance 2(1/y _i ). ² of Laplace distribution _{v i (z i, y i} ) is a coefficient variable obtained when it is assumed to follow,
The estimated signal generating unit is a cost function L(x, z, y) defined by the following equation: R ^m ×R ⁿ ×R ⁿ →R ∪{-∞, +∞}

(However, y=[y ₁ , …, y _n ] ^T , u ₀ (x|d, σ ² ) is the Gaussian distribution of variance σ ² that represents the distribution of the difference between the observed signal d and the latent variable x that represents the estimated signal. , U ₁ (x|c) is a uniform distribution in the range c that represents the distribution of the latent variable x that represents the estimated signal, and w _i (y _i |ζ _i ,μ _i ,ρ _i ² ) is the distribution of the coefficient variable y _i The lower and upper bounds ζ _i ={ζ _min,i ,ζ _max,i }(ζ _min,i >0, ζ _max,i >0), mean μ _i , Gaussian distribution with variance ρ _i ² ) A noise eliminator that solves the problem to generate a latent variable x representing an estimated signal, an auxiliary variable z, and a coefficient variable y obtained as a result of optimizing a latent variable x representing an estimated signal as an estimated signal from which the noise has been removed. .. The noise eliminator according to claim 3,
α and η are predetermined constants,
The estimated signal generation unit,
A first variable updating unit that updates the latent variable x, the auxiliary variable z, and the dual variable λ representing the estimated signal by the following equation,

A second variable updating unit that updates the coefficient variable y by the following equation,

Noise removal device including. A variable optimizing device is a variable optimizing method for performing an optimization step of optimizing a latent variable x by inputting observation data,
Let f ₀ (x) be the loss term of the latent variable x, and y _i (i=1, …, n) be a variable that represents the coefficient of the normalization term f _i (x) for the latent variable x (hereinafter referred to as the coefficient variable). ,
The optimization step is
Using the normalization term g _i (y _i ) (i=1, …, n) for the coefficient variable y _{i, the} cost function L(x, y):R ^m ×R ⁿ →R∪ {-∞, +∞}

A variable optimization method that optimizes the latent variable x and the coefficient variable y by solving the minimization problem of (where y=[y ₁ , …, y _n ] ^T ). A noise removal device is a noise removal method for performing an estimated signal generation step of generating an estimated signal from which noise is removed, using an observation signal in which noise is mixed as an input,
x ∈ R ^m is a latent variable representing the estimated signal from which the noise is removed, z is an auxiliary variable of the latent variable x representing an estimated signal satisfying z=Dx (where matrix D is a matrix derived from the generation of the observed signal), {z ₁ , …, z _n } ∈ z is an auxiliary variable obtained by dividing the auxiliary variable z into n, and y _i (i=1, …, n) is the auxiliary variable z _i with variance 2(1/y _i ). ² of Laplace distribution _{v i (z i, y i} ) is a coefficient variable obtained when it is assumed to follow,
The estimated signal generating step is a cost function L(x, z, y) defined by the following equation: R ^m ×R ⁿ ×R ⁿ →R ∪{-∞, +∞}

(However, y=[y ₁ , …, y _n ] ^T , u ₀ (x|d, σ ² ) is the Gaussian distribution of variance σ ² that represents the distribution of the difference between the observed signal d and the latent variable x that represents the estimated signal. , U ₁ (x|c) is a uniform distribution in the range c that represents the distribution of the latent variable x that represents the estimated signal, and w _i (y _i |ζ _i ,μ _i ,ρ _i ² ) is the distribution of the coefficient variable y _i The lower and upper bounds ζ _i ={ζ _min,i ,ζ _max,i }(ζ _min,i >0, ζ _max,i >0), mean μ _i , Gaussian distribution with variance ρ _i ² ) A noise removal method that generates a latent variable x representing an estimated signal as an estimated signal from which the noise has been removed by solving the problem by optimizing the latent variable x representing the estimated signal, the auxiliary variable z, and the coefficient variable y. .. A program for causing a computer to function as any one of the variable optimizing device according to claim 1 or 2 and the noise removing device according to claim 3 or 4.

Images