JP5713986B2 - Encoding apparatus, method, program, and recording medium - Google Patents

Description

  The present invention relates to a technique for encoding information.

  The probability of an actual value that is a sequence of a plurality of values is a probability corresponding to the probability that the actual value appears in a predetermined first probability distribution, and the actual value that is a sequence of all the multiple values that satisfy a predetermined condition A technique for generating a random number sequence according to the second probability distribution from the above probability distribution and a technique for encoding information based on this technique has not been proposed.

  An object of the present invention is to provide an encoding apparatus, method, program, and recording medium for encoding information based on a technique for generating a sequence of random numbers according to the second probability distribution.

  An encoding apparatus according to an aspect of the present invention sets a probability for an actual value that is a sequence of a plurality of values as a probability corresponding to a probability that the actual value appears in a predetermined first probability distribution, and satisfies all the predetermined conditions The probability distribution for the realized value that is a series of a plurality of values is defined as the second probability distribution, and the third probability that is the probability for the realized value is the condition when the already generated random number sequence is used as a condition. Product of the first probability of occurrence and the second probability that the sequence of random numbers that have already been generated and the random number that is generated this time satisfy the predetermined condition A probability distribution generation unit that generates a third probability distribution that is a distribution of the third probability for all possible real values, and a random number generator that generates one real value as a random number according to the third probability distribution Part and probability distribution students A random number sequence generation device including a control unit that generates a sequence of random numbers according to the second probability distribution by repeating the processing of the unit and the random number generation unit, and using the generated random number sequence and a predetermined second function A compression unit that generates a codeword, and the predetermined condition is a condition expressing a relationship between predetermined shared information and a sequence of a plurality of values using a predetermined first function, and a predetermined first probability distribution Is a conditional probability that is conditional on the information to be encoded.

  Information can be encoded based on a technique for generating a sequence of random numbers according to the second probability distribution.

The block diagram for demonstrating the example of a random number series production | generation apparatus. The block diagram for demonstrating the other example of a random number series production | generation apparatus. The flowchart for demonstrating the example of the random number series production | generation method. The block diagram for demonstrating the example of an encoding apparatus and a decoding apparatus. The block diagram for demonstrating the example of an encoding apparatus and a decoding apparatus. The flowchart for demonstrating the example of the encoding method. The flowchart for demonstrating the example of a decoding method.

[Description of notation]
First, the notation is explained. When X is a random variable, the set of possible values of X is denoted as [X]. The size of the set S, that is, the number of elements included in the set S is denoted as | S |. The difference set is denoted as S \ S ′, which represents a set that is included in the set S and is composed of all the elements that are not included in the set S ′.

X ⁿ ≡ (X ₁ , X ₂ ,..., X _n ) is an n-dimensional random variable, and its realization value is denoted as x ⁿ ≡ (x ₁ , x ₂ ,..., X _n ).
[X ⁿ ] ≡ [X ₁ ] × [X ₂ ] ×… × [X _n ]. The probability distribution corresponding to the random variable X is denoted as p _X. It should be noted, it is sometimes referred to as p_ the p _{X (X).}

Φ generally represents a function, and Φ (x ⁿ ) represents the value of the function Φ in the vector x ⁿ . If q is a prime number and the function Φ: GF (q) ⁿ → GF (q) ^m is a linear mapping of the vector space over the finite field GF (q) of q,

It is expressed as a matrix. Where x ⁿ is a column vector

It becomes. In this case, Φ (x ⁿ ) is denoted as Φx ⁿ .

  If it is not explicitly stated that the set of possible values of a variable is finite, continuous values can be taken. If the variable for which the sum Σ is taken is a continuous value, Σ is an integral ∫.

[A simple random number generation method that satisfies the constraints]
Φ (x ⁿ ) ∈V ₁ × V ₂ ×… × V _l for a set of functions Φ≡ {φ _s : [X ⁿ ] → V _s } _{s = 1} ^l

It is defined as Given the distribution p _X ^{n of} v ^l ≡ (v ₁ , v ₂ , ..., v _l ) and the random variable X ⁿ ≡ (X ₁ , X ₂ , ..., X _n ), Consider μ (x ⁿ ).

When the predetermined condition is Φ (x ⁿ ) = v ^l and the predetermined first probability distribution is p _X ⁿ , the probability μ (x for the real value x ⁿ ≡ (x ₁ , x ₂ , ..., x _n ) ⁿ ) is a probability corresponding to the probability that the actual value x ⁿ appears in the predetermined first probability distribution p _X ⁿ .

Here, any one real value x ⁿ according to the distribution (second probability distribution) of the probability μ (x ⁿ ) for all real values satisfying the predetermined condition Φ (x ⁿ ) = v ^l is used as a sequence of random numbers. Two simple methods to generate are described.

<Method 1>
First, a set Ω that enumerates x ⁿ satisfying Φ (x ⁿ ) = v ^l is obtained. Then, for each x ⁿ ∈Ω, the relational expression

The probability distribution μ is obtained by calculating the right side of (a2) using. Finally, x ⁿ ∈Ω is selected according to the probability distribution μ.

In <Method 1>, the input is p _X ⁿ , the function set Φ and the vector v ^l , and the output is a random number x ⁿ .

<Method 2>
Select x ⁿ ∈ [X ⁿ ] according to the probability distribution p _X ⁿ . Then, x ⁿ to see if it meets the conditions ^{Φ (x n) = v l} . This is repeated until x ⁿ satisfying Φ (x ⁿ ) = v ^l is selected. Finally, x ⁿ satisfying this condition is output as a sequence of random numbers, and the process ends.

In <Method 2>, the input is p _X ⁿ , the function set Φ and the vector v ^l , and the output is a random number x ⁿ .

In <Method 1>, it is necessary to obtain a set Ω that enumerates x ⁿ satisfying Φ (x ⁿ ) = v ^l , but in general, | Ω | increases exponentially as n increases. It can be difficult to find. Since this fact makes it difficult to calculate (a3), it may be difficult to obtain the probability distribution μ. Moreover, even if the probability distribution μ is obtained, it may be difficult to select x ⁿ ∈Ω according to μ. Furthermore, when p _X ⁿ is fixed in advance, Ω and μ can be calculated in advance and recorded in the storage unit, but in this case, the storage amount may increase exponentially with n. is there.

In <Method 2>, random numbers are repeatedly generated until x ⁿ satisfying Φ (x ⁿ ) = v ^l is selected. Generally, the probability of selecting x ⁿ satisfying Φ (x ⁿ ) = v ^l is an exponent together with n. Since it approaches zero, the number of iterations may increase exponentially.

[First Embodiment of Random Number Sequence Generation Apparatus and Method]
In the following, a set of functions Φ≡ {φ _s : [X ⁿ ] → V _s } _{s = 1} ^l and v ^l ≡ (v ₁ , v ₂ ,…, v _l ) are given, and Φ (x ⁿ ) ∈V A random number sequence generation device that generates a sequence x ⁿ of random numbers according to the probability distribution μ of (a2) when ₁ × V ₂ ×... × V _l is defined by (a1) will be described.

T (s) ⊂ {1,2, ..., n} is used to calculate the function φ _s (x ⁿ ) and the set of indices of variables x ⁿ ≡ (x ₁ , x ₂ , ..., x _n ) To do. Let _{xT (s) be} a | T (s) | -dimensional vector obtained by extracting a portion corresponding to the index set T (s) from ^xn . In the following, focusing on only variables used in calculating the function φ _s (x ⁿ ), φ _s (x ⁿ ) is ^denoted as φ _s (x _{T (s)} ). For example, if the variables used when calculating the function φ _s (x ⁿ ) are x ₁ and x ₂ , T (s) = {1,2}, φ _s (x _{T (s)} ) ≡φ _s (x ₁ , x ₂ ) = φ _s (x ⁿ ) is satisfied.

Χ (x _{T (s)} , v _s ) determined from Φ≡ {φ _s : [X ⁿ ] → V _s } _{s = 1} ^l is defined as follows.

The probability distribution p _X ⁿ is decomposed as follows.

  Here, when t = 1 on the right side of the above equation

And For example, if p _X ⁿ is memoryless,

When p _X ⁿ has δ-order Markov property

It becomes.
As shown in FIG. 1, the random number sequence generation device 1 includes, for example, a control unit 11, a probability distribution generation unit 12, a random number generation unit 13, a storage unit 14, and a random number estimation unit 15.

In order to generate random numbers according to the probability distribution μ defined in (a2), the random number sequence generation device executes the following processing. In the following, Σ represents the sum of all possible values of the variable written below. For example, Σ _xt is the sum over all x _t ∈ [X _t ].

  The control unit 11 sets k = 1 (step A1, FIG. 3).

The probability distribution generation unit 12 obtains a function f _k (x _k ) defined by the following equation (step A2). The function f _k (x _k ) can be obtained, for example, by executing a sum-product algorithm. The sum-product algorithm will be described later.

In the right side of the above equation, x ₁ ,..., X _k−1 recorded so far are substituted.

Further, if a record in which χ (x _{T (s)} , v _s ) is a constant is recorded, future calculations can be omitted. For example, if T (s) is {1, ..., k}, the value of χ (x _{T (s)} , v _s ) is 0 or 1 regardless of the values of x _{k + 1} , ..., x _n And become a constant. In this case, a constant χ (x _{T (s)} , v _s ) is recorded in the storage unit 14 for use in the next and subsequent iterations.

x ₁ , ..., x _k-1 is a sequence of already generated random numbers, a random number generated this time is x _k, and a sequence of remaining random numbers generated after the next time is x _{k + 1} , ..., x _n Then, it appears in the molecule on the right side of (a5)

Can be said to be the first probability that the realization value x _t appears when the sequence of generated random numbers x ₁ ,..., X _t−1 is used as a condition.

  It also appears in the molecule on the right side of (a5)

Is a sequence of random numbers x ₁ ,..., X _k−1 that have already been generated and a random number x _k that is generated this time, and the remaining random number sequences x _{k + 1} ,. It can be said that x _n is a second probability that satisfies a predetermined condition (in this case, Φ (x ⁿ ) = v ^l ).

Since the denominator of (a5) is a normalized term, for example, f _k (x _k ) represented by (a5) is a probability distribution whose real value is a value corresponding to the product of the first probability and the second probability. It can be said.

The random number generation unit 13 generates a random number x _k according to the probability distribution f _k (x _k ) and records it in the storage unit 14 (step A3).

  The controller 11 determines whether k = n (step A4).

If k = n, the control unit 11 outputs x ⁿ ≡ (x ₁ , x ₂ ,..., x _n ) obtained by concatenating the values of the variables recorded so far, and ends (step A5).

If k <n, the control unit 11 determines the remaining variable values (x _{k + 1} ,..., x) satisfying Φ (x ⁿ ) = v ^l from (x ₁ ,..., x _k ) recorded so far. It is confirmed whether or not x _n ) is uniquely determined (step A6).

If If the only determined, the random number estimating unit 15 _{(x k + 1, ...,} x n) x n ≡ (x 1, x 2, ..., x n) seeking to terminate with a (step A7 ). An example of a criterion for confirming whether or not it is determined and an example of how to obtain (x _{k + 1} ,..., X _n ) in this case will be described in the second embodiment of the random number sequence generation device and method.

  If not uniquely determined, the controller 11 increases the value of k by 1 and returns to step A2 (step A8).

  In this manner, the control unit 11 repeats the processing of the probability distribution generation unit 12 and the random number generation unit 13 to generate a random number sequence according to the probability distribution μ.

Note that the processing of step A6 and step A7 may not be performed. For example, if it takes time to confirm whether (x _{k + 1} , ..., x _n ) is uniquely determined or if this confirmation cannot be performed, it is also determined (x _{k + 1} , ..., x _{If the} process for obtaining _n ) takes time or cannot be performed, the confirmation process may be omitted, and if k <n, the value of k may be increased by 1 and the process may return to step A2. In this case, the random number sequence generation device 1 may not include the random number estimation unit 15 as illustrated in FIG.

The input of the random number sequence generator 1 is a set of probability distributions {p_ (X _t | X ₁ ^t-1 )} _{t = 1} ⁿ and a set of functions {χ (x _{T (s)} , v _s )} _{s = 1} ^l And the output of the random number sequence generation device 1 is ^xn . As described above, the storage unit 14 may record data in which χ (x _{T (s)} , v _s ) becomes a constant in addition to the input.

When calculating (a5) in step A2, the variable of interest (x _{k in the} above) can be freely selected from the variables (x _k , x _{k + 1} ,..., X _n ). Therefore, in step A2, a marginalization function that is convenient for later calculations can be used from the result of obtaining the marginalization function for all variables (x _k , x _{k + 1} ,..., X _n ). . This means that the order of marginalizing the variables (x ₁ ,..., X _n ) can be adaptively changed in step A2. Here, it should be noted that re-calculation of p_ (X _n | X ₁ ^t−1 ) appearing on the right side of (a5) may be required in accordance with the change of the order of the variables of interest. However, when p _X ⁿ is memoryless, that is, satisfies (a4), the variable to be noted among the variables (x _k , x _{k + 1} ,..., X _n ) when calculating (a5) in step A2. Even if (x _{k in the} above) is changed, it is not necessary to recalculate p_ (X _t | X ₁ ^t−1 ) = p _{Xt according to} the change of the order of the variables of interest.

Note that a random number generation algorithm that satisfies the constraint conditions may be applied after the conditional expression is transformed into a conditional expression that is equivalent to the conditional expression Φx ⁿ = v ^l and that improves the calculation accuracy of the sum-product algorithm.
In order to increase the approximation accuracy of sum-product, for example, the condition is set so that the size of the set T (s) becomes small and the factor graph corresponding to the conditional expression Φx ⁿ = v ^l does not have a short loop. It is conceivable to change the formula.

In order to generate a desired random number in step A2, x _k may be obtained from the uniform random number sequence using the interval algorithm described in Reference Document 1. In Reference Document 1, it is assumed that the generated random number sequence has independent and same distribution. However, since this assumption does not hold here, the following improvements are applied. Now, suppose that there is a sufficiently long uniform ω-element random number sequence r ₁ , r ₂ ,. This random number sequence r ₁ , r ₂ ,... Is generated by, for example, the uniform random number generation unit 16 indicated by a broken line in FIG. Below
[ζ, η) ≡ {θ: ζ ≦ θ <η}
And

  [Reference 1] [11] TS Han and M. Hoshi, “Interval algorithm for random number generation,” IEEE Trans. Inform. Theory, vol. IT-44, no. 2, pp. 599-611, Mar. 1997 .

In step A1, the initial value of the section is set to [ζ ₁ , η ₁ ) ≡ [0,1).

In step A3, the section [ζ _k , η _k ) is divided into partial sections of length {f _k (x)} _{x∈ [Xk]} , and the section corresponding to the length f _k (x) is labeled x∈ [ Add X _k ]. Then, the corresponding label and ω advance decimal 0.r ₁ r ₂ ... section, including the x _k. At the end of step A2, a section corresponding to the previously obtained x _k , that is, a section including the ω-adic decimal 0.r ₁ r ₂ ... Is set as [ζ _{k + 1} , η _{k + 1} ).

With this improvement, the width of the interval corresponding to x ⁿ = (x ₁ , ..., x _n ) becomes μ (x ⁿ ), so the interval corresponding to (x ₁ , ..., x _n ) The probability that the obtained ω-adic decimal 0.r ₁ r ₂ ... Is included is μ (x ⁿ ). Here, when the algorithm stops at k = k ′, random numbers are generated only until (x ₁ ,..., X _{k ′} ), but the division of the section at the time point of k = k ′ corresponds to (a11) described later. Therefore, a random number sequence (x ₁ ,..., X _n ) according to a desired probability distribution is obtained.

Assuming that the sum-product algorithm is applied at step A2 and executed at O (n), and assuming that the calculation time at step A6 and step A7 is O (n ² ), this algorithm is End with the computation time of O (n ² ). For this reason, this algorithm can generate a random number sequence in a shorter calculation time than the above-mentioned [simple random number sequence generation method that satisfies the constraint conditions].

  In the following, it is proved that this algorithm outputs the expected result. The expression that appears in the numerator on the right side of (a5)

Then,

Is true.

Let x ⁿ = (x ₁ , x ₂ ,..., x _n ) be the finally output random number sequence. If it ends with k = n in step A4 of the algorithm,

It becomes. On the other hand, when k = k ′ <n ends in step A6 and step A7 of the algorithm, the remaining variables satisfying Φ (x ⁿ ) = v ^l from (x ₁ , x ₂ ,..., X _{k ′} ) Since the value of (x _{k '+ 1} , ..., x _n ) is only determined,

It becomes. Since (a8) includes (a7) as the case of k ′ = n, the case where it ends with k = k ′ will be considered below.

  When k ≧ 2 from (a6)

Holds. And when k = 1

Holds. The probability of finally outputting x ⁿ from (a8), (a9), and (a10) is

It becomes. Therefore, this algorithm outputs a random number sequence x ⁿ according to the probability distribution (a2).

<About the sum-product algorithm>
In general, the sum-product algorithm means that when a function e _s (x _{T (s)} ) determined by a variable vector x _{T (s)} consisting of a part of a variable vector x ⁿ is given by each s∈S, Function

It is a procedure to ask for. Here, T (s) ⊂N for each s∈S, where N≡ {1, 2,..., N}. Σ _{xn \ {xt}} is the sum when all the values of x ⁿ other than the variable x _t appearing in the vector x ⁿ are moved. Also, it appeared in the numerator on the right side instead of g (x _t ) defined in (a12)

In some cases, a sum-product algorithm is described as a procedure for obtaining. However, hereinafter, a procedure for obtaining g (x _t ) defined in (a12) is referred to as a sum-product algorithm.

The sum-product algorithm is described below. Functions π _{xt → es} (x _t ) and σ _{es → xt} (x _t ) called messages are defined as follows.

Here, Σ _{x_ (T (s) \ {t})} is the sum when all {x _{t '} } _{t'∈T (s) \ {t}} are moved, and Σ _{x_ (T (s) )} Represents the total sum when all {x _t } _{t∈T (s)} are moved. And when T (s) = {t}

When there is no s'∈S \ {s} that satisfies x _t ∈T (s')

It is determined.
Hereinafter, specific processing of the sum-product algorithm will be described.
Step 1 For t∈N, s∈S satisfying T (s) = {t}

And initialize.
_{Step 2 x t ∈T (s'} ) as s'∈S\ {s} does not exist that meets the T∈N, against s∈S

And initialize.
Step 3 For t∈N and s∈S satisfying t∈T (s), if π _{xt → es} (x _t ) is not initialized in Step 2, set an appropriate initial value. For example, if the set of possible values of x _t is [X _t ],

And
Step 4 Find σ _{es → xt} (x _t ) by (a13) for all t∈N, s∈S that satisfy t∈T (s). Here, for the combination of s and t initialized in Step 1, the calculation of σ _{es → xt} (x _t ) is omitted.
Step 5 Find π _{xt → es} (x _t ) by (a14) for all t∈N, s∈S that satisfy t∈T (s). Here, for the combination of s and t initialized in Step 2, the calculation of π _{xt → es} (x _t ) is omitted.
Step 6 Step 4 and Step 5 are repeated a predetermined number of times, and then the following function is obtained and finished.

  When using the sum-product algorithm for the random number generation algorithm that satisfies the constraints, in (a5)

Is applied as {e _s } s∈S. The right side of (a15) obtained after execution of the sum-product algorithm is an approximate expression of f _k (x _k ).

[Second Embodiment of Random Number Sequence Generation Apparatus and Method]
The second embodiment of the random number sequence generation device and method is a random number sequence generation device and method when the function has linearity. Below, it demonstrates centering on a different part from 1st embodiment of a random number sequence generation apparatus and method, and it abbreviate | omits duplication description about the part similar to 1st embodiment of a random number sequence generation apparatus and method.

In the following, [X ₁ ] = [X ₂ ] =… = [X _n ] = [V ₁ ] = [V ₂ ] =… = [V _l ] = GF (q), where l ≦ n, Full rank l × n matrix

think of. Function φ _s : [X ⁿ ] → V _s

It is defined as S-th row of the n-dimensional vector of the right side of the above equation is a matrix _{Φ (φ s, 1, φ} s, 2, ..., φ s, n) and _{^{x n ≡ (x 1, x}} 2, ..., x n ) ∈ [X _n ] inner product.

Φ≡ {φ _s } _{s = 1} ^l and vector defined as above

The specific processing of the random number sequence generation device 1 for generating a distribution determined by the probability distribution (a2) will be described.

Here, Φ and v ^l may use the given matrix and vector as they are, but the following matrix connecting Φ and v ^l

Result of applying row basic transformation to

Left l × n matrix Φ'and right l × 1 matrix (l-dimensional vector) v ^'l, may be substituted as Φ and v ^l, respectively. For example, by using Φ ′ and v ′ ^l such that the factor graph determined from Φ ′, x ^n, and v ′ ^l does not have a short loop due to many components that are 0 in the matrix Φ ′, the sum-product algorithm is used. Since the approximation accuracy may be improved, the accuracy of the probability distribution of the random numbers output by the random number sequence generation device of the present invention can be further improved.

Hereinafter, even if it is assumed that Φ is a full rank matrix and the l × l matrix Ψ ₂ on the right side of the matrix Φ is a regular matrix, the generality is not lost. In practice, a matrix obtained by removing redundant Φ rows and v ^l components from an arbitrary matrix Φ and vector v ^l is obtained, and the row vector and variable (x ₁ , x ₂ , ..., A desired matrix can be obtained by appropriately rearranging the order of x _n ).

Assuming that the l × (nl) matrix on the left side of the matrix Φ is Ψ ₁ , from the above assumption, if x ^nl ≡ (x ₁ , ..., x _nl ) is determined
(x _{n-l + 1} ,…, x _n ) ≡Ψ ₂ ^-1 (v ^l -Ψ ₁ x ^nl )
The remaining variables (x _{n−l + 1} ,..., X _n ) satisfying Φx ⁿ = v ^l can be obtained. Here, Ψ ₂ ⁻¹ represents an inverse matrix of Ψ ₂ and can be obtained before the algorithm is executed.

T (s, k) ⊂ {k, ..., n} is a set of indices s satisfying φ _{s, t} ≠ 0 among vectors (φ _{s, k} , ..., φ _{s, n} ) appearing in the matrix Φ To do. That is, T (s, k) is defined as follows.
T (s, k) ≡ {t∈ {k,…, n}: φ _{s, t} ≠ 0}

Here, χ (x _{T (s, k)} , v _s (k)) is defined as follows.

  With the above settings, the specific processing of the random number sequence generation device 1 can be changed as follows.

The control unit 11 sets k = 1. Let _s (k) = v _s for each s∈ {1, ..., l}.

The probability distribution generation unit 12 obtains a function f _k (x _k ) defined by the following equation (step A2). The function f _k (x _k ) can be obtained, for example, by executing a sum-product algorithm.

Random number generator 13 generates a x _k with probability distribution f _k (x _k), and records in the storage unit (step A3).

  The controller 11 determines whether k = n (step A4).

If k = n, the control unit 11 outputs x ⁿ ≡ (x ₁ , x ₂ ,..., x _n ) obtained by concatenating the values of the variables recorded so far, and ends (step A5).

If k <n, the control unit 11 determines the remaining variable values (x _{k + 1} ,..., x _n ) satisfying Φx ⁿ = v ^l from (x ₁ ,..., x _k ) recorded so far. To see if it is the only one. In this example, the control unit 11 determines whether k = l (step A6).

If k = l, the random number estimator 15 calculates and records the remaining variable values (x _{l + 1} ,..., x _n ) ≡Ψ ₂ ⁻¹ v ^l (k + 1), and so far The recorded x ⁿ ≡ (x ₁ , x ₂ ,..., X _n ) is output and the process ends (step A7).

If k <l, the control unit 11 increases the value of k by 1, and returns to step A2 (step A8). When returning to step A2, v _l (k + 1) ≡ (v ₁ (k + 1),..., V _l (k + 1)) is expressed as s∈ {1,.
v _s (k + 1) ≡v _s (k) -φ _{s, k} x _k
And

In the above specific processing, the processing of step A4 and step A5 may be omitted, and the processing may proceed immediately from step A3 to step A6. Alternatively, if it takes time to calculate Ψ ₂ ⁻¹ v ^l (k + 1) in step A7, the processing of step A6 and step A7 may be omitted, and the process may proceed immediately from step A4 to step A8.

The input of the random number sequence generation device 1 of the second embodiment is a set of probability distributions {p_ (X _t | X ₁ ^t−1 )} _{t = 1} ⁿ and a matrix Φ, and the output is x ⁿ . In addition to the input q and information q necessary for calculating GF (q), Ψ ₂ ⁻¹ can be obtained from Φ and recorded in the storage unit 14. (x _{T (s, k)} , v _s (k)) is updated with Φ using k.

[Third embodiment of random number sequence generation apparatus and method]
The third embodiment of the random number sequence generation device and method is a random number sequence generation device and method when p _X ⁿ is a conditional probability distribution. Below, it demonstrates centering on a different part from 1st embodiment of a random number sequence generation apparatus and method, and it abbreviate | omits duplication description about the part similar to 1st embodiment of a random number sequence generation apparatus and method.

Given a conditional probability distribution p_ (X ⁿ | U ⁿ ), it satisfies the constraints,

When generating random numbers according to

And the specific process of the random number sequence generation device is executed. That is, for example, (a5) is expressed by the following equation.

The input of the random number sequence generation device 1 of the third embodiment is a set of probability distributions {p_ (X _t | X ₁ ^t−1 U ⁿ )} _{t = 1} ⁿ , a set of functions Φ, v ^l , u ⁿ , The output is ^xn .

  In addition to input, the storage unit 14

The information necessary for calculating is stored and used for calculating the probability distribution.

Similarly to the above, p _X ⁿ may be a conditional probability distribution also in the second embodiment of the random number sequence generation device and method.

[Notes on embodiments regarding information compression that allows distortion]
Before describing [Embodiment relating to information compression allowing distortion and embodiment relating to information compression and decoding apparatus and method] which is an embodiment relating to information compression allowing distortion, [Encoding apparatus relating to information compression allowing distortion] And embodiment of the decoding apparatus and method].

  Information compression that allows distortion is information compression in the case where mismatch between the output of the information source and the output of the decoding device is allowed.

In the following, the base of the logarithm is unified to 2 etc., for example.
A and B generally represent functions, and A (x ⁿ ) and B (x ⁿ ) represent values in the vector x ⁿ of the functions A and B, respectively. The ranges of the functions A and B are written as ImA≡ {A (x ⁿ ): x ⁿ ∈ [X ⁿ ]} and ImB≡ {B (x ⁿ ): x ⁿ ∈ [X ⁿ ]}, respectively.

In the following embodiment, a set [A], [B] of functions having [X ⁿ ] as a domain is considered. Im [A] and Im [B] are set as follows.

[A] and [B] are used only for describing the conditional expression, and in the encoding, functions A and B are selected one by one from [A] and [B], respectively. [X ⁿ ], ImA, ImB, Im [A], and Im [B] may or may not be a linear space. When the factor graph determined by the variable x ⁿ and the functions A and B has a sparse structure, particularly when the functions A and B are expressed by a sparse matrix, the approximation performance by the sum-product algorithm can be improved. Further, in the decoding configuration, the maximum likelihood decoding in the random number sequence estimator is approximated by using, for example, the sum-product algorithm described in References 2 and 3 or the linear programming described in References 4 and 5, for example. Can be realized.

[Reference 2] SM Aji and RJ McEliece, “The generalized distributive law,” IEEE Trans. Inform. Theory, vol. 46, no. 2, pp. 325-343, Mar. 2000.
[Reference 3] FR Kschischang, BJ Frey, and HA Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Transactions on Information Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
[Reference 4] J. Feldman, MJ Wainwright, and DR Karger, “Using linear programming to decode binary linear codes,” IEEE Trans. Inform. Theory, vol. IT-51, no. 3, pp. 954-972, Mar. 2005.
[Reference 5] T. Wadayama, “An LP decoding algorithm based on primal path-following interior point method,” Proc. 2009, Int. Symp. Inform. Theory, Seoul, Korea, pp. 389-393, 2009.

When [X ⁿ ] is an n-dimensional vector space over a finite field GF (q) and the functions A and B used for encoding are matrices whose elements are elements of the finite field GF (q), [A] Is a set consisting of the entire m × n matrix, and [B] is a set consisting of the entire m ′ × n matrix,

Therefore, the conditional expression regarding log | Im [A] |, log | Im [B] | that appears later can be converted into the conditional expression regarding q, n, m, m ′. A is an m-dimensional vector and b is an m′-dimensional vector.

The conditional entropy of X ⁿ when the entropy Y ⁿ random variables X ⁿ given are defined as follows.

  When applying a random number sequence generation method that satisfies a constraint condition, if a random number that satisfies a constraint condition determined using a conditional probability distribution is required, [random number sequence generation apparatus and method first embodiment] to [random number sequence] The method described in the third embodiment of the generation apparatus and method may be applied.

“Recorded function” means that a means for calculating the function is recorded. If the function has linearity, the coefficients should be recorded in the form of a matrix. The fact that the probability distribution is recorded means that means for calculating the probability of the actual value is recorded. The means for calculating the function and the means for calculating the probability of the actual value are implemented by, for example, a program as will be described later. For example, if a parametrized continuous distribution such as a Gaussian distribution, a steady memorylessness, or a Markov property is assumed, a means for calculating a probability with a small capacity can be recorded. When a constrained random number generation algorithm is used, it can be replaced with information necessary for calculating f _k (x _k ).

  In the following, a general flow of data in an embodiment relating to information compression that allows distortion will be described.

The output y ⁿ ≡ (y ₁ , y ₂ ,..., Y _n ) of the information source is an element of the set [Y ⁿ ], and y ⁿ is output from the information source. The compression handled here is a block code, and encoding is performed with y ⁿ as one block. Assume that the set [Y ⁿ ] is not necessarily a finite set.

The predetermined shared information a is an element of the set Im [A], and when A is an m × n matrix, a is an m-dimensional vector. Encoding apparatus obtains the code word b is data compressed from a source output y ⁿ and a, are stored in a transmission using a communication path with no error, or a memory or the like. Here, b is an element of Im [B], and when B is an m ′ × n matrix, it is an m′-dimensional vector. Note that ^xn calculated in the middle of encoding is an element of a finite set [ ^Xn ].

The decoding apparatus obtains ξ ⁿ from b and a. Here, ξ ⁿ is an element of the finite set [X ⁿ ]. However, if the decoding apparatus observes the output u ⁿ of the auxiliary information, the decoding apparatus obtains the w ⁿ from b and a and u ^n. u ⁿ is an element of [U ⁿ ], ξ ⁿ is an element of [X ⁿ ], and w ⁿ is an element of [W ⁿ ]. It is assumed that u ⁿ and w ⁿ are not necessarily finite sets.

[Embodiment relating to encoding apparatus and method and decoding apparatus and method for information compression allowing distortion]
<When decoding device does not observe u ^n>
Hereinafter, with reference to FIG. 4, a configuration in the case where the decoding apparatus 200 does not observe the output u ⁿ of the information source correlated with the output y ⁿ of the information source will be described.

  As illustrated in FIG. 4, the encoding device 100 includes, for example, a random number sequence generation device 1, a storage unit 101, and a compression unit 103. The decoding device 200 includes a storage unit 201 and a random number sequence estimation unit 202, for example.

In the storage unit 101, functions A and B, predetermined shared information a, and probability distribution p _X ⁿ _{| Y} ⁿ are recorded. The random number sequence generation device 1 and the compression unit 103 read these recorded information from the storage unit 101 as necessary, and perform processing described later.

In the storage unit 201, functions A and B, predetermined shared information a, and probability distribution p _X ⁿ are recorded. The random number sequence estimation unit 202 reads the recorded information from the storage unit 201 as necessary, and performs processing described later.

  Here, [A] and [B] are set so as to satisfy the following relationship.

Let x ⁿ ∈ [X ⁿ ], y ⁿ ∈ [Y ⁿ ]. _{Let the} distortion scale be d _n : [X ⁿ ] × [Y ⁿ ] → [0, ∞). ^Let p _Y ⁿ be the probability distribution that characterizes the information source. The conditional probability distribution p _X ⁿ _{| Y} ⁿ is a design parameter and can be given arbitrarily, but (H (X ⁿ ) -H (X ⁿ | Y ⁿ )) and the mean strain E _X ⁿ defined below Select a parameter with a small _Y ⁿ [d _n (x ⁿ , y ⁿ )].

Further, p _X ⁿ (x ⁿ ) is defined as follows.

The output y ^{n of the} information source is input to the random number sequence generation device 1 of the encoding device 100.

The random number sequence generation device 1 generates a random number sequence x ⁿ according to the following probability distribution μ for the output y ^{n of the} information source and the predetermined shared information aεIm [A] (step Ce1, FIG. 6). The random number sequence x ⁿ is transmitted to the compression unit 103.

The random number sequence generation device 1 can generate the random number sequence x ⁿ by applying the method described in [Third embodiment of the random number sequence generation device and method]. Here, in step Ce1, one selected from a plurality of random number sequences generated according to the probability distribution μ may be selected as x ⁿ .

The compression unit 103 generates a codeword b using the random number sequence ^xn and the predetermined second function B (step Ce3). Specifically, the compression unit 103 outputs b≡B (x ⁿ ) as a code word. That is, B (x ⁿ ) is calculated and the calculation result is set as a code word b.

  The codeword b, which is the output of the encoding device 100, is input to the random number sequence estimation unit 202 of the decoding device 200.

The random number sequence estimation unit 202 obtains an estimated value ξ ⁿ of x ⁿ defined by the following equation from the code word b and the predetermined shared information a (step Cd1, FIG. 7). Here, instead of obtaining (c1), ξ ⁿ is approximated using, for example, the sum-product algorithm described in References 2 and 3 or the linear programming described in References 4 and 5, for example. You may ask for it.

  A specific example is described below. The function A is expressed as an m × n matrix and the function B is expressed as an m ′ × n matrix as follows.

Here, it is assumed that the matrices A and B are full rank. The m-dimensional vector a is expressed as follows.

Let the function Φ be an m × n matrix, and let l≡m be a vector v ^l as follows:

Φ is the same as the matrix A, and v ^l is the same as the vector a. For the sequence y ⁿ to be compressed, U ⁿ ≡Y ⁿ , u ⁿ ≡y ⁿ , the encoding apparatus 100 generates the distribution μ by the method described in [third embodiment of random number sequence generation apparatus and method]. Random number x ⁿ

And the following m′-dimensional vector b is defined as a codeword b which is a sequence after compression.

Here, the conditional expression Φ (θ ⁿ ) = v ^l is as follows.

The decoding apparatus 100 obtains an estimated value ξ ⁿ of x ⁿ from (c1) from the code word b which is a compressed sequence.

Here, the conditional expression A (θ ⁿ ) = a is as follows.

Conditional expression B (θ ⁿ ) = b is as follows.

  Other specific examples will be described below.

Assume that q = 2, n = 7, and [X ⁿ ] ≡GF (2) ⁷ and [Y ⁿ ] ≡GF (2) ⁷ . d _n (x ⁿ , y ⁿ ) is determined as follows.

  Assume the probability distribution of the information source as follows.

  Here, η is a real number between 0 and 1. A conditional probability distribution is defined as follows.

Here, ζ ₀ and ζ ₁ are real numbers from 0 to 1. At this time, the following expression is satisfied.

The probability distribution p_ (X _t | X ₁ ^t−1 Y ⁿ ) is given by

The probability distribution p _X ⁿ is as follows.

When η = 3/7, ζ ₀ = 1/7, ζ ₁ = 5/7, the following results.

  Assuming that functions A and B are matrices and the number of columns is m = 4 and m ′ = 2, the following equations are satisfied. Here, the base of the logarithm is 2.

  A 4 × 7 matrix A and a 2 × 7 matrix B are defined as follows.

  A four-dimensional vector a is set as follows.

From now, the following 7-dimensional vector y ⁿ to be compressed is encoded.

  Let Φ = A. That is, Φ is set as follows.

Also, let l≡m = 4 and let the vector v ^l = a. That is, it is assumed that ^vl is set as follows.

In generating random numbers according to [Second Embodiment of Random Number Sequence Generating Apparatus and Method], Ψ ₁ and Ψ ₂ are determined as follows.

Since Ψ ₂ is a regular matrix, the inverse matrix Ψ ₂ ⁻¹ is as follows.

  Here, the specific processing of [second embodiment of random number sequence generation apparatus and method] is performed.

1. In Step A1, set k = 1. v _s (1) is as follows.

2. In Step A2, T (s, 1), χ (x _{T (s, 1)} , v _s (1)) is as follows.

  function

By calculating

Get. In this example, the sum-product method is not used.

  3. Probability distribution in Step A3

Generate a random number 0 or 1. Hereinafter, it is assumed that 0 is generated, and x ₁ = 0.

4. Set k = 2. v _s (2) is as follows.

  Then, return to Step A2.

5. In Step A2, T (s, 2), χ (x _{T (s, 2)} , v _s (2)) is as follows.

  function

By calculating

Get. In this example, the sum-product method is not used.

  6. Probability distribution in Step A3

Generate a random number 0 or 1. Hereinafter, it is assumed that 1 is generated, and x ₂ = 1.

7. Set k = 3 in Step A4. v _s (3) is as follows.

  Then, return to Step A2.

8. In Step A2, T (s, 3), χ (x _{T (s, 3)} , v _s (3)) is as follows.

  function

By calculating

Get. In this example, the sum-product method is not used.

  9. Probability distribution in Step A3

Generate a random number 0 or 1. Hereinafter, it is assumed that 0 is generated, and x ₃ = 0.

10. Here, v _s (4) is as follows.

Here, (x ₄ , x ₅ , x ₆ , x ₇ ) is obtained as follows.

The random number generator outputs x ⁿ = (0,1,0,0,0,1,0) and ends.

The encoding device obtains the code word b from the output x ⁿ = (0,1,0,0,0,1,0) of the random number generator.

The decoding apparatus that receives the codeword b = (1,0) first estimates x ⁿ

Result in an estimate of x ⁿ

Get.
Note that, in the compression unit 103, instead of using the function B, a variable length code may be configured by using the method described in Reference 6 using an arithmetic code. Specifically, in step A3 constraints satisfying the random number sequence generation algorithm, the caused x _k

A code word based on an arithmetic code that can be assigned the ideal code length is obtained, and the code word obtained when the algorithm is completed is output. The decoding device performs decoding using a decoding device corresponding to the arithmetic device of the arithmetic code used by the encoding device. Here, (c2) can be calculated using a sum-product algorithm. Note that the probability distribution used to generate the random number sequence x ⁿ is p _X ⁿ _{| Y} ⁿ , and the probability distribution used to obtain the codeword is p _X ⁿ . Decoding apparatus, (c2) sequentially sought by obtaining x ^k with ideal code length from the code word (c2), the x ⁿ can be reproduced without error. When the algorithm stops at k <n, the remaining variables (x _{k + 1} , ..., x _n ) satisfying A (x ⁿ ) = a are determined from (x ₁ , ..., x _k ). Note that the codeword corresponding to (x _{k + 1} ,..., x _n ) is not necessary. The difference between the proposed method and the method described in Reference 6 is that the method described in Reference 6 uses a sum-product method or linear programming to obtain a distorted sequence and then applies an arithmetic coding algorithm. On the other hand, in the proposed method, an arithmetic code algorithm is applied to a randomly generated random number ^xn .
[Reference 6] J. Honda and H. Yamamoto, “Variable length lossy coding using LDPC code,” Proc. 2009 Int. Symp. Inform. Theory, Seoul, Korea, pp. 1973-1977.

In this case, the storage unit 101 stores a function A, predetermined shared information a, and probability distributions p _X ⁿ _{| y} ⁿ and p _X ⁿ _{| y} ⁿ . The random number sequence generation device 1 and the compression unit 103 read these recorded information from the storage unit 101 as necessary, and perform processing described later. The input of the encoding device 100 is y ⁿ and the output of the encoding device 100 is b.

In the storage unit 201, a function B, predetermined shared information a, and a probability distribution p _X ⁿ are recorded. The random number sequence estimation unit 202 reads the recorded information from the storage unit 201 as necessary, and performs processing described later. The input of the decoding device 200 is b, and the output of the decoding device 200 is ξ ⁿ .

<When decoding apparatus observes u ^n>
Hereinafter, a configuration in which the decoding apparatus 200 can observe the output u ⁿ of the information source correlated with the output y ⁿ of the information source will be described.

  As illustrated in FIG. 4, the encoding device 100 includes, for example, a random number sequence generation device 1, a storage unit 101, and a compression unit 103. The decoding device 200 includes, for example, a storage unit 201, a random number sequence estimation unit 202, and a conversion filter unit 204.

In the storage unit 101, functions A and B, predetermined shared information a, and probability distribution p _X ⁿ _{| Y} ⁿ are recorded. The random number sequence generation device 1 and the compression unit 103 read these recorded information from the storage unit 101 as necessary, and perform processing described later.

In the storage unit 201, functions A and B, predetermined shared information a, and probability distributions p _X ⁿ _{| U} ⁿ and p _W ⁿ _{| X} ⁿ _U ⁿ are recorded. The random number sequence estimation unit 202 and the conversion filter unit 204 read the recorded information from the storage unit 201 as necessary, and perform processing described later.

  Here, [A] and [B] are set so as to satisfy the following relationship.

^Let u ⁿ ∈ [U ⁿ ], w ⁿ ∈ [W ⁿ ], x ⁿ ∈ [X ⁿ ], y ⁿ ∈ [Y ⁿ ]. _{Let the} distortion scale be d _n : [W ⁿ ] × [Y ⁿ ] → [0, ∞). ^Let p _Y ⁿ _U ⁿ be the probability distribution that characterizes the information source. The conditional probability distribution p _X ⁿ _{| Y} ⁿ , p _W ⁿ _{| X} ⁿ _U ⁿ is a design parameter and can be given arbitrarily, but (H (X ⁿ | U ⁿ ) -H (X ⁿ | Y ⁿ ) ) And the average distortion E _W ⁿ _Y ⁿ [d _n (w ⁿ , y ⁿ )] defined below are selected.

Further, p _X ⁿ _{| U} ⁿ (x ⁿ | u ⁿ ) is defined as follows.

The output y ^{n of the} information source is input to the random number sequence generation device 1 of the encoding device 100.

The random number sequence generation device 1 generates a random number sequence x ⁿ according to the following probability distribution μ for the output y ^{n of the} information source and the predetermined shared information aεIm [A] (step Ce1, FIG. 6). The random number sequence x ⁿ is transmitted to the compression unit 103.

The random number sequence generation device 1 can generate the random number sequence x ⁿ by applying the method described in [Third embodiment of the random number sequence generation device and method]. Here, in step Ce1, one selected from a plurality of random number sequences generated according to the probability distribution μ may be selected as x ⁿ .

The compression unit 103 generates a codeword b using the random number sequence ^xn and the predetermined second function B (step Ce3). Specifically, the compression unit 103 outputs b≡B (x ⁿ ) as a code word. That is, B (x ⁿ ) is calculated and the calculation result is set as a code word b.

  The codeword b, which is the output of the encoding device 100, is input to the random number sequence estimation unit 202 of the decoding device 200.

The random number sequence estimation unit 202 obtains an estimated value ξ ⁿ of x ⁿ defined by the following equation from the code word b, the output u ^{n of the} information source, and the predetermined shared information a (step Cd1, FIG. 7). ). ξ ⁿ is transmitted to the conversion filter unit 204.

Instead of obtaining ξ ⁿ by the above formula, ξ ⁿ is approximated using, for example, the sum-product algorithm described in References 2 and 3 or the linear programming described in References 4 and 5, for example. You may ask for.

Conversion filter 204, the distribution p _W ⁿ _| follow _X ⁿ _U ^n, an output of the decoding device 200 the random or deterministic sequence obtained through conversion ^{F w n ≡F (ξ n,} u n).

  In the prior art according to the prior art described in Reference 7, the calculation time may increase exponentially as the block length n increases.

  [Reference 7] J. Muramatsu and S. Miyake, “Hash property and coding theorems for sparse matrices and maximal-likelihood coding,” IEEE Trans. Inform. Theory, vol. IT-56, no. 5, pp. 2143- 2167, May 2010.

On the other hand, in the generation of the random number sequence in [Embodiment relating to encoding apparatus and method and information decoding apparatus and method for information compression allowing distortion], for example, in step A2, a sum-product algorithm is applied to generate O ( If it is assumed that the calculation time is executed in step n) and the calculation time of step A6 and step A7 is O (n ² ), the calculation time of encoding and decoding is O (n ² ). Thus, by generating a sequence of random numbers in polynomial time, each of encoding and decoding can be performed in polynomial time.

  In the following, it will be proved that the error probability of information compression allowing distortion according to [Embodiment relating to encoding apparatus and method and decoding apparatus and method for information compression allowing distortion] converges to zero. In the following proof, [1] to [5] indicate the following documents.

[1] TM Cover and JA Thomas, “Elements of Information Theory 2nd. Ed.,” John Wiley & Sons, Inc., 2006.
[2] JL Carter and MN Wegman, “Universal classes of hash functions,” J. Comput. Syst. Sci., Vol. 18, pp. 143-154, 1979.
[3] J. Muramatsu and S. Miyake, “Hash property and coding theorems for sparse matrices and maximal-likelihood coding,” IEEE Trans. Inform. Theory, vol. IT-56, no. 5, pp. 2143 {2167, May 2010. Corrections: vol.IT-56, no.9, p. 4762, Sep. 2010.
[4] J. Muramatsu and S. Miyake, “Construction of Slepian-Wolf source code and broadcast channel code based on hash property,” available at arXiv: 1006.5271 [cs.IT], 2010.
[5] J. Muramatsu and S. Miyake, “Construction of strongly secure wiretap channel code based on hash property,” Proc. Of 2011 IEEE Int. Symp. Inform. Theory, St. Petersburg, Russia, Jul. 31-Aug. 5, 2011, pp.612-616.

  First, the hash property will be described.

1.1 Definition and Lemma Let φ be an empty set. A set of functions whose domain is [U ⁿ ] is denoted as [A]. ImA and Im [A] are defined as follows.

The sets C _A (a) and C _AB (a, b) are defined as follows.

  A random variable having a function A∈ [A] and a∈ImA are written as As and as, respectively. In formulas, As and as may be expressed using A and a in the sans serif font, respectively.

Note that a random variable whose value is an n-dimensional vector u ⁿ ∈ [U ⁿ ] is written as U ⁿ .

Definition 1. [A _n] the function A _n: a set of ^{[U n] → Im [A} n], consider the sequence _{{[A n]} n =} 1 ∞ set. [A _n] takes a p _{As, n} random variables on the sequence _{{([A n], p} As, n)} n = 1 ∞ is called ensemble. {([A _n ], p _{As, n} )} _{n = 1} ^∞ for {p _{As, n} } _{n = 1} ^∞ real sequence {α _As (n)} _{n = 1} ^∞ , {β _As (n)} _{n = 1} ^∞ , and for any n and u ⁿ ∈ [U ⁿ ]

as well as

{([A _n ], p _{As, n} )} _{n = 1} ^∞ satisfies {α _As (n), β _As (n)} _{n = 1} ^∞ -hash. Thereafter, [A], p _As , α _As , and β _As depend on n, but are omitted when n does not move.

For example, when [A] has 2-universal hash [2] and p _As is a uniform distribution on [A], {([A _n ], p _{As, n} )} _{n = 1} ^∞ is {(1,0)} _{n = 1} ^∞ -Hashes. _{{(1,0)} n = 1} ∞ is alpha _As ≡1 in any n, a string to be β _As ≡0. The hash of the ensemble of linear mapping (matrix) will be explained in the next section.

  The following lemma holds.

Lemma 1 ([4, Lemma 4]). {([A _n ], p _{As, n} )} _{n = 1} ^∞ , ({([B _n ], p _{Bs, n} )} _{n = 1} ^∞ ) {α _As (n), β _As (n)} _{n = 1} ^∞ _-An ensemble having hash properties ({α _Bs (n), β _Bs (n)} _{n = 1} ^∞ -hash properties). The function (A, B) ∈ [A] × [B] is defined as follows.

The probability distribution p _AsBs on [A] × [B] is defined as follows.

(α _AsBs , β _AsBs ) is defined as follows.

At this time, the new ensemble ([A _n ] × [B _n ], p _{AsBs, n} )} _{n = 1} ^∞ becomes {α _AsBs (n), β _AsBs (n)} _{n = 1} ^∞ _-hash Have.

Lemma 2 ([4, Lemma 1]). When ([A], p _As ) satisfies (H3), the following expression holds for any T, T′⊂ [U ⁿ ].

Lemma 3 ([3, Lemma 1]). When ([A], p _A ) satisfies (H3 ′), the following equation holds for any T ⊂ [U ⁿ ], u ⁿ ∈ [U ⁿ ].

Lemma 4 ([5, Lemma 4]). When ([A], p _As ) satisfies (H3), the following equation holds for an arbitrary function Q: [U ⁿ ] → [0, 1) and T⊂ [U ⁿ ].

  Here, Q (T) is defined as follows.

1.2 Hash properties of ensembles consisting of linear maps
^Let [U ⁿ ] ≡GF (q) ⁿ and [U ^l ] ≡GF (q) ^l . ^Let [A] be a set of linear mapping A: [U ⁿ ] → [U ^l ]. A can be expressed as an l × n matrix.

^Let t (u ⁿ ) be the type of u ⁿ ∈ [U ⁿ ] (relative frequency distribution of u ⁿ ). Let [H] be a set consisting of all types of sequences of length n except t (0 ⁿ ). S (p _As , t) is defined as follows for the probability distribution pAs and the type t∈ [H] on the set of the entire l × n matrix.

When a set [H '] ⊂ [H] is given, α _As (n) and β _As (n) are defined as follows.

Here, p ^{表 す} represents a uniform distribution on the set of the entire l × n matrix.

Lemma 5 ([4, Theorem 1]). ([A], p _As) as a set of linear _{mapping, p As ({A: Au} n = 0 l}) depends only on ^{t (u n) (t (} u n) = t (v ⁿ ), p _As ({A: Au ⁿ = 0 ^l }) = p _As ({A: Av ⁿ = 0 ^l }). {α _As (n) defined in (B2) and (B3) , β _As (n)} _{n = 1} ^∞ satisfies (H1), (H2), {([A _n ], p _{As, n} )} _{n = 1} ^∞ is {α _As (n), β _As (n)} _{n = 1} ^{∞ -has a} hash property.

Hereinafter, U≡GF (q), 0 <R <1, and l≡nR. An l × n matrix A is generated by the following procedure.
1. Initialize all elements of the matrix with zeros.
2. Repeat the following procedure τ times for each i∈ {1, ..., n}.
(a) (j, θ) ∈ {1,..., l} × [GF (q) \ {0}] is uniformly selected.
(b) Add θ to the (j, i) -component of A (addition of GF (q)).

It is assumed that τ = O (log n) is an even number and ([A], p _As ) is determined by the above procedure. [H ′] ⊂ [H] is appropriately determined, and (α _As , β _As ) is defined by (B2) and (B3). At this time, it is proved by [3, Theorem 2] that {α _As (n), β _As (n)} _{n = 1} ^∞ satisfies (H1) and (H2) for this ensemble. Therefore, according to Lemma 5, this ensemble has {α _As (n), β _As (n)} _{n = 1} ^∞ -hash.

  Next, we prove that the error probability of information compression that allows distortion converges to zero.

A set [Y ⁿ ] of information source outputs is an infinite set, an arbitrary set including a continuous set, and a reproduction alphabet [X ⁿ ] is a finite set.

Given r, R> 0, consider an ensemble of functions A: [X ⁿ ] → Im [A], B: [X ⁿ ] → Im [B]. Here, the set of compressed data is Im [B].

  In the following, it is assumed that the functions A∈ [A], B∈ [B] and a∈Im [A] are fixed and can be used by the encoding device and the decoding device.

^Given the probability distribution p _Y ⁿ and probability distribution p _X ⁿ _{| Y} ⁿ of the information source, p _X ⁿ is defined as follows.

A random number generator that satisfies the constraint conditions is used to configure the encoding device. Let X ~ ⁿ ≡ X ~ _A ⁿ (a | y ⁿ ) be a random variable that follows the following distribution.

The encoding device Φ _n : [Y ⁿ ] → Im [B] and the decoding device ψ _n : Im [B] → [X ⁿ ] are defined as follows.

Here, when p _X ⁿ _{| Y} ⁿ (C _A (a) | y ⁿ ) = 0, an encoding error is declared. Also, g _AB is defined as follows.

  The error probability Error (A, B, a, D) is defined as follows.

Here, when p _X ⁿ _{| Y} ⁿ (C _A (a) | y ⁿ ) = 0, d _n (ψ _n (Φ _n (y ⁿ )), y ⁿ ) ≡max _x ⁿ _{, y} ⁿ d _n It is defined as (x ⁿ , y ⁿ ).

  Theorem 2. If r, R> 0

, A∈ [A], B∈ [B], and a∈Im [A] exist for any δ> 0 and sufficiently large n, and satisfy the following relationship.

P _X ⁿ _{| Y} ⁿ achieves inf _pX ⁿ _{| Y} ⁿ _{: EX} ⁿ _Y ⁿ _{[dn (X} ⁿ _{, Y} ⁿ _{)] <D} I (X ⁿ ; Y ⁿ ) for the source distribution p _Y ⁿ Assuming that At this time, by bringing r closer to H (X ⁿ | Y ⁿ ), R is changed to inf _pX ⁿ _{| Y} ⁿ _{: EX} ⁿ _Y ⁿ _{[dn (X} ⁿ _{, Y} ⁿ _{)] <D} I (X ⁿ ; Y ⁿ ) On the other hand, the proposed code is (C30)

So that the average distortion E _X ⁿ _Y ⁿ [d _n (X ⁿ , Y ⁿ )] can be approximated to D by assuming max _x ⁿ _{, y} ⁿ d _n (x ⁿ , y ⁿ ) <∞. it can.

  Proof:

Then, from (C28), (C29),

There exists ε> 0 that satisfies

The T_ and subscript bar T, T ^- was a superscript bar of _{^{T, T - X ⊂ [X}} n], T_ X | is defined _Y ⊂ [X ^n] as × follows [Y ^n] .

  From [1, Theorem 16.8.1], the following equation holds.

Assume x ⁿ ⊂T ^- _X , g _AB (Ax ⁿ , Bx ⁿ ) ≠ x ⁿ . At this time, θ ⁿ ≠ x ⁿ and

There exists θ ⁿ ∈C _AB (Ax ⁿ , Bx ⁿ ) that satisfies Therefore, T ^- _X ∩C _AB (Ax ⁿ , Bx ⁿ ) ≠ φ holds, so

Get. Here, χ is defined as follows.

  In the first inequality, Lemma 3 and in the second inequality

Was used. Subsequently, the following equation holds.

  Here, (C35) was used in the last inequality. Further, the following formula is established.

  Here, Lemma 4 is used in the second inequality. When the distribution of a is a uniform distribution, the following equation holds.

  Here, the following relation is used in the second inequality.

In the third inequality, (C36) and (C37) are used. (C31), (C32) (C38) and n → 1 and α _As → 1, β _As → 0, α _AB → 1, β _AB → 0, p _X ⁿ ([T_ _X ] ^c ) → 0, p _X ^{Since n} _Y ⁿ ([T_X _{| Y} ] ^c ) → 0, the right side of (C38) converges to 0 with n → ∞. From this, the theorem can be established.

[Modifications, etc.]
The processes described in the above apparatus and method are not only executed in time series according to the order of description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the process.

  In addition, when the processing means in the random number sequence generation device, the encoding device, and the decoding device is realized by a computer, the processing contents of the functions that the random number sequence generation device, the encoding device, and the decoding device should have are described by a program. By executing this program on a computer, processing means in the random number sequence generation device, encoding device, and decoding device is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Random number sequence generation apparatus 11 Control part 12 Probability distribution generation part 13 Random number generation part 14 Storage part 15 Random number estimation part 16 Uniform random number generation part 100 Encoding apparatus 101 Storage part 102 Conversion filter part 103 Compression part 200 Decoding apparatus 201 Storage part 202 Random number sequence estimation unit 204 Conversion filter unit

Claims (6)

Realization that is a series of all the above-mentioned multiple values that satisfy a predetermined condition, with the probability of the realization value being a series of multiple values as a normalized value of the probability that the realization value appears in the predetermined first probability distribution as a second probability distribution a distribution of probability of the values,
The third probability, which is the probability of the actual value, is the first probability that the actual value will appear when the sequence of already generated random numbers is generated , the sequence of already generated random numbers, and this time a plurality of values of the series already composed of sequences and the remaining random sequence generated random number and the next time the currently generated random number generated first said predetermined condition is satisfied when that random numbers with the proviso that the resulting two probability, of a product as a normalized value, a probability distribution generating unit for generating a third probability distribution is a distribution of the third probabilities for all be taken of realization, one in accordance with the third probability distribution Random number generation unit including: a random number generation unit that generates an actual value as a random number; and a control unit that generates a sequence of random numbers according to the second probability distribution by repeating the processing of the probability distribution generation unit and the random number generation unit. And location,
A compression unit that generates a codeword using the generated random number sequence and a predetermined second function,
The predetermined condition is a condition expressing the relationship between the predetermined shared information and a series of a plurality of values using a predetermined first function,
The predetermined first probability distribution is a conditional probability distribution on condition that information to be encoded occurs .
Encoding device. The encoding device according to claim 1, comprising:
The predetermined first function is A, the predetermined second function is B, the range of the function A is Im [A], the range of the function B is Im [B], and the predetermined shared information is a∈ Im [A], the codeword is b∈Im [B], the series of values (x ₁ , x ₂ ,..., X _n ) is x ⁿ ∈ [X ⁿ ], and the predetermined condition Is A (x ⁿ ) = a, the encoding target information is y ⁿ , the first probability distribution is p _X ⁿ _{| Y} ⁿ ,
The probability for each real value x ⁿ in the second probability distribution is μ (x ⁿ ) defined by the following equation:

The compression unit calculates B (x ⁿ ) and sets the calculation result as a codeword b.
Encoding device. Realization that is a series of all the above-mentioned multiple values that satisfy a predetermined condition, with the probability of the realization value being a series of multiple values as a normalized value of the probability that the realization value appears in the predetermined first probability distribution as a second probability distribution a distribution of probability of the values,
When the probability distribution generation unit uses the third probability, which is the probability of the realized value, on the condition that a sequence of already generated random numbers is generated, the first probability that the realized value appears, and the already generated random number above sequence and sequence of a plurality of values comprising a series of the remaining random numbers already generated sequence and the random number and the next time the currently generated random number generated when the condition that the random numbers generated current caused A probability distribution generation step for generating a third probability distribution, which is a distribution of the third probability for all possible real values, and a product obtained by normalizing a product of the second probability satisfying a predetermined condition, and random number generation parts comprises a random number generation step of generating a random number one realization according to the above third probability distribution, the control unit is, by repeating the above probability distribution generation step and the random number generating step, the second probability A random number sequence generating step includes a control step of generating a sequence of random numbers according to the fabric, the,
A compression unit that generates a codeword using the sequence of generated random numbers and a predetermined second function, and
The predetermined condition is a condition expressing the relationship between the predetermined shared information and a series of a plurality of values using a predetermined first function,
The predetermined first probability distribution is a conditional probability distribution on condition that information to be encoded occurs .
Encoding method. The encoding method according to claim 3, comprising:
The predetermined first function is A, the predetermined second function is B, the range of the function A is Im [A], the range of the function B is Im [B], and the predetermined shared information is a∈ Im [A], the codeword is b∈Im [B], the series of values (x ₁ , x ₂ ,..., X _n ) is x ⁿ ∈ [X ⁿ ], and the predetermined condition Is A (x ⁿ ) = a, the encoding target information is y ⁿ , the first probability distribution is p _X ⁿ _{| Y} ⁿ ,
The probability for each real value x ⁿ in the second probability distribution is μ (x ⁿ ) defined by the following equation:

In the compression step, B (x ⁿ ) is calculated and the calculation result is a codeword b.
Encoding method.   A program for causing a computer to function as the encoding device according to claim 1.   A computer-readable recording medium on which a program for causing a computer to function as the encoding device according to claim 1 or 2 is recorded.

Images