JP2003218806A - Apparatus for generating sampled signal and apparatus for reproducing sampled signal and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable faithful reproduction of an original signal in an accuracy of its sampling frequency or more without increasing the data amount of a sampled signal. <P>SOLUTION: The apparatus for generating a sampled signal comprises a means for providing a sampled signal obtained by sampling the original signal by a predetermined first resolution, a means for providing a teacher signal obtained by sampling the original signal by a second resolution finer than the first resolution, and a neutral network for generating a weighting factor for calculating the sampled signal by learning based on the teacher signal, and outputs the generated weighting factor attached to the sampled signal. The apparatus for reproducing the sampled signal comprises a means for reproducing the sampled signal, and a neutral network for calculating the generated sampled signal with the weighting factor attached to the sampled signal, and restores the original signal at a precision corresponding to the second resolution. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for generating a sampling signal such as audio or image by using a neural network and an apparatus for reproducing the sampling signal generated by the apparatus, and further, Regarding the method.

[0002]

As is well known, when the original signal f is restored from the sampling signal fs, if the sampling period of the sampling signal fs is Ts and the time is t, the sampling theorem is expressed by the following equation (1). It is represented by.

[Equation 1] In the above formula, sin [π / Ts * (t-nTs)]
/ [Π / Ts * (t−nTs) represents the sinc function characteristic, and the nth (−∞ <n <∞) sample value f (nTs) sampled at the sampling period Ts is represented by the si
It is shown that the signal f (t) at a certain time point t of the original signal f can be restored by convoluting the nc function as a coefficient between −∞ and ∞. By the way, when the original signal is restored from the sampled signal by the sampling theorem as is well known, the waveform component above 1/2 (Nyquist frequency) of the sampling period Ts is cut in the restored signal. . For example, a compact disc for music (C
In the case of D), since sampling is usually performed at a sampling frequency of about 44 kHz, harmonic components of about 20 kHz or more are cut in the restored signal (that is, reproduced sound). However, for example, the frequency spectrum of an audio waveform such as an actual musical instrument sound contains a high component of 20 kHz or more, and various timbres are different depending on how the harmonic components are included. It is desirable to be able to reproduce high-quality sound that contains harmonic components, and it can be said that analog boards are still popular because of high-quality reproduced sounds that contain harmonic components. Of course, it is possible to obtain a high-quality reproduced sound including high-frequency components by setting the sampling frequency (sampling cycle Ts) high, but if sampling is performed at such a fine cycle, the amount of data required for the sampling signal will increase. It will increase enormously and requires a huge storage capacity.
Various inconveniences have occurred, such as a reduction in the substantial storage capacity such as the recording time that can be recorded on one D1 sheet. Therefore, in a normal CD, it was not possible to improve the sound quality by including a high frequency component while maintaining the current data amount.

On the other hand, it is known that the Nyquist frequency can be raised by reproducing the sound by oversampling without raising the original sampling frequency so that the reproduced sound can include higher frequency components. There is. That is, by transforming the equation (1), the following equation (2) is obtained, and based on this, oversampling reproduction is performed.

[Equation 2] In Expression (2), fs is a sampled value of a sampling signal obtained by sampling the original signal f at a sampling period Ts, and τ is Ts.
Finer predetermined period (oversampling period)
And Ts is an integral multiple of τ. fs (t-nτ)
Indicates a sampled value of the sampled signal fs at time t-nτ, which has a substantial sampled value when the time t-nτ is an integer multiple of Ts, and is 0 otherwise. Expression (2) indicates that the signal fs sampled at the sampling period Ts is subjected to convolution calculation with a resolution of a period τ finer than Ts. That is, it is shown that the original signal f can be restored with a resolution of the period τ smaller than Ts based on the sampling signal fs of the sampling period Ts. This convolution operation is realized by processing the sampling signal fs of the sampling period Ts with a digital filter operating at the timing of the period τ. In this case, as the weighting coefficient, that is, the filter coefficient, {sin [π
* Nτ / Ts]} / [π * nτ / Ts] sinc
The function is given as a fixed value for a finite number of n.
However, in such conventional reproduction by oversampling, even if a higher frequency component can be included in the reproduced sound by apparently increasing the Nyquist frequency, the sinc function that is fixed in advance is used. Since the convolution operation is performed only with the weighting characteristic, there is a limit in accurately and faithfully reproducing the waveform of the original signal f. Not only when the original signal is restored from the sampled signal in a physical storage medium such as a music CD, but for example, in the transmission and reproduction / restoration of music data and image data which have been actively performed in recent years via a communication network. Has the same problem.

[0004]

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and can faithfully restore an original signal with an accuracy higher than the sampling frequency without increasing the data amount of the sampling signal. The sampled signal generating device, the sampled signal reproducing device, and the method thereof are provided.

[0005]

According to a first aspect of the present invention, there is provided a sampling signal generating device which provides a sampling signal obtained by sampling the original signal with a predetermined first resolution, and the original signal. Means for providing a teacher signal sampled at a second resolution, which is finer than the first resolution, and inputting the sampling signal and the teacher signal, a weighting coefficient for calculating the sampling signal to the teacher signal. A neural network generated by learning based on the above, and outputting the weighting coefficient generated by attaching it to the sampling signal. Since the teacher signal is sampled at the second resolution, which is finer than the first resolution, it is high-quality data that is closer to the characteristics of the original signal than the sampled signal. Therefore, the weighting coefficient generated by the learning based on the teacher signal can reproduce a waveform having the same quality as the original signal reproduction accuracy of the teacher signal. Moreover, the data amount of the weighting factor is much smaller than the sample value itself. Further, since the sampling signal has a coarser resolution than the teacher signal, the data amount is much smaller than that of the teacher signal. Therefore, the data set consisting of the combination of the sampled signal and the weighting coefficient attached thereto can reproduce a waveform of the same quality as the original signal reproduction accuracy of the teacher signal, but its data amount is small.

A sampling signal reproducing device according to a second aspect of the present invention restores the original signal based on the sampling signal output from the sampling signal generating device according to the first aspect and the weighting coefficient attached to the sampling signal. Apparatus for reproducing the sampled signal at a first cycle corresponding to the first resolution, and the sampled signal reproduced according to a second cycle corresponding to the second resolution. And a neural network for calculating the weighting coefficient attached thereto,
An output signal obtained by restoring the original signal with an accuracy corresponding to the second resolution is output from the neural network. Since the weighting coefficient used here is generated by learning based on the teacher signal, it is possible to reproduce a waveform of quality equivalent to the accuracy of the original signal reproduction of the teacher signal. The calculation is performed with a variable weighting characteristic suitable for, and the waveform of the original signal can be accurately and faithfully reproduced.

The present invention can be constructed and implemented not only as an apparatus invention but also as a method invention.

Further, the storage medium according to the present invention samples the original signal with a predetermined first resolution and the original signal with a second resolution finer than the first resolution. It is characterized in that the sampling signal obtained by learning using a teacher signal and a weighting coefficient to be calculated are combined and stored as sampling data of the original signal. Furthermore, the data transmission method according to the present invention transmits a signal to a predetermined first
Of the sampling signal obtained by learning using a sampling signal sampled at a resolution of 2 and a teacher signal obtained by sampling the original signal at a second resolution smaller than the first resolution. It is characterized in that it is combined with a coefficient and transmitted as sampling data of the original signal.

It is preferable that the original signal is divided into a plurality of sections and each of the sections has a set of weighting factors, which can further improve the reproduction accuracy of the original signal.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION An original signal restoration from a sampled signal obtained by sampling an audio waveform of a music piece will be described below as an embodiment of the present invention with reference to the accompanying drawings. First, the outline of the necessary data generation processing on the data supply side will be described with reference to the basic block diagram of the embodiment of the present invention in FIG. 1. In FIG. 1, 10 is a supervised neural network and 20 is a sample. The converted signal supply source, 21 is a teacher signal supply source. Neural network 10
Is a delay line 11, a convolution operation unit 12, and
It is composed of a learning unit 13 with a teacher. As will be described later in detail, the neural network 10 calculates the sampling signal fs input to the neural network 10 by the convolution operation unit 12 with a predetermined weighting coefficient Wn,
The weighting coefficient W is set by the supervised learning unit 13 so that the output signal y as the calculation result becomes the same signal as the teacher signal fz.
n is appropriately changed (learned) by learning based on the teacher signal Fz, and the weighting coefficient Wn generated by this learning is attached to the sampling signal fs and output. The output data of the weighting coefficient Wn and the sampling signal f
The data of s can be stored in a storage medium such as a CD and provided to a data receiver.

The sampling signal source 20 has a sampling period Ts.
The sampling signal fs sampled at (first resolution) is supplied to the neural network 10. An example of the waveform diagram of the sampling signal fs is as shown in FIG. In (A), the original signal f is shown by a dotted line. The sampling signal fs (t) is data at a certain time point t of a signal obtained by sampling the original signal f with the resolution Ts. As the data practically supplied from the sampling signal supply source 20, for example, the master sound source of the music to be restored is sampled at a predetermined sampling period Ts (for example, the sampling frequency (about 44 kHz) of the normal CD standard). It is possible to use any of the above-mentioned data and the like, and it is also possible to use the existing data recorded on the CD or the like.
The teacher signal supply source 21 samples the original signal f with a more precise sampling period τ (second resolution)
Is supplied to the neural network 10. An example of the waveform diagram of the teacher signal fz is shown in FIG. 1 (B). Also in (B), the original signal f is shown by a dotted line. The teacher signal fz (t) is data at a certain time t of a signal obtained by sampling the original signal f with a resolution τ. The sampling period τ of the teacher signal fz is 1 / a times the sampling period Ts of the sampling signal fs (that is, aτ = Ts,
However, a is an integer), and one sampling period τ of the teacher signal fz is included in one sampling period Ts of the sampling signal fs. That is, the teacher signal fz is sampling data having a resolution more precise than that of the sampling signal fs. Therefore, as the data supplied from the teacher signal supply source 21, high quality data obtained by sampling the music audio waveform (original signal) to be restored at the sampling period τ is used. This is because it is possible to obtain a desired high-quality output signal by generating an appropriate weighting coefficient Wn based on the teacher signal fz and calculating the generated weighting coefficient Wn and the sampling signal fs. Because it will be. In addition,
In the waveform diagrams (A) and (B) in FIG. 1, as an example, the teacher signal f is generated during one sampling period Ts of the sampling signal fs.
There are four sampling periods τ of z.

The delay line 11 is composed of a plurality of delay stages, and the plurality of delay stages are connected to the delay line 1 in FIG.
It is indicated by a dotted line in 1. The sampling signal fs (t) supplied from the sampling signal supply source 20 is input to the delay line 11. The operation of the delay line 11 is controlled by the shift command CK. This shift command CK functions as a shift command having a resolution corresponding to the sampling period τ, but it is not generated in real time in the actual period τ, and it depends on the progress of the learning processing program in the neural network 10. It is generated in non-real time. The sampling signal fs (t) input to the delay line 11 is sequentially shifted by one stage for each shift command CK.

The sampling signal supply source 20 generates a sampling signal fs of one sample every time the shift command CK is generated a times.
(T) are sequentially output and input to the delay line 11. In the figure, the symbol aCK indicates an output command generated each time the shift command CK is generated a times. This output command aCK
Accordingly, the sampling signal fs (t) of one sample is input to the delay line 11 at one time out of the a times of occurrence of the shift command CK, and 0 is input to the delay line 11 at the remaining a-1 times. Is entered. An example of the input operation of the sampling signal fs (t) to the delay line 11 will be described with reference to FIG. Here, it is assumed that the resolution τ corresponding to the shift command CK is set to the sampling period T of the sampling signal fs (t).
It is set to ¼ with respect to s (that is, Ts = 4τ). Also,
The latest input data input to the delay line 11 is assumed to be input to the delay stage 11a on the left side in the figure.

Since the resolution τ corresponding to the shift command CK is Ts / 4, the shift command CK is generated four times for each interval of the sampling period Ts of the sampling signal fs (t). Therefore, the sampling signal fs of one sample at one time out of the four occurrences of the shift command CK
(T) is input to the delay line 11, and is 0 for the remaining three times.
Are input to the delay line 11. For example, if t in fs (t) changes to 0, 1, 2, ... For each cycle Ts, the sample value having a substantial value is fs in FIG.
(0), fs (1), fs (2), ... These are input to the delay line 11 once every four times the shift command CK is generated. In FIG. 2, what is shown as a fraction such as 1/4, 2/4, 3/4 as t in fs (t) indicates that it is not synchronized with the sampling cycle Ts. Value fs (1/4), fs (2/4), fs (3 /
As for 4), "0" is input as described above. As a specific example, when the sampling signal fs (0) having a substantial sampling value is input to the first delay stage 11a and the 1-shift command CK is generated next, this sampling signal fs (0) is Value is input to the delay stage 11a as the input data of the signal fs (1/4). When the next shift command CK is generated, the sampling signal fs (0) is shifted to the next delay stage 11c, and the value 0 of the delay stage 11a is shifted to the next delay stage 11b. The value 0 is input to 11a as the input data of the signal fs (2/4). In this way, according to the 1 shift command, the data of each delay stage is sequentially shifted to the next stage on the right side in the figure, and new data is input to the delay stage 11a. In the next third shift, the signal f is sent to the delay stage 11a.
The value 0 is input as the input data of s (3/4), and the sampling signal fs (1) having a substantial sample value is input to the delay stage 11a at the fourth shift. At this time, for each of the delay stages 11a to 11e, as shown in FIG.
That is, data is input such that fs (1) is input to a, 0 is input to 11b, 0 is input to 11c, 0 is input to 11d, and fs (0) is input to 11e.

The convolution operation unit 12 includes a delay line 11
A weighting factor Wn, which is a set of a plurality of coefficient values corresponding to the plurality of delay stages, is input. Convolution operation unit 12
Then, the convolution operation is performed by multiplying the data input to each of the plurality of delay stages and each coefficient value corresponding thereto. Since this convolution operation is publicly known, its description is omitted. As described above, since data is input to the delay line 11 with the resolution τ, the output signal y (t) output from the convolution operation unit 12 as a result of the operation is also a signal with the resolution τ. Here, si shown in the above-mentioned formula (1)
When the nc function is set as the initial value Wn ′ of the set of weighting factors Wn, the weighting factor Wn is expressed by the following equation (3).

[Equation 3] In this way, when the weighting coefficient Wn is initialized by the sinc function, the neural network 10 is a system equivalent to the above-mentioned equation (2). In this case, the coefficient values W-2, W-1, W0, W1 and W2 illustrated in FIG. 1 are respectively expressed by the following formula (4), formula (5), formula (6), formula (7) and formula. It is described as (8).

[Equation 4] FIG. 3 is a diagram showing the sinc function expressed by the equation (3).
Is like. .. W-2, W-1, W0, W1, W2, ... In one set of weighting factors Wn are plotted on the left and right with a resolution of .tau. Centered on W0 in a sinc function as shown in FIG. It consists of a finite number of coefficients. Here, the sample value (this is f
The timing of s (i) is the current sample timing, and the teacher signal fz (t) is the teacher signal sample value (this is fz) corresponding to the current sample timing.
(I) is used. The current sample timing is a sample timing with accuracy corresponding to the resolution τ that progresses for each shift command CK. Note that the number of weighting coefficients Wn consisting of a finite number of coefficient groups may be set arbitrarily in design, and for example, even about 1000, a sufficient reproduction accuracy can be obtained.

The teacher signal supply source 21 sequentially outputs sample values of the teacher signal fz (t) having an accuracy corresponding to the resolution τ each time the shift command CK is generated (that is, as the current sample timing progresses). , And inputs it to the supervised learning unit 13. Note that, for example, the sampling signal fs
When the sample value fs (i) at the i-th time point t in (t) is shifted to the position corresponding to the weighting coefficient W0 in the delay line 11, the teacher signal sample value fz at the same i-th time point t
The sampling signal source 20 and the teacher signal source 21 operate synchronously so as to output (i). Teacher signal fz (t)
Is a convolution operation unit 12 of the neural network 10.
The output signal y (t) output from the teacher signal fz
It is an exemplary signal for setting an appropriate weighting coefficient Wn so that it becomes the same signal as (t). The supervised learning unit 13
By learning using the teacher signal fz (t) as a model signal, the value of the weighting coefficient Wn is appropriately set so that the output signal y (t) output from the convolution operation unit 12 has the same value as the teacher signal fz (t). Change to. As a specific learning algorithm in the supervised learning unit 13, a known or undisclosed appropriate method may be used.

For the neural network 10, the sampling signal fs corresponding to a certain range of the original signal f to be restored
And learning signal fz are sequentially input to the end, and learning for the entire input range is referred to as “learn once waveform”. For example, in the case of learning the entire music piece by one-waveform learning, the sampling signals from the beginning of the music piece to the end of the music piece are sequentially input to the neural network 10, and the teacher signal also corresponds to the beginning of the music piece. From the beginning to the end of the song, switch in order and input. Convolution operation unit 12
As is clear from the fact that the output signal y (t) and the teacher signal fz (t) are input to the supervised learning unit 13 at the resolution τ, when learning one waveform time, one time t Learning is performed at a resolution equivalent to τ, and this learning processing equivalent to 1τ is named one-step learning.

For example, the i-th one-step learning is performed.
In this case, the sampling signal fs
The sample value fs (i) at the i-th time t in (t) is delayed by
Shift to the position corresponding to the weight coefficient W0
And the teacher signal sample value fz (i) at the same i-th time t
Is input to the supervised learning unit 13. On delay line 11
Convolution operation unit 1 with respect to the input sampling signal
Weighting coefficient Wn at 2₁Performs a convolution operation with
As a result, the output signal y (i) is output to the supervised learning unit 13.
Is input by the supervised learning unit 13.
The input convolutional performance based on the value of the teacher signal fz (i)
The output signal y (i) from the calculator 12 is the teacher signal fz.
The weight of an appropriate set that is the same as the value of (i)
Coefficient Wn₂Change to. Thus, one teacher signal
Corresponding to the value of fz (and corresponding sampling signal
As a learning result of 1-step learning)
Then, a set of values of the weighting coefficient Wn is calculated. Next one
In step learning, the weighting factor Wn ₂Will be updated
Become. In this way, one shift command CK
Step learning is performed sequentially, one set for each step learning
The value of the weighting coefficient Wn of is sequentially updated. in this way
Then, the weighting coefficient Wn is sequentially updated, and a predetermined value to be restored is set.
1 when the last step learning in the range is done
The data of the weight coefficient Wn of the set is used for the learning of the one waveform.
Output as a set of weighting factors Wn
Force (the Wn learning result output shown in FIG. 1) is output.
Weighting coefficient Wn output as a result of this one-waveform learning
Is a teacher signal that is high-quality data close to the characteristics of the original signal f.
Since it was generated by learning based on the issue fz, the teacher
It is possible to reproduce a waveform of the same quality as the original signal reproduction accuracy of the signal.
It is a thing. Example of change in weighting coefficient Wn due to learning processing
Is shown in FIG. As a result of learning, after learning processing
The weighting factor Wn is compared with the initial value Wn '(sinc function)
You can see that it is changing.

When the learning process is actually performed for a certain music piece, the audio waveform (original signal f) to be processed is divided into a plurality of sections, and 1 is set for each of the divided sections.
It is preferable to perform the waveform learning and to have the set of weighting factors corresponding to the respective sections, whereby the reproduction accuracy of the original signal f can be further improved. For example, a song with a song duration of 3 minutes is learned once per waveform.
Considering that the learning is performed by dividing into three waveform time sections each of which is one minute, in this case, the first one waveform time learning is performed for 1 minute from the beginning (0 minute) to 1 minute of the music as the first section. Up to 2 minutes is used as the second section, the second one-waveform learning is performed, and 2 minutes ~
The third one-time waveform learning is performed by setting up to 3 minutes as the third section. As a result, the weighting factor W used when the music is reproduced
n is one set for each section, a first weighting coefficient Wn (1) for the first section, a second weighting coefficient Wn (2) for the second section, and a third section for the third section. A total of three sets of weighting factors Wn, which are the third weighting factors Wn (3), are obtained. The time length per section does not need to be uniform, and is set appropriately, for example, the first section is 40 seconds, the second section is 1 minute 30 seconds, and the third section is 50 seconds. be able to.

The above-mentioned one-waveform learning may be performed a plurality of times for the same waveform (section). That is, the weighting coefficient Wn output after learning a certain waveform one waveform time may be used as an initial value, and the waveform may be learned once again and then learned two waveform times. Similarly, three waveforms may be learned. One-waveform learning may be repeated, such as one-time learning, four-waveform learning, and so on. By performing the one-waveform learning a plurality of times on the same waveform in this way, the weighting factor Wn
The original signal restoration ability of can be improved.

The data of the weighting coefficient generated by the above-mentioned learning process is attached to the sampling signal fs and output. As a form of this output, there is a form of recording on a physical storage medium such as a CD. When taking such an output form,
A physical storage medium such as a CD on which data including a combination of a sampling signal generated according to the present invention and a weighting factor is recorded is supplied to a data receiver such as a general user. An example of the structure of the data, which is output in this way and is composed of the combination of the sampling signal generated according to the present invention and the weighting coefficient, will be described with reference to FIG. FIG. 5 shows an example of the configuration of sampling data composed of a combination of the sampling signal fs and the weighting coefficient Wn generated according to the present invention. The original signal composed of music data is divided into a plurality of sections (three sections in the above example). ), The weighting coefficient is learned and generated for each section, and this is combined with a sampling signal and recorded on a recording medium such as a CD. In this data format, a header section for recording control information and the like necessary for reproducing the music data and a sampling signal fs obtained by sampling the music (original signal f) to be reproduced at a sampling period Ts are recorded in time series. Audio data recording unit. In the header part, the sampling signal fs
Of the sampling period Ts and data indicating a sampling period τ smaller than the sampling period Ts of each section (first to third
Weighting coefficient Wn (1)
(3) and time interval data T indicating the duration of each section
(1) to (3) are recorded. Time interval data T
(1) corresponds to the first section, and the time interval data T (2)
Corresponds to the second section, and the time interval data T (3) is the third
Corresponds to the section. This time interval data T (1)-
According to the time designated by (3), the corresponding weighting factors Wn (1) to (3) are used, and the switching timing is controlled. It is to be noted that not all the weighting factors Wn (1) to (3) and the time interval data T (1) to (3) as shown in FIG. Weighting coefficient Wn (1)
And time interval data T (1) are stored in the header part, and weighting factors Wn (2) and Wn (3) and time interval data T
As for (2) and T (3), the weighting coefficient Wn of the succeeding section is set as the weighting coefficient Wn of the succeeding section during the reproduction of the preceding section, as appropriate while being woven into the audio data in the preceding section (for example, the subcode area of the CD format) Time interval data T
May be read out.

For example, assuming that the sampling frequency of the sampling signal fs is 44.1 kHz as in the conventional CD standard, the data amount of the sampling signal fs is only the data amount (the number of samples) necessary for one second. Although there are 44100 data items, the data amount of one set of weighting coefficients Wn for one waveform learning may be, for example, about 1000 data items per one set. From this, it is understood that the amount of data required as the weighting coefficient Wn for one waveform learning is extremely small. Therefore, even if the data of the weighting factor Wn is attached to the sampling signal fs, the total amount of data required for restoration hardly increases. in this way,
The data set including the combination of the sampling signal fs and the weighting coefficient Wn attached to the sampling signal fs can reproduce a waveform of the same quality as the original signal reproduction accuracy of the teacher signal fz, but the amount of data is small. .

The output form of data consisting of the combination of the sampling signal generated according to the present invention and the weighting coefficient, that is, the data supply method is not limited to the physical storage medium as described above, but may be, for example, the Internet. Data may be transmitted via a communication line such as. In that case, the data receiver may store the data received via the communication line such as the Internet in its own memory.

Next, the reproducing process of the sampled data recorded on the CD or the like or transmitted via the communication line will be described with reference to FIG. As shown in FIG. 6, the reproduction processing side (data receiver side) has a neural network 30 and a sampling signal reproduction device 40. The neural network 30 has a delay line 31 and a convolution operation unit 32 similar to the delay line 11 and the convolution operation unit 12 in the neural network 10 of FIG.
No learning means is required.

In FIG. 6, in the sampling signal reproducing device 40, sampling data composed of a combination of the sampling signal fs of the sampling period Ts and the weighting coefficient Wn as described above,
It is supplied by a recording medium such as a CD or via a communication line. The sampling signal reproducing device 40 first reads the data of the header part of the supplied sampling data, and based on the data indicating each sampling period Ts, τ in the header part, sets the sampling period Ts and τ. Generate the corresponding clock.
In addition, the weighting coefficient Wn of the first section in the header section is read and input to the convolution operation section 32 of the neural network 30, and the weighting coefficient Wn in the convolution operation section 32 is set. After the initial setting is performed in this way, the reproduction and reading of the sampling signal fs in the audio data section is started.

As described above, the sampling signal fs reproduced by the sampling signal reproducing device 40 has a relatively coarse sampling period Ts.
The data is sampled in. Reproduction and reading of the sampling signal fs (t) from the reproducing device 40 are sequentially performed one sample at a timing corresponding to the sampling period Ts. The sampling signal fs (t) read from the reproducing device 40 is
It is input to the delay line 31 of the neural network 30. A clock having a fine cycle τ is supplied to the delay line 31 from the reproducing device 40 in real time as a shift clock. The delay line 31 takes in the sampling signal fs (t) input from the reproducing apparatus 40 at the timing of the cycle τ and sequentially shifts (delays) it every cycle τ. Similar to the delay line 11 of FIG. 1, in the delay line 31, the sampling signal f is synchronized at the timing of the period τ synchronized with the sampling period Ts.
A substantial sample value of s (t) is taken in, and 0 is taken in at the timing of the period τ other than that. In the figure, symbol Ts · τ
Indicates the timing of the period τ synchronized with the sampling period Ts. (A) in FIG. 6 shows an example of the sampling signal fs (t) read from the reproducing device 40, and a substantial sample value is reproduced and output at the timing of the period τ synchronized with the sampling period Ts. A value of 0 is output at the timing of cycle τ other than. For example, the sampling frequency (period Ts) of the sampling signal fs is 1, the frequency of the clock τ (period τ) is 4, and one sample value is S. 1 with the delay line 31 according to one clock of the clock τ
If two delay stages take in S, then
In the subsequent 3 clocks of τ, 0 is taken in as input data.

The convolution operation unit 32 performs a convolution operation on the data sequentially input to the delay line 31 with the weighting coefficient Wn supplied from the reproducing device 40 in accordance with the cycle τ corresponding to the clock τ, and the operation result y ( i) is output every period τ. The signal y (i) output as a calculation result from the convolution calculation unit 32 is a fine signal having a period τ as shown in (B) of FIG. Thus, this period τ
The output signal y (i) is output from the neural network 30 with the accuracy corresponding to. Here, the weighting coefficient Wn supplied to the convolution operation unit 32 is capable of reproducing a waveform having the same quality as the original signal reproduction accuracy of the teacher signal fz as described above, and the weighting characteristic by such a weighting coefficient Wn. Output signal y that faithfully reproduces the waveform of the original signal f with high accuracy comparable to the teacher signal
(I) is obtained.

FIG. 7 shows an example of spectrum distribution comparing the signal restored by the present invention and the signal restored by the conventional method. In the figure, the horizontal axis represents frequency, the vertical axis represents amplitude, and the waveform signal component is represented by a spectrum envelope, and 1 / Ts is a sampling frequency corresponding to the sampling period Ts. In the conventional method, the waveform component more than half the frequency (Nyquist frequency) 1 / 2Ts of the sampling frequency 1 / Ts shown by the dotted line in the figure was cut off and was not reproduced, but according to the present invention. If
It is possible to accurately and faithfully reproduce the waveform of the original signal f including such a high frequency component which has not been reproduced conventionally, without increasing the sampling frequency 1 / Ts of the sampling signal fs and increasing the data amount. For example, according to the present invention, even a sampled signal sampled at a sampling frequency of about 44 kHz according to the normal CD standard has been conventionally cut by generating the weighting coefficient used during reproduction by learning based on the teacher signal. It is possible to output a high quality reproduction signal including a high frequency component of about 20 kHz or more, and further it is possible to reproduce high quality music in which the waveform of the original signal f is faithfully reproduced.

In the above-described embodiment, the example in which the single-layer type neural network composed of only one stage is used as the neural networks 10 and 30 is shown, but the present invention is not limited to this, and the neural network is made to have multiple stages to perform nonlinear processing. If a multi-layered neural network capable of performing the above is used, it is possible to further improve the original signal restoration capability. In addition,
The sampling intervals of the input sampling signal may be equal intervals, but in the present invention, the sampling intervals may be unequal intervals. Further, as another embodiment of the present embodiment, the present invention may be carried out in combination with an appropriate data compression method such as MPG or MP3. That is, when the weighting coefficient Wn and the sampled signal fs are combined and transmitted as sampled data or recorded in a storage medium,
The sampling signal fs is compressed by an appropriate data compression method such as MPG, MP3, DPCM, ADPCM, etc., and the compressed sampling signal data is transmitted as sampling data together with the weighting coefficient Wn, or is recorded in a storage medium. . Then, at the time of reproducing the sampled data, the compression of the compressed sampled signal data may be released, and then the weighting coefficient Wn may be calculated. As yet another example of the present embodiment, the time required for learning is shortened by setting the time interval of one-waveform learning to be extremely short, and the sampling signal according to the present embodiment is based on the audio signal picked up in real time on the transmission side. By generating fs and the weighting coefficient Wn in real time, transmitting this through a communication line or the like, and reproducing at the receiving side according to the present embodiment, it is possible to perform from learning to reproduction in a form close to real time. .

In the above-mentioned embodiment, the restoration of audio waveform data of a CD or the like has been mainly described, but the present invention is not limited to this and can be applied to, for example, image data and the like. In the case of image data, highly accurate image data with a small number of pixels can be restored from image data with a rough number of pixels. Also,
If the present invention is applied to the interpolation processing at the time of image enlargement, the enlarged image is generated with a finer precision than the original image. Also,
By applying the present invention to the frame number interpolation processing of moving image data, it is possible to interpolate and generate a larger number of frames than the number of frames (frame number) of the original moving image data, and a moving image with smoother motion than the original moving image. Can be played.

[0031]

As described above, according to the present invention, it is possible to faithfully restore the original signal with accuracy higher than the sampling frequency without increasing the data amount of the sampling signal, and The sampled signal reproducing device and the method thereof can be provided effectively.

[Brief description of drawings]

FIG. 1 is a basic block diagram of a data supply side showing an embodiment according to the present invention.

FIG. 2 is a conceptual diagram showing an input operation of a sampling signal input to a delay line in the embodiment.

FIG. 3 is a diagram showing characteristics of a sinc function set as initial values of weighting factors and respective coefficient values in the same embodiment.

FIG. 4 is a diagram showing changes in weighting coefficient values after learning from the sinc function (initial value) shown in FIG. 3.

FIG. 5 is a diagram showing an example of a data configuration when an original signal is divided into a plurality of sections as another example of the learning process according to the embodiment.

FIG. 6 is a basic block diagram on the data receiver side according to the embodiment.

FIG. 7 is a spectrum distribution diagram comparing a signal restored by the present invention and a signal restored by a conventional method.

[Explanation of symbols]

10,30 Neural network 11,31 delay line 12,32 Convolution operation unit 13 Supervised learning section 20 Sampling signal source 21 Teacher signal source 40 Sampling signal regenerator

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5D044 AB05 AB07 GK08 GK11 GK14                 5D045 DA20                 5J022 BA01 CA10 CE04                 5K041 CC01 CC02 CC07 DD00 EE01                       EE17 EE21 FF11 HH11

Claims (7)

[Claims] 1. A means for providing a sampled signal obtained by sampling an original signal at a predetermined first resolution, and a teacher signal obtained by sampling the original signal at a second resolution finer than the first resolution. A means for providing the sampled signal and the teacher signal, and a neural network for generating a weighting coefficient that operates with the sampled signal by learning based on the teacher signal. A sampling signal generation device characterized in that the sampling signal generation device outputs the sampled signal. 2. A device for restoring the original signal based on the sampling signal output from the sampling signal generation device according to claim 1 and the weighting coefficient attached thereto, wherein the sampling signal is converted into the first signal. Means for reproducing at a first cycle corresponding to the resolution, and a neural network for calculating the reproduced sampled signal and the weighting coefficient attached thereto according to a second cycle corresponding to the second resolution. Includes the second
An output signal obtained by restoring the original signal with an accuracy corresponding to the resolution of 1. is output from the neural network. 3. A neural network is inputted with a sampled signal obtained by sampling an original signal at a predetermined first resolution and a teacher signal obtained by sampling the original signal at a second resolution finer than the first resolution. In the neural network, a step of generating a weighting coefficient that operates with the sampling signal by learning based on the teacher signal, and a step of outputting the generated weighting coefficient attached to the sampling signal. A sampling signal generation method comprising: 4. A method of restoring the original signal based on the sampling signal output by the sampling signal generation method according to claim 3 and the weighting coefficient attached to the sampling signal, wherein the sampling signal is converted into the first signal. A step of reproducing at a first cycle corresponding to the resolution; and, in a neural network, calculating the reproduced sampled signal and the weighting coefficient attached thereto according to a second cycle corresponding to the second resolution. , The second
And outputting a signal obtained by reconstructing the original signal with an accuracy corresponding to the resolution of the sampling signal reproduction method. 5. The learning using a sampling signal obtained by sampling the original signal at a predetermined first resolution and a teacher signal obtained by sampling the original signal at a second resolution finer than the first resolution. A storage medium in which the obtained sampling signal and a weighting factor to be calculated are combined and stored as sampling data of the original signal. 6. A learning signal using a sampling signal obtained by sampling an original signal at a predetermined first resolution and a teacher signal obtained by sampling the original signal at a second resolution finer than the first resolution. A data transmission system characterized in that the obtained sampling signal and a weighting coefficient to be calculated are combined and transmitted as sampling data of the original signal. 7. The apparatus or method according to claim 1, wherein the original signal is divided into a plurality of sections and each of the sections has a set of the weighting factors. Storage medium or data transmission method.