JP5089651B2 - Speech recognition device, acoustic model creation device, method thereof, program, and recording medium - Google Patents

Description

  The present invention provides a speech recognition apparatus that generates less recognition errors even when unknown non-stationary noise is input, an acoustic model generation apparatus that generates a highly accurate acoustic model even when unknown non-stationary noise is input, and their The present invention relates to a method, a program, and a recording medium.

  In recent years, the progress of speech recognition technology based on statistical methods has made it possible to perform speech recognition in a quiet environment with high accuracy. However, in an actual environment, there is a problem that the recognition performance deteriorates due to the presence of noise, particularly unknown non-stationary noise.

  FIG. 9 shows a functional configuration of a conventional speech recognition apparatus 900. The speech recognition apparatus 900 includes an A / D conversion unit 10, a feature amount analysis unit 20, a speech recognition processing unit 30, an acoustic model parameter memory 40, and a language model parameter memory 50.

The A / D converter 10 converts the sound of the input analog signal into a discrete digital signal, for example, at a sampling frequency of 16 kHz. The feature amount analysis unit 20 receives the speech digital signal that has been converted into discrete values, and calculates the speech feature amount O _t for each frame in which, for example, 320 speech digital signals are one frame (20 ms). The voice feature amount O _t is calculated, for example, by Mel frequency cepstrum coefficient (MFCC) analysis.

The speech recognition processing unit 30 refers to the acoustic model recorded in the acoustic model parameter memory 40 with the speech feature quantity O _t as an input, and the language model recorded in the language model parameter memory 50, and the likelihood of the acoustic model. And the speech recognition result candidate having the highest likelihood of the language model is output as the speech recognition result.

  In the conventional speech recognition apparatus 900, a method of correcting the likelihood of the acoustic model has been taken for the purpose of dealing with sudden noise that is unknown unsteady noise. The acoustic model is expressed by an HMM (Hidden Markov Model), and a normal distribution is widely used as the appearance probability distribution. The likelihood, which is the logarithm of the appearance probability, is a quadratic function and exhibits a characteristic that decreases according to the square of the deviation from the distribution average. The difference in the characteristics between the appearance probability and likelihood of the acoustic model is considered to be a cause of recognition error due to sudden noise. The idea of correcting the difference is disclosed in Non-Patent Document 1, for example.

FIG. 10 shows the idea of likelihood correction of Non-Patent Document 1. The upper left side of FIG. 10 shows the appearance probability distribution of phonemes | a | and phonemes | o |. The lower left side shows the likelihood characteristics of each phoneme. The horizontal axis of FIG. 10 is the audio feature amount. When the speech feature amount _s is observed, the appearance probability is higher for the phoneme | o | than for the phoneme | a |. The likelihood shown on the lower left side shows the same tendency.

However, when the speech feature amount _s changes to _yo due to the influence of superimposed noise or the like, the appearance probabilities are both small, but the likelihood shown in the lower left side has a magnitude relationship between the phoneme | a | and the phoneme | o |. Not only does it reverse, but the quadratic curve makes a big difference between them. Thus, although the difference in appearance probability is small, a large difference in likelihood is considered to contribute to recognition errors.

  Therefore, in Non-Patent Document 1, in order to improve robustness against sudden noise, a small positive correction constant ε is added to the observed data distribution N (y), and the likelihood of the value (formula (1)) ) Is used to avoid the problem that a difference in linear appearance probability becomes a difference in likelihood.

Here, N (y) is the distribution of the voice feature amount of the observed data. That is, the distribution of phonemes | a | and phonemes | o | in FIG. ε is a correction constant.
The process of adding the correction constant ε is performed by the voice recognition processing unit 30. By this processing, it is possible to reduce the difference in likelihood as shown in the lower right side of FIG. Therefore, since the amount of change in likelihood when sudden noise occurs can be reduced, the effect of suppressing the occurrence of recognition errors can be expected.

Hitoshi Yamamoto, Koichi Shinoda, Shigeki Hiyama “Speech Recognition Robust against Sudden Noise by Correcting Likelihood of Normal Distribution”, Acoustical Society Autumn Meeting, 1-9-10, pp.19-20,2002

  The idea of introducing the conventional correction constant ε has a risk that the recognition accuracy may deteriorate depending on the setting of the constant. The optimum constant varies depending on the data to be recognized and cannot be determined uniformly. If the constants are fixed uniformly, the phoneme model may affect the effect of correction, and even if the correct likelihood is obtained, there is a concern that it may be hindered.

  The present invention has been made in view of the above points, and a speech recognition apparatus that reduces the occurrence of recognition errors by excluding data when the likelihood exceeds a certain range, and its It is an object of the present invention to provide a method, an acoustic model creation device and method based on the same idea, a program, and a recording medium.

  The speech recognition apparatus according to the present invention includes a feature amount analysis unit, a GMM likelihood calculation unit, a GMM likelihood determination unit, and a speech recognition processing unit. The feature amount analysis unit analyzes the speech feature amount of the input speech digital signal in units of frames. The GMM likelihood calculation unit collates GMM (Gaussian Mixture Model: mixed normal distribution model) with the above-mentioned speech feature quantity, and calculates the GMM likelihood for each frame. The GMM likelihood determination unit determines whether or not the GMM likelihood is within a predetermined range, and outputs the determination result and the GMM likelihood. The speech recognition processing unit receives the speech feature amount, the GMM likelihood, and the determination result, performs speech recognition processing on the frame within the predetermined range based on the acoustic likelihood corresponding to the speech feature amount, For frames outside the range, speech recognition processing is performed using acoustic likelihood using GMM likelihood.

  Further, the acoustic model creation device of the present invention includes a learning processing unit and the same GMM likelihood calculating unit and GMM likelihood determining unit as described above, and the learning processing unit determines that the determination result is out of range. An acoustic model is generated after learning with the frame excluded from the statistical model calculation target.

The speech recognition apparatus and acoustic model creation apparatus of the present invention use the likelihood obtained from the GMM, which is a mixed Gaussian distribution model that includes most phonemes and has a wide dispersion. Therefore, it is possible to reduce the likelihood reversal phenomenon that occurs at the end of the distribution, which has been a problem in the past, and the problem that the difference in likelihood increases.
That is, since the acoustic likelihood of the frame determined to be out of the range by the GMM likelihood determination unit is substituted for the GMM likelihood calculated based on the GMM by the GMM likelihood calculation unit, sudden noise is input. The acoustic likelihood at the time can be stabilized. As a result, there is an effect of improving the accuracy of the speech recognition process and the acoustic model learning process.

The figure which shows typically 1 state which comprises a phoneme model. The figure which shows an example of a phoneme model. The figure which shows the function structural example of the speech recognition apparatus 100 of this invention. The figure which shows the operation | movement flow of the speech recognition apparatus 100. The figure which shows the relationship between the audio | voice feature-value using GMM, and GMM likelihood. The figure which shows the operation | movement flow of speech recognition apparatus 100 '. The figure which shows the function structural example of the acoustic model production apparatus 200 of this invention. The figure which shows the operation | movement flow of the acoustic model production apparatus 200. The figure which shows the function structure of the conventional speech recognition apparatus 900. The figure which shows the idea of the likelihood correction | amendment disclosed by the nonpatent literature 1. FIG.

Before describing the embodiments of the present invention, the idea of the present invention will be described.
[Concept of this invention]
In describing the idea of the present invention, an acoustic model will be described first. A phoneme model constituting the acoustic model is constructed by a probability chain of about three states. Each state is expressed as a mixed normal distribution. FIG. 1 shows, for example, three normal distributions when the number of mixtures is 3, N (μ ₁ , U ₁ ), N (μ ₂ , U ₂ ), N (μ ₃ , U ₃ ), weight coefficient c ₁ , A state s composed of c ₂ and c ₃ is shown. μ is an average vector, and U is a covariance matrix.

FIG. 2 shows, as an example, a conceptual diagram of a phoneme model composed of three states. This example is called a left-to-right type HMM, which is an array of three states s ₁ (first state), s ₂ (second state), and s ₃ (third state). As a probability chain (state transition probability), self-transitions a ₁₁ , a ₂₂ , a ₃₃ and a ₁₂ , a ₂₃ , a ₃₄ to the next state are included. A combination of phoneme models having the highest likelihood in the state transition series is output as a speech recognition result. A set of phoneme models is an acoustic model.
The appearance probability P (s, O _t ) obtained from the state s is obtained by Expression (2).

Here, O _t is an audio feature amount of frame t, N (O _t ; μ _ms , U _ms ) is a probability calculated from a normal distribution consisting of an average vector μ _ms and a covariance matrix U _ms , and c _ms is a mixing weight coefficient. , M _s is the number of distributions belonging to state s. The sum of the logarithmic value of the appearance probability P (s, O _t ) in each state and the above-described state transition probability is the acoustic likelihood.
In the concept of adding the correction constant ε described in the background art to the distribution of the speech feature amount, when sudden noise is input, the acoustic likelihood may greatly vary as is apparent from the above description. Was the cause of recognition errors.

In contrast to the conventional method, the speech recognition method of the present invention calculates the GMM likelihood by comparing the GMM with the speech feature before speech recognition processing. Then, it is determined whether or not the GMM likelihood is within a predetermined range. If the GMM likelihood is within a predetermined range, the acoustic likelihood based on the speech feature amount is obtained in the speech recognition process, and the speech recognition process is performed using the acoustic likelihood.
On the other hand, when the GMM likelihood is out of the predetermined range, for example, the GMM likelihood obtained from the GMM to which sudden noise has been input is subjected to speech recognition processing instead of the acoustic likelihood, so that the acoustic likelihood changes greatly. There is nothing to do.

Therefore, unlike the conventional method, the small difference in appearance probability with respect to the speech feature amount is not reversed, or the small difference in appearance probability does not change to a large likelihood difference. Thus, according to the idea of the present invention, it is possible to stabilize the value of acoustic likelihood. As a result, misrecognition of voice recognition can be reduced. This idea can also be applied to an acoustic model creation device.
Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 3 shows a functional configuration example of the speech recognition apparatus 100 of the present invention. The operation flow is shown in FIG. The speech recognition apparatus 100 includes an A / D conversion unit 10, a feature amount analysis unit 20, an acoustic model parameter memory 40, a GMM likelihood calculation unit 60, a GMM likelihood determination unit 70, and a speech recognition processing unit 80. The language model parameter memory 50 and the control unit 90 are provided. The speech recognition apparatus 100 is realized by reading a predetermined program into a computer configured with, for example, a ROM, a RAM, a CPU, and the like, and executing the program by the CPU.

  Compared with the conventional speech recognition apparatus 900, the speech recognition apparatus 100 is new in that it includes a GMM likelihood calculation unit 60 and a GMM likelihood determination unit 70. Further, the operation of the voice recognition processing unit 80 is different from that of the conventional voice recognition processing unit 30. Other functional configurations are the same as those of the speech recognition apparatus 900. In the following description, the description will focus on the different parts.

The A / D converter 10 converts the sound of the input analog signal into a discrete digital signal, for example, at a sampling frequency of 16 kHz (step S10, FIG. 4). The feature amount analysis unit 20 receives the speech digital signal converted into a discrete value, and calculates the speech feature amount O _t for each frame in which a predetermined number of speech digital signals are one frame (for example, 20 ms) (step S20). ).
The GMM likelihood calculating unit 60 calculates the GMM likelihood for each frame by comparing the GMM with the speech feature amount O _t (step S60). The GMM is a mixed normal distribution model (GMM) learned from all phonemes in the learning data of the acoustic model (except for silence in some cases). The GMM is recorded in the acoustic model parameter memory 40 in this example.

FIG. 5 schematically shows a method for obtaining the GMM likelihood by comparing the GMM and the speech feature amount O _t . FIG. 5 shows a case where the GMM likelihood distribution is assumed to be close to a normal distribution. The horizontal axis is the voice feature amount O _t , and the vertical axis is the GMM likelihood.
The calculation of the GMM likelihood is obtained as a logarithmic value of the appearance probability obtained by the above equation (2). In this case, the GMM is expressed by a mixing weight coefficient c _ms , an average vector μ _ms , and a covariance matrix U _ms as shown in Equation (2). GMM likelihood corresponding to the audio feature amount y _s and y _o as shown in FIG. 5 are calculated.

  The GMM likelihood determining unit 70 determines whether or not the GMM likelihood is within a predetermined range, and outputs the determination result (step S70). The predetermined range is, for example, a range from the maximum value to the minimum value of the GMM likelihood distribution shown in FIG. This range is the range of the upper and lower limit values of the GMM likelihood for the learned speech data. That is, the GMM likelihood determination unit 70 can filter frames that are affected by sudden noise that has not been learned.

When the GMM likelihood is within a predetermined range (Y in step S70), the speech recognition processing unit 80 obtains an acoustic likelihood corresponding to the speech feature amount Ot (step S801), and based on the acoustic likelihood. A voice recognition process is performed (step S802). If the GMM likelihood is outside the predetermined range (N in step S70), the speech recognition process is performed using the GMM likelihood instead of the acoustic likelihood (step S803).
The above operation is repeated until all frames are completed (N in step S90). The control unit 90 controls the operation and repetitive operation of each unit of the speech recognition apparatus 100.

According to the speech recognition apparatus 100, by using the GMM likelihood obtained from the speech feature amount O _t and the GMM, it is determined whether or not the speech feature amount O _t greatly deviates from the learned feature amount set. . Then, when a speech feature quantity O _t that is not included in the set of learning data such as sudden noise is input, the speech likelihood processing is performed by replacing the acoustic likelihood with the GMM likelihood. Therefore, even when sudden noise or the like is input, the acoustic likelihood can be stabilized. As a result, occurrence of misrecognition of voice recognition can be suppressed.

  Note that the predetermined range may be not more than the lower limit value of the GMM distribution of FIG. Alternatively, as described above, the upper limit value and the lower limit value of the GMM likelihood distribution may be set, or either of them may be used. When the GMM likelihood determining unit 70 also determines the upper limit value of the GMM likelihood, the acoustic likelihood of the frame exceeding the upper limit value is also substituted for the GMM likelihood. Since the GMM likelihood value is obtained from a GMM having a large distribution including most phonemes, it does not change greatly. Therefore, the likelihood value does not become unstable.

  Although the predetermined range has been described as the range of the upper and lower limits of likelihood corresponding to all learned speech feature quantities, the present invention is not limited to this example. For example, the predetermined range setting provided in the GMM likelihood calculation unit 60 is based on the average value μ and the standard deviation σ of the GMM likelihood obtained in advance by processing the distribution of the GMM likelihood at the time of acoustic model learning as a normal distribution. The means 601 may calculate and set the predetermined range as μ ± 2σ (upper limit = μ + 2σ, lower limit = μ−2σ). In this way, the predetermined range can be set to an arbitrary range from the learned GMM likelihood value. An arbitrary predetermined range may be set in advance based on the average value μ and standard deviation σ of GMM likelihood, and the value may be recorded in the acoustic model parameter memory. In that case, the predetermined range setting means 601 may be omitted.

Further, the upper and lower limit value setting means 701 may be provided in the GMM likelihood determining unit 70, and the GMM likelihood of a frame outside a predetermined range may be set to a predetermined upper and lower limit value. That is, the GMM likelihood may be rounded to the upper and lower limit values. The likelihood range can be further narrowed by rounding.
FIG. 6 shows an operation flow of the speech recognition apparatus 100 ′ that performs the operation of rounding the GMM likelihood outside the predetermined range to the upper and lower limit values. Except for the GMM likelihood determination process (step S70), it is the same as the speech recognition apparatus 100.
The GMM likelihood determining unit 70 ′ determines whether or not the GMM likelihood is within a predetermined range (step S701). If it is within the range (Y in step S701), the operation after step S801 is performed in which the speech likelihood processing is performed by obtaining the acoustic likelihood from the speech feature amount.

If the GMM likelihood is outside the predetermined range, it is determined whether the GMM likelihood is equal to or lower than the lower limit (step S702) or equal to or higher than the upper limit (step S704). The upper and lower limit value setting means 701 sets the GMM likelihood to the lower limit value or the upper limit value based on the determination result and outputs it to the speech recognition processing unit 80 (steps S703 and S705).
Although FIG. 6 illustrates an example in which both upper and lower limits are set to predetermined upper and lower limits, either one of the upper and lower limits may be set.

  FIG. 7 shows a functional configuration example of the acoustic model creation device 200 of the present invention. The operation flow is shown in FIG. The acoustic model creation device 200 includes a feature amount analysis unit 20, a GMM likelihood calculation unit 60, an acoustic model parameter memory 40, a GMM likelihood determination unit 70, a learning processing unit 90, a post-learning acoustic model memory 95, And a control unit 96. The acoustic model creation apparatus 200 is realized by a predetermined program being read into a computer composed of, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

The feature amount analysis unit 20, the GMM likelihood calculation unit 60, and the GMM likelihood determination unit 70 of the acoustic model creation device 200 are the same as those of the speech recognition device 100.
The learning processing unit 90 receives the learning label, the voice feature amount, the GMM likelihood, and the determination result, and associates the voice feature amount with the learning label for a frame in which the GMM likelihood is within a predetermined range, thereby learning the acoustic model. (Step S90). Frames outside the predetermined range are not subject to acoustic model statistic calculation (N in step S70), are discarded as abnormal frames, and the next frame is processed (step S98).

  The above operation is repeated for all frames (N in step S97). The control unit 95 controls the operation and repetitive operation of each unit of the acoustic model creation device 200. When the predetermined range is set based on the average value μ and standard deviation σ of GMM likelihood, the average value μ and standard deviation σ of GMM likelihood updated by learning are stored in the post-learning acoustic model memory 95. To be recorded. In the learning processing unit 90, the predetermined range may be updated in conjunction with the updated average value μ and standard deviation σ, and the value may be recorded in the after-learning acoustic model memory 95. When the predetermined range is set based on the upper and lower limit values of the GMM likelihood, the upper and lower limit values of the GMM likelihood are recorded in the post-learning acoustic model memory 95. In the learning processing unit 90, the predetermined range may be updated in conjunction with the updated upper and lower limit values, and the value may be recorded in the after-learning acoustic model memory 95.

  The acoustic model creation apparatus 200 also includes a GMM likelihood calculation unit 60 and a GMM likelihood determination unit 70, and performs learning of the acoustic model while excluding frames outside the predetermined range, so that it is not affected by sudden noise or the like. An acoustic model can be created. Therefore, it is possible to create a clean acoustic model with high accuracy.

According to the speech recognition apparatus 100 described above, the GMM likelihood obtained from the GMM that includes most phonemes and has the widest variance is substituted for the acoustic likelihood of frames outside the predetermined range. Even if is input, the acoustic likelihood does not change greatly. That is, the acoustic likelihood can be stabilized. Further, since the predetermined range is determined based on the GMM likelihood of the GMM at the time of learning, development data for determining the range is unnecessary.
Moreover, according to the acoustic model creation apparatus 200, an abnormal frame is removed at the time of learning, so that the possibility that an abnormal distribution is generated can be reduced. Therefore, it is possible to create a more accurate acoustic model.

  The method and apparatus of the present invention are not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention. For example, in the GMM, silence data including silence data may be set as a learning target, or only a voice feature amount may be recorded and only a voice section may be set as a learning target. When only the speech section is a learning target, it is possible to directly use the acoustic model that is also used in the acoustic model parameter memory 40 for purposes such as speech section detection of speech recognition preprocessing.

Note that the processes described in the above method and apparatus are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.
Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.
Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (10)

A feature amount analysis unit that analyzes the speech feature amount of the input speech digital signal in units of frames;
A GMM likelihood calculating unit that compares a GMM (Gaussian Mixture Model) with the speech feature and calculates a GMM likelihood for each frame;
A GMM likelihood determination unit that determines whether the GMM likelihood is within a predetermined range and outputs the determination result;
Using the speech feature amount, the GMM likelihood, and the determination result as input, a frame within the predetermined range is subjected to speech recognition processing based on the acoustic likelihood corresponding to the speech feature amount, and out of the predetermined range. A speech recognition processing unit that performs speech recognition processing using the acoustic likelihood using the GMM likelihood,
A speech recognition apparatus comprising: The speech recognition apparatus according to claim 1,
The speech recognition apparatus, wherein the predetermined range is a likelihood distribution range of a GMM (Gaussian Mixture Model) of the learned acoustic model. The speech recognition apparatus according to claim 1,
The GMM likelihood calculator is
A predetermined range setting means for setting the predetermined range by inputting the average value μ of the GMM likelihood and the standard deviation σ of the GMM likelihood;
A speech recognition apparatus comprising: A feature amount analysis unit in which a feature amount analysis unit analyzes a speech feature amount of an input speech digital signal in units of frames;
A GMM likelihood calculation process in which a GMM likelihood calculator calculates a GMM likelihood for each frame by comparing a GMM (Gaussian Mixture Model) with the speech feature amount;
A GMM likelihood determination unit that determines whether the GMM likelihood is within a predetermined range and outputs the determination result;
A speech recognition processing unit receives the speech feature value, the GMM likelihood, and the determination result as input, and performs speech recognition processing based on the acoustic likelihood corresponding to the speech feature value for frames within the predetermined range. A speech recognition process for performing speech recognition processing using the acoustic likelihood using the GMM likelihood for a frame outside the predetermined range;
A speech recognition method comprising: A feature amount analysis unit that analyzes the speech feature amount of the input speech digital signal in units of frames;
A GMM likelihood calculating unit that compares a GMM (Gaussian Mixture Model) with the speech feature and calculates a GMM likelihood for each frame;
A GMM likelihood determination unit that determines whether or not the GMM likelihood is within a predetermined range, and outputs the determination result and the GMM likelihood;
The learning label, the speech feature value, the GMM likelihood, and the determination result are input, and an acoustic model learning process is performed on the frame within the predetermined range based on the speech feature amount, and the frame outside the predetermined range Is a learning processing unit that generates a post-learning acoustic model that is not subject to acoustic model statistic calculation,
An acoustic model creation device comprising: In the acoustic model creation device according to claim 5,
The acoustic model generating apparatus, wherein the predetermined range is a likelihood distribution range of a GMM (Gaussian Mixture Model) of the learned acoustic model. In the acoustic model creation device according to claim 5,
The GMM likelihood calculator is
A predetermined range setting means for setting the predetermined range by inputting the average value μ of the GMM likelihood and the standard deviation σ of the GMM likelihood;
An acoustic model creation device comprising: A feature amount analysis unit in which a feature amount analysis unit analyzes a speech feature amount of an input speech digital signal in units of frames;
A GMM likelihood calculation process in which a GMM (Gaussian Mixture Model) is compared with the speech feature amount to calculate a GMM likelihood for each frame;
A GMM likelihood determination unit that determines whether or not the GMM likelihood is within a predetermined range and outputs the determination result and the GMM likelihood;
The learning processing unit receives the learning label, the voice feature quantity, the GMM likelihood, and the determination result, performs a learning process of the acoustic model based on the voice feature quantity for the frame within the predetermined range, and performs the predetermined process. The learning process for generating a post-learning acoustic model for out-of-range frames is excluded from the calculation of the acoustic model statistics,
An acoustic model creation method comprising:   An apparatus program for causing a computer to function as each apparatus according to any one of claims 1 to 3 or claims 5 to 7.   A computer-readable recording medium on which any of the apparatus programs according to claim 9 is recorded.

Images