EP1010289A2

EP1010289A2 - Defining quality of transmission in a transmission system

Info

Publication number: EP1010289A2
Application number: EP98940288A
Authority: EP
Inventors: Jouni c/o Nemo Techn. INc. SAASTAMOINEN
Original assignee: Nemo Technologies Oy
Current assignee: Nemo Technologies Oy
Priority date: 1997-08-26
Filing date: 1998-08-25
Publication date: 2000-06-21
Also published as: FI105735B; CA2302469A1; AU8865698A; FI973502A0; CN1268269A; FI973502A; NO20000740D0; NO20000740L; WO1999011014A2; AU733335B2; JP2001514460A; WO1999011014A3

Abstract

The invention relates to a method and arrangement for defining the quality of transmission in a transmission system, which arrangement comprises means (504) for transmitting test information that can be recognized over a transmission channel. To achieve a reliable estimate of the transmission quality, the arrangement comprises means (510) for performing a measurement of information revealing whether the information is recognized or not to the received test information, and means (510) for defining the quality of transmission at the time on the basis of the measurements performed over a predetermined period of time.

Description

DEFINING QUALITY OF TRANSMISSION IN A TRANSMISSION SYSTEM

FIELD OF THE INVENTION

The invention relates to a method of defining the quality of transmission in a transmission system, in which method test information that can be recognized is transmitted over a transmission channel.

BACKGROUND OF THE INVENTION

In a radio system, the quality of the connection between a base station and a subscriber terminal varies constantly. This variation is caused by interfering factors occurring on the radio path and radio wave attenuation as a function of distance. For example, when the terminal moves farther away from the base station, the attenuation on the transmission path between the terminal and the base station increases. Efforts are usually made to compensate for this attenuation by power regulation.

A commonly used variable for denoting the quality of a connection in digital cellular radio systems is the bit error rate (BER) which denotes the number of erroneously received bits compared with all received bits. In the GSM cellular radio system, for example, a base station and a terminal continuously measure the bit error rate in the connection between them.

Another way to evaluate connection quality is based on the evalua- tion of the quality of the information transmitted on the connection; in view of speech, for example, how the receiver finds the quality of the speech. The above mentioned interference on the signal path also affect the quality of received voice. A so-called threshold phenomenon, which means that the voice quality stays good for a long period of time when the distance between the base station and the terminal increases and then suddenly radically degrades, is typical for digital systems which allow error correction methods to be used. In analogue systems the quality degrades steadily. Quality evaluation is a useful method when communication networks implemented by different techniques are compared with one another. The bit error rates in different radio systems, for example, are not necessarily comparable, or even accessible (as in analogue systems).

In the quality evaluation of information, for example in the quality evaluation of speech, a signal transmitted to the signal path is usually compared with a signal received over the signal path. A discontinuous signal, for example speech, is commonly used as an audio signal, since the codecs in digital systems are not optimized for a continuous signal, for example a sine wave.

Methods of evaluating quality have been developed. In general, quality is denoted on a scale of one to five (bad, poor, satisfactory, good and excellent) in which the best value is denoted by the value five. A known method is a so-called subjective measurement. In this method a group of test persons evaluates the quality of a received signal within a given procedure. The best known procedure is the so-called mean opinion score (MOS). The recorded speech samples to be evaluated are played to a heterogeneous group of persons who give their opinion about the quality, usually on a scale of one to five, and a mean value is formed from the opinions.

Another known method is based on an objective measurement in which speech quality is defined computationally by means of mathematical models. At its simplest, the signal-to-noise ratio or the above bit error rate is used here to measure speech quality, and in more complex solutions, psy- choacoustically weighted computational methods are used.

Methods have also been disclosed in which the above evaluation manners are combined by using neuronetwork applications which are taught to give a similar response as a heterogeneous group of persons. In general, the aim in all known methods is to evaluate how closely a received test information signal resembles a transmitted one, and the aim is to define the quality of the transmission on the basis of a detected difference.

The prior art solutions have several problems, however. The subjective methods are not practical; it is expensive and time-consuming to em- ploy a group of persons for the evaluation. Furthermore, irrespective of averaging, the results obtained are not necessarily repeatedly similar.

The objective methods, on the other hand, do not correlate closely with the speech quality experienced by persons. The bit error rate, for example, denotes poorly how well speech is recognized. The combinations of the subjective and objective methods, in turn, require heavy computational power. The neuronetwork applications are complex and demanding to implement.

BRIEF DESCRIPTION OF THE INVENTION

It is thus an object of the invention to provide a method and an arrangement implementing the method in such a manner that the above prob- lems can be solved. An object of the invention is to implement a method by which the evaluation of the quality of information can be cost-effectively and reliably implemented. This is achieved by a method of the type described in the introduction, the method being characterized in that the same predetermined symbol is repeatedly transmitted in the transmission of the test informa- tion, and that a measurement of information revealing whether the information is recognized or not is performed to the received test information, and that the measurements are performed over a predetermined period of time, and that the quality of transmission at the time is defined on the basis of the measurements performed during the time. The invention further relates to an arrangement for defining the quality of transmission in a transmission system, which system comprises means for transmitting test information that can be recognized over a transmission channel. The arrangement of the invention is characterized in that the arrangement comprises means for repeatedly transmitting the same predeter- mined test information symbol, means for performing a measurement of information revealing whether the information is recognized or not to the received test information, and means for defining the quality of transmission at the time on the basis of the measurements performed over a predetermined period of time. The preferred embodiments of the invention are disclosed in the dependent claims.

Several advantages can be achieved by the method and arrangement of the invention. The invention enables speech quality, for example, to be evaluated more reliably without expensive equipment and demanding com- putation. The solution of the invention utilizes speech recognition methods and equipment to evaluate speech quality. Similarly, pattern recognition methods are utilized in the quality definition of image information. In the solution of the invention, the aim is not to define how closely a received signal resembles a transmitted one, but only to find out whether the received test information is recognized or not, and the quality of the connection is estimated on the basis of several successive recognition decisions.

The solution of the invention is also suitable to be used in any transmission system in which information is transferred via a transmission path subject to interference. A typical example is a cellular radio system, but the invention can also be applied to transmission systems which employ fixed cabling. BRIEF DESCRIPTION OF THE DRAWINGS

The invention is now described in closer detail in connection with the preferred embodiments with reference to the accompanying drawings, in which Figure 1 is a flow diagram showing an example of the method of the invention,

Figures 2a and 2b are flow diagrams of a definition of a quality grade,

Figure 3 is a flow diagram of changing test information, Figures 4a and 4b illustrate transmission and reception of test information, and

Figure 5 illustrates a structure of the arrangement in accordance with the invention.

DETAILED DESCRIPTION OF THE INVENTION In the following, the invention will be described when it is applied particularly to the evaluation of speech quality. Similarly, the invention can, however, be applied in connection with transmission of other kind of information, image transmission, for example, as it is obvious to those skilled in the art. In such a case the speech recognition algorithms and equipment are re- placed by pattern recognition systems within the basic idea of the invention.

The aim is thus to estimate the quality of speech transmitted over a transmission path. In the solution of the invention, a transmitter repeatedly transmits desired test information to the transmission path. The same predetermined symbol is repeatedly transmitted in the test information transmission. A receiver receives the information and performs a measurement of information revealing whether the information is recognized or not. The measurements can be performed over a predetermined period of time, and the quality of transmission is defined on the basis of the measurements performed during the time. Figure 1 illustrates the operation of the method in accordance with the invention. In a first step 100 a transmitter transmits the desired information which is preferably a particular predetermined word. In a second step 102 a receiver receives the word proceeded over the transmission path. In a third step 104 a recognition is performed which reveals whether the transmitted word is recognized or not. In a subsequent step 106 the information on the transmission quality is updated on the basis of the recognition. Next, a return to the first step 100 follows.

In a preferred embodiment of the invention the desired test information symbol is thus a particular predetermined word which is recorded and stored in advance, and which can thus be repeatedly transmitted unchanged over the transmission path. This particular word is continuously transmitted. The interval between the words can be set as desired in such a manner that successive words can be distinguished from one another. The words are received at the other end of the transmission path. On the transmission path, the signal has met with interference typical of the transmission path, which effect the quality of the signal. A speech recognition which, on the basis of the word, gives a decision value indicating whether the word is recognized or not is performed to the received word. If desired, the received words can be stored in a memory for a later examination or check. The storage can be performed by a predetermined criterion, in such a manner that only the unrecognized words are stored, for example. The words are received over a predetermined period of time, and the quality estimate of transmission at the time is defined on the basis of the decisions made during the time. A typical measurement time can be five seconds, for example. The quality estimate is given on the known scale of one to five.

Speech recognition can be performed by known speech recognition methods and algorithms. A received word is evaluated in speech recognition, and as a result of the evaluation a binary decision whether the word was recognized or not is thus obtained. Since the solution of the invention only aims to recognize a word and not to estimate how near or far the received word is in view of the transmitted word, the solution of the invention is essentially easier to implement in practice than the previous methods.

Figure 2a illustrates a definition of the estimate of the transmission quality, in other words steps 104 and 106 of Figure 1. A recognition test is thus performed to each word in step 200 by using a known speech recognition method. If the word was recognized, the quality estimate is upgraded in step 202. Next, the evaluation of the following word can be introduced in step 204. If the word was not recognized in step 200, the quality estimate is degraded in step 206. Next, the evaluation of the following word can also be introduced in step 204. The update of quality information in steps 202 and 206 can be performed in such a manner that word recognitions performed over a particular determined period of time are taken into account when defining the grade. The transmission quality can be defined on the basis of the words received during the last five seconds, for example, and when a new word arrives the recognition information of the oldest word is removed from the computation. Figure 2b illustrates another embodiment of the definition of the estimate of the transmission quality. It is possible to predict the changes in the quality of the signal more accurately in this embodiment. A recognition test is thus performed to each word in step 200. If the word was recognized, it is next checked in step 208 whether successive recognitions have occurred several times. If not, the quality estimate is upgraded by an increment Λ given in step 210. If several successive recognitions occur, the quality estimate is upgraded by an increment n*Λ given in step 212, which is thus greater than the increment used in step 210. Next, the evaluation of the following word can be introduced in step 204. If the word was not recognized in step 200, it is next checked in step 214 whether successive failures in recognition have occurred several times. If not, the quality estimate is degraded by a decrement Λ given in step 216. If several successive failures in recognition occur, the quality estimate is degraded by a decrement n^*Λ given in step 218, which is thus greater than the decrement used in step 216. Next, the evaluation of the following word can also be introduced in step 204. It is an aim of this method to make the quality estimation react quicker to great and abrupt changes. In a cellular radio system, for example, great changes can take place when a terminal enters or exits a tunnel. The quality of a signal can then suddenly change from good to bad, or vice versa. If several parallel decisions occur, they are detected in the above method, and thus the change unit of the quality information can be increased. The upgradation of the change unit and the number of the required successive similar decisions are system-dependent parameters.

In accordance with a preferred embodiment of the invention, sev- eral words that belong to different levels with respect to recognition can alternatively be used as a test information symbol. The test word used at a particular time can be selected on the basis of the quality information. Let us assume, for example, that three words of different degrees of difficulty with respect to recognition are used. The test can be started with a word of medium degree of difficulty. If it is found that the quality information is continuously good, in other words the percentage of recognition of the word is high, the transmission of a more difficult word can be introduced. Similarly, if a word of a medium degree of difficulty is rarely recognized, in other words the percentage of recognition is low, an easier word can be introduced for testing the connection. The recognition percentages for changing the test word are system- specific parameters and they can vary. This embodiment is illustrated in the flow diagram of Figure 3. A recognition test is performed to a test word in step 300 and at once, the recognition percentage at the time is computed, in other words how many words from the transmitted ones are recognized. If the word was recognized, it is checked in step 302 whether the recognition percentage has reached the word upgradation threshold. If not, a transfer to the next word takes place in step 304. If the threshold is reached, the next most difficult word is selected as the test word in step 306. Next, a transfer to the following word takes place in step 304.

If the word was not recognized in step 300, it is checked in step 308 whether the recognition percentage has reached the word degradation threshold. If not, a transfer to the following word takes place in step 304. If the threshold is reached, the next easiest word is selected as the test word in step 310. Next, a transfer to the following word takes place in step 304. The flow diagram only shows the word changing algorithm, not an update of the quality information, which is performed as shown in Figure 2a or 2b. The word can also be changed directly on the basis of the quality estimate, or on the basis of a parameter computed on the basis of another recognition test.

Let us next take a closer look at an arrangement of the invention for defining the quality of transmission. The transmission and reception of test information in accordance with the invention can be performed in various ways. Examples of different alternatives are shown in Figures 4a and 4b. Figure 4a shows a transmitter 400 and a receiver 404. The transmitter 400 transmits test information 404 to the receiver 402 in which the recognition and quality evaluation in accordance with the invention are performed. Figure 4b shows a so-called loop-back solution, in which a first transceiver 406 transmits test information 404 to a second transceiver 408. The second transceiver 408 transmits received test information 410 back to the first transceiver 406 which receives the signal and performs the recognition and quality evaluation in accordance with the invention. An advantage of the latter solution is that trans- mission and reception can be performed physically in the same place. A disadvantage is that the test information travels twice over the transmission path, which makes it impossible to define which transmission direction includes more interference.

Figure 5 is a flow diagram illustrating an example of the structure of the arrangement in accordance with the invention. The arrangement of the invention for defining the quality of transmission comprises memory means 500 in which previously known test information to be transmitted can be stored, in other words a number of words of different levels with respect to recognition in a preferred embodiment of the invention. The arrangement further comprises control means 502 which read the test information from the memory means 500. The arrangement comprises means 504 for continuously transmitting test information over a transmission channel 506, controlled by the control means 502. The control means 502 are typically implemented by a processor and the transmission means 504, in turn, by means of a modem or a radio transmitter. The arrangement of the invention further comprises means 508 for receiving test information from the transmission channel 506. The reception means 508 typically comprise a radio receiver and a modem. From the modem, the test information is conveyed to processing means 510 which are arranged to perform an information measurement revealing whether the information is recognized or not to the received information by means of speech recognition software. The arrangement further comprises processing means 510 for defining the transmission quality at the time on the basis of the measurements performed during a predetermined period of time. The measurement results are stored in memory means 512. If desired, the received test information, in other words the received words, can be stored in the memory for a later examination or check in the memory means 512. The storage can be performed by a predetermined criterion, for example in such a manner that only unrecognized words are stored.

The arrangement of the invention can be preferably implemented by a microcomputer apparatus which comprises a voice card and software capa- ble of producing voice information stored in a memory, and a modem via which the apparatus is connected to a transceiver in a cellular radio system, for example. The recognition of information can preferably be performed by known speech recognition software. The transmission and reception functions can both be preferably implemented by the same apparatus, or alternatively, two separate apparatuses can be used. In accordance with a preferred embodiment of the invention, in connection with the reception of test information other measurements are also performed, such as field strength or bit error ratio measurements. When the test information results are stored for a later examination, the results of the measurements performed at the same time are also stored. The reception and measurement times are attached to the stored material, whereby results obtained by different methods can be compared with one another.

Although the invention is described above with reference to the example in accordance with the accompanying drawings, it is to be understood that the invention is not restricted thereto but it can be modified in various ways within the scope of the inventive idea disclosed in the appended claims.

Claims

1. A method of defining the quality of transmission in a transmission system, in which method test information that can be recognized is transmitted over a transmission channel, characterized in that the same prede- termined symbol is repeatedly transmitted in the transmission of the test information, and that a measurement of information revealing whether the information is recognized or not is performed to the received test information, and that the measurements are performed over a predetermined period of time, and that the quality of transmission at the time is defined on the basis of the meas- urements performed during the time.

2. A method as claimed in claim 1, characterized in that after the reception of each new test information symbol the transmission quality estimate is updated in such a manner that the recognition information of the oldest symbol is left out in the computation.

3. A method as claimed in claim 1, characterized in that a change parameter of the information symbol is defined on the basis of the information measurement, and that the parameter obtained is compared with a predetermined threshold value, and that if the parameter is smaller than the threshold value the test information symbol is changed to a symbol that is easier to recognize.

4. A method as claimed in claim 1, characterized in that a change parameter of the information symbol is defined on the basis of the information measurement, and that the parameter obtained is compared with a predetermined threshold value, and that if the parameter is greater than the threshold value the test information symbol is changed to a symbol that is more difficult to recognize.

5. A method as claimed in claim 1, characterized in that the received test information is stored in a memory by a predetermined criterion.

6. A method as claimed in any one of claims 1 to 5, charac- t e r i z e d in that speech is transmitted in the transmission system and that the test information is speech information consisting of words and that speech recognition revealing whether each transmitted word is recognized or not is performed to the received words, and that the measurements are performed over a predetermined period of time and the quality of transmission at the time is defined from the number of the words recognized during the time in relation to all words.

7. A method as claimed in any one of claims 1 to 5, characterized in that image is transmitted in the transmission system and that the test information is information comprised of images and that pattern recognition revealing whether each transmitted image is recognized or not is performed to the received images, and that the measurements are performed over a predetermined period of time and the quality of transmission at the time is defined from the number of images recognized during the time in relation to all images.

8. A method as claimed in any one of the preceding claims, characterized in that the transmission system is a cellular radio system.

9. An arrangement for defining the quality of transmission in a transmission system, which system comprises means (504) for transmitting test information that can be recognized over a transmission channel, characterized in that the arrangement comprises means (504) for repeatedly transmitting the same predetermined test information symbol, means (510) for performing a measurement of information revealing whether the information is recognized or not to the received test information, and means (510) for defining the quality of transmission at the time on the basis of the measurements performed over a predetermined period of time.

10. An arrangement as claimed in claim 9, characterized in that the means (504) continuously transmit previously known speech informa- tion over a transmission path and that speech recognition is performed in the means (510).

11. An arrangement as claimed in claim 9, characterized in that the means (504) continuously transmit previously known image information over a transmission path and that pattern recognition is performed in the means (510).

12. An arrangement as claimed in claim 9, characterized in that the arrangement comprises means (502) for changing the test information symbol on the basis of the quality of the estimated signal.

13. An arrangement as claimed in claim 9, characterized in that the arrangement comprises means (512) for storing the received test information by a predetermined criterion.