JP4462766B2 - Universal quality measurement system for multimedia and other signals - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a system for measuring signal quality, and more particularly to a system for measuring the quality of multimedia signals communicated over a network.
[0002]
[Prior art]
A number of processing and transmission methods have been devised for the communication of voice, voice, video and multimedia information. These types of signals are collectively referred to as “Voice, Audio and Video (VAV)” signals. The VAV signal can be processed in many different ways before being sent to the receiver. For example, a VAV signal can be digitized, compressed, and modulated onto a carrier. The receiver must then convert the received signal back into a perceptible representation of the VAV signal.
[0003]
[Problems to be solved by the invention]
Traditionally, the received VAV signal quality is evaluated by subjective tests. However, this type of test is not practical as an in-service test method, and this type of test does not produce consistent and reproducible results. If VAV communication is incorporated into a very complex transmission system such as a cellular code division multiple access (CDMA) system or a wideband CDMA system, the evaluation of signal quality becomes more complex. There are different but equally challenging challenges when evaluating VAV quality in a non-deterministic environment such as the Internet.
[0004]
It is known to indirectly evaluate signal quality by measuring parameters such as signal-to-noise ratio (S / N), carrier-to-interference ratio (C / I), lost packet rate, and bit error rate (BER). It has been. However, it is difficult to correlate these parameters with the user's perception of received VAV signal quality, especially when such signals are highly compressed or processed. Also, these parameters are not well suited to accurately and accurately reflect VAV signal quality over a period of time.
[0005]
Algorithms that estimate perceptual conversation quality are known. For example, International Telecommunication Union standard P.861 is an objective algorithm that automatically calculates qualitative values such as Mean Opinion Score (MOS) for conversational transmission. Can be used to calculate. However, these types of tests require static analysis of both the original signal and the test signal according to the type of test equipment. Thus, if a quality assessment is to be made in the field with real signals, the test equipment must incorporate the network architecture and protocol into its hardware, software or firmware. Often the network architecture is not available or it becomes difficult or expensive to implement. Even monopoly restrictions can make it impossible to do so.
[0006]
In view of the above, a method and apparatus for assessing received VAV signal quality without requiring explicit knowledge of network operation and configuration, such as transmission techniques and network access techniques used, is desired. . It would also be desirable for the method and apparatus to provide automatic real-time or non-real-time measurement of quality, thereby enabling, for example, interactive field testing. In addition, a common test platform for a variety of VAV signals that easily connects to the equipment under test would be beneficial. Furthermore, a method and apparatus that provides reliable, repeatable, easily understandable quantitative quality service measurements for one-way, multi-hop or round trip measurements is desired.
[0007]
[Means for Solving the Problems]
In one embodiment, the present invention is a method for measuring the perceptual quality of voice signals, audio signals, audio and video signals, and multimedia signals in a communication device. The processed test signal is sent from the first device to the second device over the network. The processed signal is then received by the device under test, which further processes the processed test signal to reproduce the representation of the test signal. The reconstructed test signal is then objectively analyzed to determine the degree of perceptual quality by comparing the reconstructed test signal with a pre-stored representation of the test signal.
[0008]
In another embodiment, the present invention is a quality measurement unit that identifies and evaluates the quality of a test signal reproduced with the equipment under test. The test signal is sent from the remote device and sent from the device under test. In the latter case, the quality measurement unit is configured to communicate with the device under test and the network so that the quality measurement unit itself receives a representation of the test signal sent by the device under test.
[0009]
It will be appreciated that in the method and apparatus of the present invention, the quality of the received VAV signal is evaluated without explicit knowledge of the network with which the device under test communicates. In addition, real-time and non-real-time quality analysis is possible and uses a common test platform for various VAV signals. The present invention is applicable to a variety of different quantitative quality service measurements, including one-way, multi-range and round-trip measurements.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
In one embodiment, referring to FIG. 1, a quality measurement system 10 for a VAV signal is shown in the context of a device 16 under test and a network 22. The quality measurement system 10 includes a local quality measurement unit (LQMU) 12. The port 14 of the LQMU 12 is connected to an equipment under test (EUT) 16. In one exemplary embodiment, the EUT 16 is a mobile or cellular telephone and the port 14 is electrically connected to the EUT 16 using, for example, a set of “hands free” or “car kit” electronic terminals (not shown). Is done. However, the present invention is not limited to mobile and cellular telephone applications.
[0011]
The EUT 16 sends information to the network 22 and receives information from the network 22 through the communication paths 18 and 20, respectively. For example, in the case of a cellular telephone, the network 22 includes a wireless network 24 and the communication paths 18 and 20 are radio links. A remote reference data storage unit (RRDSU) 26 is also operatively connected to the network 22. In one exemplary embodiment, RRDSU 26 is a device for recording and playing back a reference signal, such as an answering machine, which is connected to wireless network 24 through wireline network 28. A network interface adapter (NIA) 30 is provided to adjust the signal output from the RRDSU 26 to a form suitable for transmission to the network 28. Similarly, the signal transmitted from the network 28 to the RRDSU 26 is converted by the NIA 30 into a form suitable for the RRDSU 26 to process.
[0012]
In one embodiment, a local storage device in the form of a local reference data storage unit (LRDSU) 32 is provided for the LQMU 12 and is connected to the LQMU 12 via a second port 34. In another embodiment, the local storage device 32 can be an internal device of the LQMU 12. In another embodiment, a local storage device, such as an electrical or magnetic memory, can be an internal device of the LQMU 12. LQMU 12 indicates a test signal quality measurement 36 that EUT 16 received from RRDSU 26 over network 22.
[0013]
Knowledge of the protocol, signaling and network 22 configuration required for the implementation of the present invention is separated within the EUT 16, an existing device and the NIA 30. The NIA 30 is in many cases a readily available device supplied by an operator of the network 28 or an electronics distributor. The present invention allows different types of networks 22 to be accommodated without requiring the RRDSU 26, LQMU 12 or local storage device 32 to know or be able to know detailed information regarding the nature of the network 22. . As a result, the RRDSU 26 can be connected via the NIA 30 to a service access point accessible to the public of the network 22 or to the user.
[0014]
In one embodiment, port 14 and LQMU 12 are configured to collect received test signals from EUT 16 so that LQMU 12 is responsive to signals received by EUT 16. In one embodiment, port 14 and LQMU 12 are configured to control EUT 16 to transmit uplink signals over wireless link 18. The LQMU 12 is also configured to analyze the received test signals and perform objective measurements of these signal qualities. In one embodiment, the algorithms embedded within LQMU 12 are configurable and selectable for one or more types of VAV signals.
[0015]
The RRDSU 26 is a storage unit including, for example, a recording device and a playback device (not shown separately in FIG. 1). The recording device is a readable and writable mass storage device such as a hard disk or other device suitable for recording VAV signals. In one embodiment, the playback device accesses and plays back recorded VAV signals including, for example, a reference VAV signal. The RRDSU 26, in one embodiment, includes a control module (not shown) that is controllable by signals received over the network 22 and commands the RRDSU 26 to start, stop, play, record, Perform tasks such as pause, seek, and timestamp. In one embodiment, RRDSU 26 includes a digital answering machine for performing these functions. In one embodiment, RRDSU 26 is configured to transmit a test signal selected from voice, audio, video and multimedia test signals.
[0016]
In one embodiment, referring to FIG. 2, the quality measurement system 10 is used to test the EUT 16 in the context of a network 22, which network VAV quality from one end to the other end of the unidirectional downlink, That is, it includes a wireless telephone network 24 for evaluating the quality of signals received by the EUT 16 over the radio link 20 from a base station (not shown) of the network 22. Initially, in step 100, LQMU 12 is turned on and connected to EUT 16, which enables LQMU 12 to respond to receipt of test messages from EUT 16. The EUT 16 places a call by dialing the RRDSU telephone number in step 102, and the RRDSU 26 sends a test signal to the EUT 16. In another embodiment, if the EUT 16 is configured to communicate over a network other than the telephone network, a suitable outgoing technology is used instead of placing a telephone call to the RRDSU 26. In another embodiment, such a call can also be made manually, for example by dialing a number on the keypad (not shown) of the EUT 16.
[0017]
In one embodiment, RRDSU 26 is configured as a voice mailbox, so dialing RRDSU at step 102 also includes dialing the voice mailbox of RRDSU 26 and eliciting a test signal as a voice mail message at step 104. Prior to transmission of a test signal, such as a voice message, the test signal is processed by the NIA 30 so that the test signal is in a suitable form for transmission over the network 28. Next, LQMU 12 attempts to synchronize itself to the test signal at step 106. The synchronization search at step 106 continues until synchronization is achieved at step 108. In one embodiment, if the synchronization of step 108 is not achieved within the determined time (step 110), an error condition is signaled at step 112. In one embodiment, in synchronization with the test signal at step 108, the LQMU 12 analyzes the signal received by the EUT 16 and determines whether a test signal has been received. The EUT 16 performs further processing of signals received over the network 24 to regenerate the representation of the test signal analyzed by the EUT 16. This further processing, among other things, allows the EUT 16 to be independent of the signal coding and network protocol used by the network 22.
[0018]
After obtaining synchronization in step 108, the regenerated test signal is received by the LQMU 12 in step 114. The reconstructed test signal is compared to a pre-stored representation of the reference signal in LRDSU 32 and a quality of service (QoS) measurement is determined at step 116 to provide an objective measurement of received signal quality. This measurement or its representation is displayed at step 118, for example on a display screen (not shown). In another embodiment, quality measurements are recorded for future display or analysis. In one embodiment, a predetermined number of tests are performed in step 120 during a single call to RRDSU 26. The call from EUT 16 to RRDSU 26 can be terminated either manually or automatically when the determined number of tests has been completed.
[0019]
In one embodiment, when the EUT 16 is receiving a test signal, if the LQMU 12 does not have sufficient processing power to evaluate the quality of the received test signal, the received test signal is For example, it is stored in the LRDSU 32 for subsequent offline quality evaluation, such as after a call from the EUT 16 to the RRDSU 26 is terminated.
[0020]
In yet another embodiment, referring to FIG. 3, the uplink test system 200 does not require the RRDSU 26. Test system 200 is shown in FIG. 3 in the context of network 22 and EUT 16. In this embodiment, the EUT 16 is a cellular telephone that transmits to the wireless network 24 via the wireless link 18. The LQMU 12 receives a signal from the NIA 30 through the wireline network 28. In one embodiment, the NIA is a modem, such as a computer modem, and the LQMU 12 is configured with that modem for communication. The EUT 16 starts the test by dialing the NIA 30 telephone number. The LQMU 12 then inputs a local-to-remote test signal to the EUT 16 using, for example, a hands-free kit, and the signal is returned to the LQMU 12 via the network 22 and the NIA 30. Next, the LQMU 12 performs service quality evaluation on the received signal.
[0021]
In another embodiment, referring to FIG. 4, a two-way test system 300 is provided, which includes a master quality measurement unit (MQMU) 302 and a slave quality measurement unit (SQMU) 304. The configuration of the two-way test system 300 is illustrated, and the EUTs 306 and 308 are tested in relation to the network 310. The MQMU 302 initiates and coordinates the communication between the EUT 306 and the EUT 308, for example terminals providing VAV capabilities, via a network 310 such as the Internet, and provides time stamps for test results produced by both the MQMU 302 and the SQMU 304. Configured to do. Otherwise, MQMU 302 and SQMU 304 are configured similarly to LQMU 12, MQMU 302 is connected to EUT 306 and reference data storage unit (RDSU) 312, and SQMU 304 is connected to EUT 308 and RDSU 314. Both MQMU 302 and SQMU 304 are configured to provide quality measurements of received signals. MQMU 302 provides quality measurements 316 of signals transmitted from EUT 308 through network 310 to EUT 306, and SQMU 304 provides quality measurements 318 of signals transmitted from EUT 306 through network 310 to EUT 308. In one embodiment, the quality measurement timestamps are determined so that the quality measurements 316 and 318 can be later correlated and post-processed by the post-processing module 320.
[0022]
In one embodiment, none of LQMU 12, RRDSU 26, MQMU 302 and SQMU 304 require a direct connection to any network. For this reason, each of these devices can operate regardless of the type of network, including network characteristics and configuration, and can be used in relation to both owned and unowned networks It is. As a result, the present invention can be implemented in any of a variety of different networks including switched lines, packet switched, frame-relay, internet protocol and asynchronous transfer mode (ATM) networks. The present invention is also suitable for various network transmission technologies including wireless, wired and satellite networks. Furthermore, the components of the present invention can be designed to connect to existing terminals or ports of communication equipment as well as to service access points available to the public or user of the network. Thus, no special setup or modification of the network or equipment under test is required. In various embodiments, some or all of LQMU 12, MQMU 302 and SQMU 304 are non-intrusive to allow for operational deployment.
[0023]
In one embodiment, at least one of LQMU 12, MQMU 302, and SQMU 304 is performed utilizing digital signal processing to achieve real-time quality assessment. If network architecture or protocol information is available, the obtained quality measurements are used to adaptively adjust network parameters.
[0024]
In other embodiments, voice quality assessment is improved, for example, by adaptively or automatically selecting an appropriate algorithm for quality assessment. For example, a multimedia signal transmitted through CDMA and Voice Over IP (Voice Over IP) is different from the standard quality measurement algorithm, for example, perceptual speech quality measurement (PSQM) of International Telecommunication Union standard P.861. ) Or a standard quality measurement algorithm such as a measurement normalization block (MNB) is better evaluated by an algorithm. Another measurement algorithm developed by the ITU for measuring voice quality offline uses files stored on a computer. In one embodiment, at least one of LQMU 12, MQMU 302, and SQMU 304 is programmable for different system applications. In one embodiment, various test signals are selected either manually or under the control of at least one of LQMU 12, MQMU 302 and SQMU 304. For example, a signal similar to an artificial voice designed by a standard creation body such as ITU can be selected.
[0025]
In one embodiment, the storage devices 32, 310, and 312 include a relay device that communicates with any of several different types of storage media or another device having a storage medium.
[0026]
In one embodiment, at least one of LQMU 12, MQMU 302, and SQMU 304 is functionally coupled with at least one of a global positioning satellite (GPS) receiver and a digital map (not shown) to provide a quality of service map. Generate automatically. For example, test signals for a radiotelephone network are repeatedly transmitted to the moving EUT 16, 306 or 308, and performance measurements are performed and analyzed as a function of location.
[0027]
It will be apparent to those skilled in the art that many other modifications can be made within the spirit of the invention. Accordingly, the scope of the invention should be determined by reference to the claims and their equivalents.
[0028]
【The invention's effect】
In accordance with the present invention, a method and apparatus is provided for evaluating received VAV signal quality without requiring explicit knowledge of network operation and configuration, such as transmission techniques and network access techniques.
[Brief description of the drawings]
FIG. 1 is a block diagram of an embodiment of a quality measurement system for a VAV signal suitable for one-way quality measurement according to the present invention.
FIG. 2 is a flowchart of an embodiment of a quality measurement method applicable to the system of FIG.
FIG. 3 is a block diagram of an embodiment of a quality measurement system suitable for round trip quality measurement according to the present invention.
FIG. 4 is a block diagram of an embodiment of a quality measurement system suitable for two-way quality measurement according to the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Quality measurement system 12 ... Local quality measurement unit 14, 34 ... Port 16 ... Device under test 18, 20 ... Communication path 22, 24, 28 ... Network 26 ... Remote reference data storage unit 30 ... Network interface adapter

Claims (27)

In an operable environment, the perceptual quality of at least one of a voice signal, an audio signal, an audio and video signal, and a multimedia signal in a communication device (for audio and video at the time of actual expression by the signal) A quality evaluation method, the same shall apply hereinafter)
Transmitting a test signal processed by the RRDSU from a remote reference data storage unit (RRDSU) at a remote location through a communication network;
The test signal is one of a voice signal, an audio signal, an audio and video signal, and a multimedia signal,
Receiving the processed test signal using one of the devices under test (EUT);
The EUT is operably coupled to the RRDSU via a network, and the EUT is connected to the network to receive the test signal;
The EUT sends one signal to the RRDSU for transmission of the test signal and to the network for reception of the test signal,
Said EUT processing said processed test signal to reproduce a representation of said test signal within EUT ;
Using said EUT, a step of transmitting a representation of the test signal to the RRDSU,
Further processing the representation of the test signal using the RRDSU;
The RRDSU transmits a representation of the processed test signal to the equipment under test (EUT) over a network;
Analyzing the quality of the representation of the reproduced test signal as perceptual quality by comparing the reproduced test signal with a pre-stored representation of the test signal by a quality measurement unit . The RRDSU transmits a processed test signal over an operably connected network;
The EUT is operably connected to the network, connected to a quality measurement unit (QMU) for receiving the processed test signal and analyzing the quality of the regenerated test signal,
The method of claim 1, further comprising analyzing the signal reproduced by the EUT to determine that the test signal is being reproduced prior to the step of analyzing the quality of the reproduced test signal. Method.   The method of claim 1, wherein the QMU comprises a step for displaying a quality of the reproduced test signal.   The method of claim 2, further comprising contacting the RRDSU from the EUT via the network prior to transmitting the processed signal.   5. The method of claim 4, wherein the network comprises a network of telephone lines, the EUT comprises a telephone, and through the network contacting the RUTSU from the EUT comprises dialing a RRDSU telephone number.   6. The method of claim 5, wherein contacting the RRDSU from the EUT via the network includes dialing an access code for the RRDSU voice mailbox.   The method of claim 2, wherein the step of analyzing the quality of the regenerated signal is performed while the processed test signal is received by an EUT.   The method of claim 2, further comprising the step of storing a regenerated test signal, wherein the step of analyzing the quality of the regenerated test signal is performed after the test signal is stored.   9. The method of claim 8, wherein the step of analyzing the quality of the regenerated signal is performed after disconnecting the EUT from the network.   The method of claim 2, wherein the quality analysis of the regenerated test signal is performed digitally.   The method according to claim 2, further comprising selecting a quality evaluation method according to the characteristics of the network. The network is a wireless telephone network, and the steps of communicating the processed test signal and analyzing the quality of the regenerated signal are performed iteratively;
further,
Moving the EUT so that the repeated communication of the processed test signal occurs at a different location;
The method of claim 2, wherein the step of analyzing the quality of the reconstructed test signal is performed as a function of different positions of the EUT. The processed test signal communicated from the RRDSU to the device under test is a representation representing the processed signal of the first test signal;
further,
Using QMU, input representation of second test signal towards EUT,
And processing the second test signal using an EUT;
Communicating the processed second test signal from the EUT to the first device over a network;
Further, processing the communicated and processed second test signal for reproduction as a representation of the second test signal using a first device;
Analyzing the perceptual quality of the reproduced second signal by comparing the reproduced second test signal with a pre-stored representation of the second test signal.
The method of claim 2. Separately storing the quality analysis results of the reproduced first test signal and the reproduced second test signal;
Performing each separately stored result in association with a timestamp for later correlation;
14. The method of claim 13, further comprising processing the separately stored results.   The method according to claim 1, wherein an EUT is used instead of the RRDSU, and a quality measurement unit (QMU) is used instead of the EUT. A quality measuring device for measuring the quality of a signal transmitted over a communication network to an equipment to be tested (EUT),
The quality measuring device has a local reference data storage unit (LRDSU);
The LRDSU has a stored representation of the test signal and is configured to control the EUT to communicate the processed representation of the test signal over a network;
The quality measuring device includes:
Communicate with the EUT,
Analyzing to identify signals received and played back by the EUT,
Analyzing the perceived quality of the identified and regenerated test signal while the processed test signal is received by the EUT;
Receiving the representation of the test signal communicated by the EUT under the control of the quality measuring device through the network and reproducing the perceptual quality analysis of the test signal communicated to the quality measuring device through the network An apparatus adapted to perform a test signal by comparing it with a pre-stored representation of the test signal .   17. Quality according to claim 16, configured to analyze at least one signal selected from the group of voice signals, audio signals, video signals or multimedia signals as one type of test signal. measuring device.   The quality measuring device according to claim 17, wherein the quality measuring device is configured to display an indicator indicating the quality of the identified and reproduced signal.   18. A quality measurement device according to claim 17, wherein the quality measurement device is configured to be tuned with a signal reproduced by an EUT to identify the reproduced test signal. The quality measuring device has a local storage device;
18. The apparatus of claim 17, further configured to store the identified and reproduced test signal in the local storage device and analyze the identified and reproduced test signal after being stored. Quality measuring device.   21. The quality measurement device of claim 20, wherein the quality measurement device is configured to disconnect an EUT from a network prior to analyzing the identified and regenerated test signal.   18. The method of claim 17, wherein the identified and regenerated signal is configured to select a best evaluation method according to network characteristics to analyze the perceptual quality according to network characteristics. Equipment.   The quality measuring device according to claim 16, wherein the quality measuring device is configured to communicate by radio telephone as an EUT.   17. A quality measuring device according to claim 16, configured to record the perceptual quality indicator of the reproduced and identified test signal and to record a time stamp on the recorded perceptual quality indicator. . A system for measuring at least one perceptual quality of an audio signal, an audio signal, an audio video signal and a multimedia signal in a communication device in an operable environment,
A transmitting device configured to communicate a processed test signal through a communication network, wherein the test signal is at least one of an audio signal, an audio signal, an audio video signal, and a multimedia signal;
A receiver configured to process the processed test signal to receive the processed test signal over a network and regenerate the representation of the test signal;
Quality configured to analyze the perceptual quality of the regenerated test signal by comparing the regenerated test signal operably connected to the receiver with a pre-stored representation of the test signal A system comprising a measurement unit (QMU). The transmitting device further comprises a remote data storage unit (RRDSU);
The RRDSU transmits the processed test signal through an operably connected network and determines that one test signal has been received prior to the QMU analyzing the quality of the regenerated test signal. 26. The system of claim 25, configured to:   27. The system of claim 26, wherein the QMU comprises a display device configured to display a quality indicator of a received test signal.

Images